Compare commits

...
Author SHA1 Message Date
Quick fe934b59d3 feat(audiobooks): add first-party audio playback API 2026-06-08 17:32:26 -04:00
9e29e7b330 feat(security): encrypt server-owned credentials at rest (#45) (#95)
* feat(security): encrypt server-owned credentials at rest

Introduce AES-256-GCM at-rest encryption (HKDF-derived from a required
SECRET_KEY) for server-owned credentials, with row-bound AAD, a versioned
enc:v1: envelope, and an idempotent startup backfill.

- internal/secret: cipher + RowAAD/SettingsAAD + the startup backfill engine.
- SECRET_KEY required at bootstrap; cipher threaded as an explicit dependency.
- server_settings: EncryptedSettingsRepo decorator over the audited
  SensitiveSettingKeys (also drives admin redaction); the config watcher and
  watch-sync settings reads decrypt too.
- Arr keys inline-encrypted; the ambiguous SecretResolver indirection removed
  from requests/autoscan.
- Per-table columns encrypted: subtitles, watch-sync, webhook-sync (not
  webhook_secret), history-import, and the jellycompat session's bridged Silo
  access/refresh tokens.
- Startup backfill (resolve-then-encrypt for arr refs) is best-effort and
  primary-node gated.

Equality-looked-up secrets and plugin_runtime_configs.config_value are out of
scope (need hashing / cross-repo design) — see
docs/architecture/secret-encryption.md.

Refs #45

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(compose): require SECRET_KEY in docker-compose

The server now fatals without SECRET_KEY, so the integrated service (and the
commented distributed proxy/transcode examples) pass it through with a
fail-fast guard matching the existing MEDIA_ROOT pattern. Distributed worker
nodes must use the SAME key as the primary to decrypt shared data.
Generate with: openssl rand -base64 48.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): encrypt history import session credentials

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:25:48 -04:00
d3626fca00 feat(jellycompat): add /Users endpoint and unified sa_ admin-key auth (#91)
* fix(api): allow HEAD on /api/v1/direct-download

Firefox (and some download managers) issue a HEAD request before
starting a download. The route only registered GET, so HEAD returned
405 Method Not Allowed and the browser aborted the download.

Mirrors the pattern already used by /stream/{session_id}, which
registers both GET and HEAD on the same handler. ServeDirect is built
on http.ServeContent / ServeFile, which natively handle HEAD by
writing headers without a body, so no handler changes are needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: design jellyfin autoscan scan compatibility

* refactor(scan): extract scan trigger resolver

* refactor(api): share scan target resolution

* feat(jellycompat): accept admin api keys for autoscan

* feat(jellycompat): add autoscan media update route

* docs: document jellyfin autoscan setup

* fix(jellycompat): harden autoscan auth and batch scan enqueue

- Reject nil API keys and bound last-used update with a 5s timeout
- Stop leaking internal queue errors in autoscan responses
- Batch scan enqueues via new CreateBatch and reuse folder list across path resolves

* chore: add planning docs and requests updates

- Add plans for date-named episodes and Jellyfin autoscan compat
- Update requests handlers, service, and UI hooks
- Remove Makefile.local.example

* refactor(scantrigger): drop redundant Target.LibraryID field

- Read library ID from Target.Folder.ID everywhere
- Guard scan queue enqueue against nil Folder
- Simplify admin API key auth error plumbing

* fix(catalog): gate search overview-only matches behind title FTS

- Always apply stats CTE + CROSS JOIN so single-word queries no longer flood results with description-only hits
- Require overview_rank >= 0.15 for overview-only fallback rows
- Switch title gate from contiguous LIKE to title_rank > 0 so reordered-token title matches aren't demoted

* ci(docker): build image on push to main via self-hosted runner

- Trigger Docker image builds on pushes to main instead of nightly cron
- Run on self-hosted Linux runner
- Drop the `nightly` tag

* docs(specs): add design for TMDB-backed request section in search

Adds the design for surfacing requestable TMDB results inside the main
catalog search (Cmd+K dialog and full results page) as a clearly
delimited "Request to Add" section that never blocks or displaces
library results.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(specs): address Codex adversarial review for search request section

Splits discovery eligibility from submission eligibility so blocked
and quota-exhausted viewers still see the requestable section with
disabled per-row CTAs, matching the documented behavior. Documents
the required extensions to useRequestSearch — signal forwarding,
viewer-identity-keyed cache, and invalidation on auth/profile/
settings/limit changes — so the planned 5-minute staleTime is
safe and cancellation works as described.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(plans): add implementation plan for search request section

Twelve TDD tasks covering: api() signal contract test, useCanRequest
hook, viewer-keyed requestKeys.search, useRequestSearch extension
(signal + viewer key + 5min staleTime + enabled override), invalidation
cascade tests, RequestPosterCard optional onRequest, RequestToAddSection
component (dialog + grid variants), GlobalSearch and Catalog wiring
with empty-state suppression for the library-0/TMDB-pending edge case,
final lint/test pass, and manual smoke. Notes a single deviation from
the spec: submitDisabledReason is null in the initial implementation,
with per-row request data driving disabled UI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(plans): address Codex adversarial review for search request section

Fixes the high-severity finding that RequestToAddSection's internal
useRequestSearch call was not gated on discoveryEnabled, allowing
/api/v1/requests/search and TMDB lookups to fire for users without
request access. The plan now (1) passes { enabled: discoveryEnabled }
to the section's hook, (2) gates the parent mount in GlobalSearch and
Catalog on canRequest.discoveryEnabled as defense in depth, and (3)
adds tests asserting both the enabled forwarding and the no-mount
behavior when discovery is disabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(api): pin AbortSignal forwarding contract on api()

* feat(hooks): add useCanRequest gating hook for discovery eligibility

* refactor(keys): add viewerKey to requestKeys.search

* feat(requests): key useRequestSearch by viewer, forward signal, raise staleTime

* test(requests): document viewer-keyed cache isolation and invalidation cascade

* feat(request-card): make onRequest optional on discover variant

* feat(search): add RequestToAddSection dialog variant

* feat(search): add RequestToAddSection grid variant for Catalog page

* feat(search): render RequestToAddSection in the Cmd+K dialog with empty-state suppression

* feat(catalog): render RequestToAddSection grid with empty-state suppression

* chore(web): format request search section

* test(web): avoid unsupported Array.at in search request tests

* commit message

{"subject":"fix(search): prevent empty-state flash before TMDB fallback renders","body":"- Add isResolving to useCanRequest and gate empty states on it across GlobalSearch and Catalog\n- Debounce TMDB query in Catalog and hide ItemGrid when the request section may rescue an empty library\n- Track per-card submit state in RequestToAddSection grid so concurrent requests don't trample each other\n- Suppress anonymous TMDB request-search fetches to avoid cross-viewer cache leakage"}

* fix(jellycompat): tolerate autoscan sidecar updates

* fix(webhooksync): skip events for unmapped external users

- Require explicit profile mapping instead of falling back to the default profile
- Update settings UI copy to reflect that unmapped users are ignored

* refactor(admin): show per-section loading and error states

- Replace page-level loading gate with skeletons per section on dashboard and stats
- Surface query errors inline instead of blocking the whole page
- Disable "Scan All Libraries" when no libraries are configured

* fix(search): address request search review feedback

* feat(subtitles): restore upload management

* feat(auth): add assignable user permissions

* feat(auth): expose user permissions

* feat(api): authorize item metadata curation

* feat(api): route metadata curation by permission

* feat(web): add permission helpers

* feat(web): assign metadata curation permission

* fix(web): keep device profile hooks unconditional

* feat(web): show metadata tools to curators

* fix(auth): address metadata curation review issues

* fix(auth): tighten curator job response review fixes

* docs: add metadata curation permission plan

* test(auth): expand session revocation coverage

* docs: add PageBack component design spec

* fix(auth): gate media file paths on metadata curation permission

- Allow curators (not just admins) to view media file paths and locations
- Apply library access filter to file-level access checks

* fix(metadata): break duplicate provider candidate ties

- Score candidate metadata completeness and auto-match the richer duplicate when title/year/type tie
- Enrich near-duplicate candidates via the provider chain before initial match selection
- Seed both movie and series match queues for mixed-type libraries and wait for TV queue settle
- Add taskmanager worker test coverage and a plan doc for the tie-breaker work

* feat(ui): add shared PageBack component for consistent back navigation

Replace the eight inconsistent back affordances across user-facing pages
with a single absolute-positioned chevron pill, so the control lives in
the same screen position regardless of title length or hero content.

DetailBreadcrumb keeps its textual hierarchy path but no longer owns the
back chevron; PageBack does. DetailHero gains a topNav slot consumed by
Movie/Series/Season/Episode/Request detail pages. Non-hero pages drop
their bespoke back buttons and add PageBack inside a relative wrapper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(ui): add floating variant to PageBack for sticky nav

- Add `floating` prop to pin PageBack to viewport on lg+ screens
- Switch styling from glass-subtle to glass with shadow for better contrast
- Use floating variant on SettingsLayout

* fix(ui): make PageBack destinations deterministic

* feat(episode-carousel): highlight currently viewed episode

- Add "Now Viewing" badge with pulsing indicator on current episode
- Replace border with primary-color ring for current episode card
- Set aria-current="page" on links to the current episode

* feat(jellycompat): sign image tags and accept them without session

- HMAC-sign image tags using the configured JWT secret
- Serve item/season/episode images via signed tag without requiring a session or cache hit

* fix(jellycompat): harden signed image tags

* fix(jellycompat): stabilize signed image tags across restarts

{"subject":"fix(jellycompat): stabilize signed image tags across restarts","body":"- Sign library poster and episode parent series image tags from canonical paths/thumbhashes instead of presigned URLs so tags survive restarts\n- Accept signed canonical tags in the image handler without a session and fall back to legacy URL-derived cache tags\n- Always fetch series detail for episodes to build stable parent image tags"}

* fix(metadata): accept exact cross-provider match ties

* Optimize episode added_at sorting

* fix(libraryingest): treat drainer shutdown cancel as clean stop

TV/series full scans (libraries with new or updated items) were recorded as
"cancelled" with an empty error message and never completed matching.

When the file-walk finishes, the ingest executor waits out a settle window and
then calls stopDrainers() to shut down the concurrent match goroutines. That
cancels the drainer context while a ProcessBatchByFolderAndPathPrefix call may
still be in flight. The drainer treated the resulting context.Canceled as a
fatal error: it pushed the error to drainerErrCh and called cancel() on the
whole scan context, so scanqueue.process() mapped it to cancelRun().

Large/slow libraries (many series, slow provider lookups) keep a batch in
flight continuously, so stopDrainers() almost always landed mid-call and the
scan was cancelled; small/fast libraries were usually idle at that instant and
completed normally.

Treat a cancelled drainer context as a deliberate shutdown: return cleanly
without escalating. Genuine external cancellation still reaches the run via the
main goroutine's scanCtx checks, so real cancels are not swallowed.

Adds a regression test (settle window made injectable) that fails against the
old handler with 'concurrent match scope ...: context canceled' and passes
with the fix.

* perf(catalog): add episode browse index fast path

* fix(catalog): address episode catalog review feedback

* Fix settle-window drainer cancellation in library ingest

* fix(catalog): support relative date filters

* fix(collections): cap smart collection results

* fix(library): preserve episode browse url

* fix(catalog): use season posters for episode cards

* feat(sections): show episode context in cards

* feat(calendar): show local episode airtimes

* Enable DRI passthrough in docker compose

Co-authored-by: Codex <noreply@openai.com>

* feat(ui): paginate the Ambiguous Roots table

Match the sibling tables on the Admin Libraries page (Troubleshooting/unmatched):
use the existing usePagination hook + PaginationBar (10/page, auto-hidden when
<=10 rows), render pag.rows, and reset to page 0 when the library selector or
search filter changes.

* feat(admin): enlarge match-candidate posters + hover-to-enlarge

Unmatched-item match dialog rendered candidate posters at 44x64px, too small
to identify a film. Bump to 64x96 (2:3) and add a portaled Tooltip hover preview
(192x288) using the existing poster image, so operators can tell candidates apart.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(admin): search unmatched items across the whole table, not just the page

The unmatched-items search filtered only the current page's rows client-side.
Push the query server-side: HandleListUnmatchedItems takes an optional 'q' param
and filters title/library/type/status with parameterized ILIKE across all rows,
paginating the filtered set. Frontend hook takes a debounced search, resets to
page 1 on change, keeps the section mounted while searching. Also fixes stale
test mocks that returned the pre-pagination array shape instead of {items,total}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(admin): prettier-format match dialog; reset unmatched page in onChange

Run prettier over the Tooltip-wrapped poster JSX, and reset the unmatched-items
page in the search input's onChange rather than a useEffect (avoids the
react-hooks/set-state-in-effect warning / cascading renders).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(admin): wire QueryClientProvider + missing hook mocks so the suite runs

The AdminLibraries test file failed all 6 tests with 'No QueryClient set' on
this branch and at the parent commit -- pre-existing infrastructure gap. With
that fixed, several hooks that the page imports (useCancelLibraryScans,
useLibraryRoots, useUpsertLibraryRootOverride, useDeleteLibraryRootOverride,
useActiveScans) and the UNMATCHED_PAGE_SIZE constant also needed mocking. One
stale assertion on the renamed 'Root path' header is updated; the deeper
troubleshooting test, which mocked useSkippedLibraryRoots but the section was
refactored to useLibraryRoots(_, 'ambiguous'), is skipped with a TODO -- a real
rewrite is needed and is out of scope for this MR.

5 of 6 tests now run and pass; the 6th is properly flagged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(admin): search all unmatched item library memberships

* fix(naming): strip unsubstituted Sonarr tokens ({TvdbId}/{imdb-}) from titles

These tokens survived the provider-tag regex ([\w]+ doesn't match braces),
polluting parsed titles (e.g. 'A Girl & Her Guard Dog [tvdb-{TvdbId}]') so
they could not score-match. Broaden the regex to drop {...} and empty tokens.

* fix(naming): numeric-only titles are not bare provider IDs

'86' / '22 7' were parsed as trailing tvdb ids, tripping the trusted-ID gate
so the correct title match was rejected. Require a letter in the name before
treating a trailing number as a bare id.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(naming): document bare-id trade-off; cover CJK title + movies numeric

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): auto-accept a year-corroborated single distinct show

A search that resolves to one distinct show (one candidate, or the same
title+year returned once per source as unmerged TVDB/TMDB rows) whose year
matches the parsed year is now auto-accepted via the existing top-ranked
candidate, even when the fuzzy title score is in the 55-69 band. The 55/70/15
thresholds are unchanged; this only adds a year-gated acceptance for
effectively-unique results (recovers lone-correct-result items like 1201 (1993)).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(matcher): exercise the single-distinct-show guard properly + conflicting-ID case; doc notes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(matcher): tolerate concurrent-merge ErrItemNotFound in series episode-link ensure

A scan drainer and the background MatchWorker can process the same folder
concurrently. When a provider-ID merge moves a series' episodes to the survivor
and deletes the source, an in-flight ensureSeriesEpisodeLinks(sourceID) hits
catalog.ErrItemNotFound and was failing the whole scan. The episodes are already
reattached, so this is benign: log and continue (matching the lenient call sites)
instead of failing. Genuine errors still abort.

* diag(matcher): debug-log per-candidate match scores

Adds a DEBUG-gated log in selectInitialMatchCandidate printing each scored
candidate (title/year/type/sources/provider_ids/score) against the hint, so
operators can see why an item did or didn't auto-match. Zero-cost when debug
logging is off; no change to matching behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): resolve cross-source ties by library provider priority

Accept a year-corroborated single distinct show when the TOP tie-group (within
15 pts of best) is one show across providers, ignoring low-score noise below it,
and pick the winner by the library's metadata-provider chain order (providerPriority,
highest-first; falls back to top-scored). Recovers items like '100 Days Wild'
that are returned identically by TVDB and TMDB. Thresholds (55/70/15) unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): accept cross-source-corroborated ties without a hint year

When the top tie-group is one distinct show returned by 2+ distinct providers
(candidatesAreSingleDistinctShow already verifies matching title+year), accept it
even if the hint has no parsed year (year-less folders like '100 Deeds for Eddie
McDowd'). Multi-source agreement substitutes for the year guard; lone single-source
no-year results stay subject to the single-candidate >=70 gate. Thresholds unchanged.

* diag(matcher): debug-log provider search query + per-provider result counts

Adds DEBUG logs in the ModeInitialMatch search path: each provider's result
count for the query, and the assembled raw/candidate totals. Lets us see when a
provider search returns zero ('no metadata found') vs a scoring/tie issue.
Zero behavior change.

* fix(naming): parse bare bracketed IMDb IDs ([tt10011226]/{tt...})

Folders tagged with a bare IMDb id in brackets (Plex/Kodi style, e.g.
'17 Blocks (2021) [tt10011226]') had the id silently dropped — folderIDPattern
needs an 'imdb-' prefix and trailingImdbIDPattern needs an un-bracketed trailing
tt-id. Recognize bracketed bare tt-ids so these items get the trusted-ID match
path instead of falling to title+year scoring.

* feat(metadata): match sole exact-title candidate despite year off by <=2

Folder years routinely differ from provider release years by a year or two
(festival vs wide release, regional dates), zeroing the year bonus and leaving
a lone exact-title candidate at 63-68 — just under the single-candidate >=70
gate (e.g. Dead Reckoning 1947 vs 1946, 17 Blocks 2021 vs 2019, Stasi FC). Add
title corroboration to the existing lone-result rule: a sole distinct show whose
normalized title exactly matches and whose year is within +/-2 is accepted. The
55 floor still rejects low-similarity titles (e.g. Hotel Transylvania Puppy! vs
Puppy!). No 55/70/15 threshold change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(lint): gofmt single-space alignment in root_inference.go var block

When inferProviderTagRe was broadened to handle unsubstituted Sonarr token
placeholders ({TvdbId}/{imdb-}), the regex grew long enough that gofmt prefers
single-space rather than column-aligned spacing across the var block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metadata): tighten matcher and bare IMDb parsing

* docs: add resilient library deletion design spec

Batched, deadlock-retrying rewrite of delete_library to replace the
single multi-minute transaction that deadlocks on large libraries.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs: add implementation plan for resilient library deletion

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(catalog): add deadlock-retry helper for batched deletes

* test(catalog): clarify cancel-path expectation in retry test

* feat(catalog): add deleteInBatches loop helper

* refactor(catalog): make image-dir helpers querier-agnostic

Add rowQuerier interface satisfied by both *pgxpool.Pool and pgx.Tx.
Split collectImageDirs into collectRawImageDirs (raw collection) and a
thin wrapper that filters via filterUnreferencedImageDirs. Both helpers
now accept rowQuerier so a later task can call them from pool-level
batch deletes without an open transaction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(catalog): delete libraries in deadlock-retrying batches

Replaces the single multi-minute delete transaction with phased, batched
autocommit deletes (orphan items, media files, memberships, folder row),
each retried on deadlock. Holds only short locks, survives concurrent
writers, and is resumable on failure.

* refactor(catalog): wrap orphan-batch iteration error

* fix(catalog): clamp still/poster/logo backdrops to largest cached variant

Episode stills used as backdrops only exist at w500/w300 in the cache, so
requesting a w1280/w1920 backdrop width 404s. Add catalog.BackdropVariantPath
+ imageTypeFromCachedPath and route featured (w1920) and Continue Watching /
Next Up (w1280) backdrops through it; still/poster/logo paths clamp to their
type's largest cached variant while real backdrops keep the requested width.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* perf(startup): defer non-critical init off the HTTP listener path

Collect catalog-size-dependent seeding (metadata match queues, legacy
series-group cleanup) and the watch-provider scrobble sweep into a
backgroundInit slice that runs sequentially in a background goroutine after
the server is ready, instead of blocking startup before the listener accepts
connections. Steps log failures and stop early on shutdown.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(migrations): make air_timezone column add idempotent

Use ADD COLUMN IF NOT EXISTS so re-running 162 on a database that already has
the column is a no-op.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* build: stamp git revision via Makefile ldflags

`make build` did not inject buildinfo's `revisionOverride`/`dirtyOverride`
ldflags (the Dockerfile already does), so binaries built via make report
their version as "unavailable" in the admin Build panel whenever Go's VCS
metadata isn't embedded. Mirror the Dockerfile by computing the git
revision + dirty state and passing them through `-ldflags -X`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): accessibility & UX fixes from visual QA pass

Accessibility (WCAG AA):
- Lighten the Standard-theme `--muted-foreground` (#6e6e78 -> #9696a0,
  ~3.4:1 -> >=5.3:1) and darken the light-theme equivalent so secondary
  text meets 1.4.3 contrast app-wide; the opt-in High Contrast mode is no
  longer the only conformant path.
- Give icon-only controls accessible names (4.1.2): the password show/hide
  toggle (also drop tabIndex={-1} so it's keyboard reachable), and the
  Edit/Delete/health/copy/refresh actions across the Users, Libraries,
  Nodes, API Keys, Catalog Maintenance and Job History admin tables.
- Fix the Switch off-state (invisible track -> visible border + fill) and
  the PlaybackSettings SettingRow label association (the <label htmlFor>
  pointed at a wrapping <div>; the id now lands on the Switch/SelectTrigger).
- Login: wrap the card in <main> and add an <h1>; Profiles: add an
  accessible PIN-protected label and a corner lock badge.
- Player + catalog: role="status" on the initial loading overlay; scope the
  catalog count ("0 in library" for search) and announce it via aria-live;
  trim the verbose poster-link name to the title.

UX / consistency:
- Emphasize overdue scheduled tasks (warning colour + icon + word, not
  colour alone).
- Per-source catalog subtitles instead of one shared string.
- Add a Reconnect affordance when the admin log stream drops (it does not
  auto-retry).
- Page titles for Watch Party + all admin sub-pages (incl. plugins); admin
  heading capitalisation normalised to Title Case.
- Show "dev build" instead of "unavailable" when no build revision is
  stamped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): re-check orphan status when deleting library items

Orphan detection moved outside the media_items delete in the batched
library-delete rewrite, opening a TOCTOU race: a concurrent scan/import
could attach one of the collected content IDs to another library between
collectOrphanBatch and the delete, after which the unconditional
`DELETE FROM media_items WHERE content_id = ANY($1)` would still remove the
shared row and cascade away the newly-added membership — dropping the item
from the other library. Re-check the orphan invariant inside the delete
(NOT EXISTS a membership in another folder) and count rows actually deleted.

Addresses PR #21 review (P1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(metadata): persist a cleared air_timezone instead of skipping it

Clearing a previously-set air timezone sent JSON null, which decodes to a
nil *string that UpdateMetadata treats as "skip this column", so the old
value remained. The dialog now sends "" (accepted by ValidateAirTimezone),
and UpdateMetadata maps air_timezone through NULLIF so an empty value
persists as SQL NULL (matching the nullable column) rather than "".

Addresses PR #21 review (P2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(startup): sweep open scrobbles before accepting playback

The open-scrobble sweep was queued in the deferred background-init list,
which runs concurrently with the HTTP listener; a resume immediately after
restart could start new scrobbles before the previous process's open
sessions were stopped, leaving overlapping/stale scrobbles on remote
providers. Run the sweep synchronously before the listener starts, bounded
by a 30s timeout so an unreachable provider can't hang startup (the heavier
non-critical init stays deferred).

Addresses PR #21 review (P2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): scan air_timezone in paginated item queries

scanItemsWithTotal was not updated for the new air_timezone column, yet the
shared column lists it reads (itemColumns, qualifiedListItemColumns) include
it. Search and BrowseFavorites build their SELECTs from those lists with
COUNT(*) OVER (), so each row carried one more column than the scan had
destinations and every call failed at scan time with a pgx mismatch. Add the
missing &item.AirTimezone target between AirTime and ShowStatus.

Found during PR #21 review (critical: Search/Favorites runtime regression).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* revert(startup): keep open-scrobble sweep deferred for fast startup

Reverts 62563f5c. Running the sweep synchronously before the listener could
add up to 30s to restart-before-playback when a watch provider is
unreachable, which regresses the deliberate startup-deferral from a1a6c6cf.
Prefer the fast-startup behavior and accept the small window where a resume
immediately after restart may create a duplicate scrobble; the sweep returns
to the deferred background-init list. (Panic-safety for that list is added in
a follow-up commit.)

Per PR #21 review decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(startup): recover from panics in deferred background init

The deferred background-init steps run in a detached goroutine after the
HTTP listener is already accepting connections. An unrecovered panic in any
step (queue seeding, legacy cleanup, scrobble sweep) would crash the entire
live server. Wrap each step in a recover that logs the panic with a stack
and continues to the next step.

Found during PR #21 review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): make usernames and emails case-insensitive

Login identifiers were compared case-sensitively, so "John" and "john"
were distinct accounts and a user could not log in unless they matched the
exact casing used at registration.

Convert users.username and users.email to the citext type (migration 165).
citext compares case-insensitively while preserving the originally stored
casing for display, so the existing unique constraints become
case-insensitive and `WHERE username = $1` / `email = $1` lookups match
regardless of case with no change to the query code itself.

Also add auth.NormalizeUsername/NormalizeEmail (trim-only; case preserved),
applied at the repository chokepoints (Create, Update, GetByUsername,
GetByEmail) and before validation in the create paths, so surrounding
whitespace no longer defeats matching or creates lookalike accounts.

Verified non-destructively against the dev DB: mixed-case lookups resolve
to the same row, case-variant inserts are rejected by the unique
constraint, and the down migration cleanly reverts to text.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(sections): add trending_discover home section

A library-agnostic home section that surfaces external global trending
(TMDB or Trakt, admin-selectable) mixing movies + series, matched to
titles in the viewer's enabled libraries. TMDB uses /trending/all/{window}
(natively mixed); Trakt merges trending movies + shows. Fetched live with
a 1h in-process cache, so no background job or stored collection — and no
per-library duplication.

Appears in the admin section gallery via its recipe presets (TMDB Trending
Today/This Week, Trakt Trending); featured -> hero via the existing flag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add trending_discover persistent snapshot design

Replace the in-process 1h trending cache with a background-refreshed,
persisted snapshot for reliability under upstream failure and sync-run
observability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add trending_discover persistent snapshot implementation plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(sections): tidy trending_discover fetch and cache helpers

Extract newTrendingEntry, reuse orderMediaItems, and collapse concurrent
cache-miss loads with singleflight. Baseline for the persistent snapshot work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sections): add trending_discover_snapshots table

* feat(sections): add trending snapshot model and repository

* feat(sections): list enabled trending_discover section configs

* feat(sections): add trending refresher with persisted snapshots

* feat(tasks): add refresh_trending_discover task

* refactor(sections): read trending_discover from persisted snapshot

* feat: wire trending refresh task and snapshot reader

* chore(sections): satisfy lint (wrap trakt errors, lift source/window constants)

* fix(migrations): renumber trending_discover_snapshots 166 -> 167

The shared dev DB already recorded version 166 (166_trending_blend_collection_type
from another branch), so the integer-version migration runner silently skipped our
166 and the table was never created — the trending section errored out empty.
167 is the next free version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sections): harden trending refresher per PR review

- Interleave Trakt movies/shows by rank so the mixed row shows both types
  instead of burying all series past the display limit.
- Treat any Trakt sub-fetch failure as fatal (errors.Join) so a partial
  result never overwrites the last-good snapshot with a media type missing.
- Skip non-title entries (TMDB trending/all returns media_type "person") in
  both ID batching and ordering so they can't match an unrelated library title.
- Guard the refresh task against a nil refresher.
- Tests: person skip, Trakt interleave, Trakt partial-failure preserves
  last-good, snapshot read error propagation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): project air_timezone in episode catalog subquery

episodeCatalogSelectBody is the derived "mi" relation that episode catalog
hydration and preview read qualifiedListItemColumns("mi") from. The local
episode airtimes feature (808f265f) added air_timezone to the shared column
lists but not to this hand-written subquery, so the outer projection
referenced mi.air_timezone, which the subquery never exposed.

Postgres returns SQLSTATE 42703 (undefined_column), which is not one of the
codes episodeCatalogEntriesUnavailable treats as "fast path unavailable" (it
only catches 42P01/42883), so episode catalog requests failed with HTTP 500
instead of degrading. movie and series scopes query media_items directly, so
the column is present there and only episode scope broke.

Add si.air_timezone to the subquery, and add a regression test asserting that
episodeCatalogSelectBody exposes every column qualifiedListItemColumns reads
off mi, so future additions to the shared column lists cannot silently drift
from the episode read model again.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(calendar): order events by viewer-local wall-clock time

The local-airtime change re-sorted calendar events in Go using air_at,
the absolute UTC instant, which is nil whenever air_timezone is unset.
Since air_timezone is only inferred for a few networks/countries, most
events fell through to the alphabetical title tiebreak while still
displaying their raw air_time, so each day appeared scrambled.

Sort each local day by the wall-clock time the viewer actually sees,
mirroring the client: zoned events convert air_at into the viewer
timezone, unzoned events use the raw air_time, and date-only entries
(no air_time) sort last. The timezone reasoning lives in the new
catalog.CalendarEventLocalTime helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(calendar): add presets design spec and implementation plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): generalize personal filter to an id-set restriction

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): add per-profile followed/favorites/watchlist/watched resolvers

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): resolve presets to id-sets and overlay watched status

Also drops the now-unused Filter/UserID/ProfileID fields from the
blendUpcomingIntoDiscoverRows CalendarFilter literal in recommendations.go,
which only wants an unrestricted windowed query.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): wire popular and trending sources into calendar handler

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): add watched field to CalendarEvent type

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): preset selector with responsive pills, persistence, empty-state nudge

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): dim and check-mark already-watched event cards

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): hide server-wide Popular preset in web UI for now

Popular reflects server-wide watch counts, which are sparse on a
low-traffic server. Hidden from the selector, URL allowlist, and
empty-state nudge; backend filter and the CalendarFilter type are
left intact so re-enabling is a one-line change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(calendar): simplify preset handler and reuse storage util

- Extract hardcoded trending snapshot source/window to named constants.
- Collapse the three identical personal-preset nil-checks into one case.
- Persist the selected preset through the shared storage util (try/catch
  wrapped) instead of raw localStorage with manual SSR guards.
- Derive KNOWN_FILTERS from PRESET_OPTIONS so the lists can't drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(jellycompat): add /Users endpoint and unified sa_ admin-key auth

A user could not connect Tunarr to the Jellyfin-compat API. Tunarr's media-source
health check probes both /Users/Me and /Users; for API-key connections (its
recommended method) there is no "me", so it relies on GET /Users, which we did
not implement. Adding it alone was insufficient: a Silo sa_ admin key only
authorized two autoscan routes, so an API-key source would pass the ping but 401
on every browse/stream call.

This adds GET /Users and unifies auth so an sa_ admin key authorizes the same
browse/stream routes a session token does — matching Jellyfin, where an API key
authorizes every endpoint:

- GET /Users returns the caller's own user as a single-element list
  (current-profile-only; Silo is multi-account, so listing all users would leak
  across households). Behavior verified against real Jellyfin 10.11.8.
- An sa_ admin key synthesizes a compat session bound to the account's primary
  profile, injected into request context so existing handlers work unchanged.
  Applied to the browse group (RequireSessionOrAPIKeySession) and the stream
  group (PlaybackSessionAuth); /Library/VirtualFolders keeps its admin-bool path.
- The key + owning user are re-validated on every request (revocation is
  immediate); only the primary-profile lookup is cached. HLS follow-ups that
  carry only PlaySessionId resolve the negotiated session's sa_ CompatToken.

Validated end-to-end on dev: connect -> list user -> libraries -> browse ->
PlaybackInfo -> HLS stream, including PlaySessionId-only follow-ups.

Security notes: an admin key acts as the primary (parent) profile, so it bypasses
child-profile parental/PIN restrictions and can mutate the primary profile's
watch state — acceptable for an admin-trust key.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Silo Server Migration <noreply@silo-server.invalid>
Co-authored-by: zZebrahz <zzebrahz@gmail.com>
Co-authored-by: CoffeeKnyte <67730400+CoffeeKnyte@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Silo Server Developer <warmasterx555@gmail.com>
2026-06-08 14:25:10 -04:00
d3v1l1989andGitHub 65ea9b651c fix(web): drop backdrop-blur from Pill card overlay preset (#89)
The Pill overlay preset was the only one of the five to carry
`backdrop-blur-sm` (backdrop-filter: blur(4px)) on its badge class.
Each enabled badge renders its own element, so a typical library grid
produces hundreds of backdrop-filter layers the browser must re-sample
and blur on every repaint — including every scroll frame, since the
poster art behind each badge moves while scrolling. This caused severe
scroll/interaction jank, most visibly on Chrome on Windows.

The blur is also effectively invisible: the Pill background is already
rgba(20,20,30,0.7) (70% opaque), so removing it has no meaningful visual
impact while eliminating the per-frame GPU cost. The other four presets
are unaffected.
2026-06-08 09:57:47 -04:00
QuickandGitHub f5b3cba3cb fix(catalog): use semi-joins for discovery library scope (#88)
* fix(catalog): use semi-joins for discovery library scope

* fix(catalog): require membership for deny-only discovery
2026-06-08 09:33:55 -04:00
zZebrahzandGitHub 96bae2f346 fix(library): clean orphaned provisional items (#76)
* fix(library): clean orphaned provisional items

* test(metadata): use item not found sentinel in fake repo

* fix(library): preserve abs references during orphan cleanup

* fix(library): avoid dropped abs cleanup tables

* fix(library): harden provisional orphan cleanup

* fix(library): preserve home dismissal series orphans

* fix(library): Delete matched items created after the first orphan sweep
2026-06-07 21:32:40 -04:00
978e1b4954 feat(playback): NVENC support for transcoding (#79)
* feat(playback): NVENC support for transcoding

* fix(playback): probe nvenc before auto-selecting

* fix(playback): use safe NVENC smoke probe dimensions

---------

Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-06-07 21:27:56 -04:00
zZebrahzandGitHub c4e67effc3 fix(admin): correct dashboard library counts and scans (#78)
* fix(admin): correct dashboard library counts and scans

* fix(admin): handle mixed library dashboard counts

* fix(admin): refine dashboard count affordances
2026-06-07 19:59:10 -04:00
71 changed files with 6420 additions and 858 deletions
+19
View File
@@ -1,6 +1,12 @@
# Silo Docker deployment
# For the default docker compose setup, MEDIA_ROOT is the main value you need to set.
# Optional: layer compose files through env. Leave unset for CPU-only defaults.
# Linux/macOS example for enabling NVIDIA override:
# COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
# Windows example:
# COMPOSE_FILE=docker-compose.yml;docker-compose.nvidia.yml
# Required for docker compose: host path to your media library.
# Example: MEDIA_ROOT=/srv/media
MEDIA_ROOT=/srv/media
@@ -40,6 +46,19 @@ POSTGRES_SHM_SIZE=8gb
# PROXY_PORT=8083
# TRANSCODE_PORT=8082
# Optional NVIDIA override controls (used by docker-compose.nvidia.yml).
# NVIDIA_GPU_COUNT=1
# REQUIRED: master key for at-rest credential encryption. Silo refuses to start
# without it. Integration API keys, S3 keys, and other server-owned secrets are
# encrypted under a key derived from this value, so it must be at least 32
# characters and kept secret. Generate one with:
# openssl rand -base64 48
# Back it up SEPARATELY from your database dumps (treat it like a CA private
# key): losing it makes every encrypted secret unrecoverable. See
# docs/architecture/secret-encryption.md.
# SECRET_KEY=replace-with-output-of-openssl-rand-base64-48
# Run from source / advanced overrides
# Only DATABASE_URL is required when running Silo outside the default docker compose stack.
# DATABASE_URL=postgres://silo:password@localhost:5432/silo
+23
View File
@@ -34,6 +34,29 @@ The easiest way to run Silo is with Docker Compose. The default stack assumes yo
If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing `DATABASE_URL` and `REDIS_URL` instead.
### Optional NVIDIA/NVENC
GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.
Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.
Use the optional override file when you want NVENC:
```sh
docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
```
If you want this controlled from `.env`, set `COMPOSE_FILE`:
```dotenv
COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
NVIDIA_GPU_COUNT=1
```
Windows uses `;` instead of `:` between compose files.
Then `docker compose up -d` will include the NVIDIA override automatically.
4. **Configure through the admin UI**
Add libraries, users, metadata providers, and playback settings from the web interface.
+70 -15
View File
@@ -77,6 +77,7 @@ import (
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/server"
"github.com/Silo-Server/silo-server/internal/subtitles"
@@ -141,7 +142,7 @@ func mustGetSetting(store interface {
func configureOperationalLogging(
ctx context.Context,
pool *pgxpool.Pool,
settingsRepo *catalog.ServerSettingsRepo,
settingsRepo catalog.SettingsStore,
redisCfg config.RedisConfig,
logStreamHub *logstream.Hub,
baseHandler slog.Handler,
@@ -252,6 +253,38 @@ func maybeApplyPostgresTuning(ctx context.Context, pool *pgxpool.Pool, appMaxCon
}
}
// runCredentialBackfills sweeps any plaintext server-owned credential to
// ciphertext on the primary (migration-running) node. All passes are
// best-effort: a failed row leaves the prior plaintext (no new exposure) and
// still reads via the read-path pass-through, so a backfill error must never
// block boot. The sensitive-settings pass runs first so the arr
// resolve-then-encrypt pass sees consistent referenced settings.
func runCredentialBackfills(ctx context.Context, pool *pgxpool.Pool, cipher *secret.Cipher, settings *catalog.EncryptedSettingsRepo) {
settingsN, err := settings.BackfillSensitiveSettings(ctx)
if err != nil {
slog.Error("secret backfill: sensitive settings", "error", err)
}
columnsN, err := secret.BackfillColumns(ctx, pool, cipher, secret.ColumnBackfillTargets())
if err != nil {
slog.Error("secret backfill: credential columns", "error", err)
}
historyServersN, err := historyimport.NewRepository(pool, cipher).BackfillSessionServerSecrets(ctx)
if err != nil {
slog.Error("secret backfill: history import session server credentials", "error", err)
}
// The arr resolver is the encrypting settings decorator: it decrypts a
// sensitive target (e.g. requests.radarr.api_key) or passes through a
// plaintext custom key, exactly replicating the deleted resolveAPIKey.
arrN, err := secret.BackfillReferencedColumns(ctx, pool, cipher, settings.Get, secret.ArrKeyBackfillTargets())
if err != nil {
slog.Error("secret backfill: arr api keys", "error", err)
}
if total := settingsN + columnsN + historyServersN + arrN; total > 0 {
slog.Info("secret backfill: encrypted plaintext credentials at rest",
"settings", settingsN, "columns", columnsN, "history_session_servers", historyServersN, "arr_keys", arrN, "total", total)
}
}
func main() {
envFile := flag.String("env", ".env", "path to .env bootstrap file")
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
@@ -266,6 +299,15 @@ func main() {
log.Fatalf("bootstrap: %v", err)
}
// Construct the at-rest credential cipher from SECRET_KEY immediately after
// bootstrap, before any settings repo is built. It is threaded explicitly as
// a dependency into every repo that stores a server-owned secret — never a
// package-level global.
dataCipher, err := secret.New(bc.SecretKey)
if err != nil {
log.Fatalf("secret cipher: %v", err)
}
// Step 2: Connect to PostgreSQL (bootstrap pool with default max connections)
bootstrapDBCfg := config.DatabaseConfig{URL: bc.DatabaseURL, MaxConnections: 20}
pool, err := database.NewPool(ctx, bootstrapDBCfg)
@@ -314,7 +356,11 @@ func main() {
// Run migrations only for integrated/api modes. Proxy and transcode nodes
// should never alter the schema — they may scale independently and would
// race or apply migrations before the primary node is deliberately upgraded.
if bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == "" {
// The same gate decides whether this node runs the credential-encryption
// backfills: only the primary (migration-running) node sweeps plaintext to
// ciphertext; secondary nodes read whatever the primary encrypted.
isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == ""
if isPrimaryNode {
migCtx, migCancel := context.WithTimeout(ctx, 5*time.Minute)
if migErr := database.RunMigrations(migCtx, pool, migrations.FS, "sql"); migErr != nil {
migCancel()
@@ -324,8 +370,14 @@ func main() {
slog.Info("database migrations applied")
}
// Step 3: Load settings from DB
settingsRepo := catalog.NewServerSettingsRepo(pool)
// Step 3: Load settings from DB. settingsRepo is the encrypting decorator so
// every consumer (config.LoadFromDB, admin, ABS, watchers) transparently sees
// plaintext while sensitive keys rest as ciphertext. The settings backfill
// (run after migrations, before this GetAll) is wired further below.
settingsRepo := catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
if isPrimaryNode {
runCredentialBackfills(ctx, pool, dataCipher, settingsRepo)
}
settings, err := settingsRepo.GetAll(ctx)
if err != nil {
log.Fatalf("loading settings: %v", err)
@@ -396,7 +448,9 @@ func main() {
log.Fatalf("recreating pool with configured max_connections: %v", err)
}
}
settingsRepo = catalog.NewServerSettingsRepo(pool)
// Re-wrap with the encrypting decorator so the recreated pool's settings repo
// still encrypts/decrypts — no raw settings repo may escape into later wiring.
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
nodeID := resolveNodeIdentity()
// Step 9: Validate
@@ -460,7 +514,7 @@ func main() {
DatabaseURL: cfg.Database.URL,
JFListen: cfg.JellyfinCompat.Listen,
}
watcher := nodeconfig.NewWatcher(pool, eventBus, bootstrap)
watcher := nodeconfig.NewWatcher(pool, dataCipher, eventBus, bootstrap)
if err := watcher.Start(appCtx); err != nil {
slog.Error("config watcher start failed", "error", err)
os.Exit(1)
@@ -519,6 +573,7 @@ func main() {
BootstrapSensitiveValues: bootstrapSensitiveValues,
AppContext: appCtx,
DB: pool,
SecretCipher: dataCipher,
EventBus: eventBus,
LogStreamHub: logStreamHub,
RealtimeHub: realtimeHub,
@@ -595,7 +650,7 @@ func main() {
log.Fatalf("register watch provider: %v", err)
}
watchProviderService = watchsync.NewService(
watchsync.NewPostgresRepository(deps.DB),
watchsync.NewPostgresRepository(deps.DB, deps.SecretCipher),
watchProviderRegistry,
)
deps.WatchProviderService = watchProviderService
@@ -1180,7 +1235,7 @@ func main() {
deps.UserStoreProvider = userStoreProvider
}
if watchProviderService != nil {
historyRepo := historyimport.NewRepository(deps.DB)
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, catalog.NewProviderIDRepository(deps.DB))
watchProviderService.
WithMatcher(historyimport.NewMatcher(historyRepo)).
@@ -1443,6 +1498,7 @@ func main() {
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil {
taskMgr.Register(tasks.NewScanLibrariesTask(deps.FolderRepo, deps.LibraryScanQueue, deps.EventBus))
}
taskMgr.Register(tasks.NewCleanupOrphanedMediaItemsTask(catalog.NewOrphanedProvisionalCleaner(deps.DB)))
if deps.IntroAnalyzer != nil {
taskMgr.Register(tasks.NewDetectIntroMarkersTask(deps.IntroAnalyzer, settingsRepo))
}
@@ -1478,14 +1534,13 @@ func main() {
taskMgr.Register(tasks.NewSyncWatchProvidersTask(watchProviderService))
}
requestReconcileSvc := mediarequests.NewService(
mediarequests.NewRepository(deps.DB),
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
nil,
mediarequests.NewCatalogPresence(
catalog.NewItemRepository(deps.DB),
catalog.NewProviderIDRepository(deps.DB),
),
)
requestReconcileSvc.SetSecretResolver(settingsRepo)
requestReconcileSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if userStoreProvider != nil {
reconcileResolver := access.NewResolver(
@@ -1497,7 +1552,7 @@ func main() {
}
taskMgr.Register(tasks.NewReconcileRequestsTask(requestReconcileSvc, 100))
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && pluginService != nil && pluginInstallationStore != nil {
autoscanRepo := autoscan.NewRepository(deps.DB)
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if err := autoscanRepo.MarkInterruptedEvents(appCtx); err != nil {
slog.Warn("autoscan: failed to mark interrupted polls", "err", err)
}
@@ -1505,8 +1560,7 @@ func main() {
autoscanRepo,
pluginService,
pluginInstallationStore,
mediarequests.NewRepository(deps.DB),
settingsRepo,
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -1826,6 +1880,7 @@ func main() {
compatDeps := jellycompat.Dependencies{
Config: cfg,
DB: deps.DB,
SecretCipher: dataCipher,
ClientIPResolver: ipResolver,
ProxyPool: deps.ProxyPool,
TranscodePool: deps.TranscodePool,
@@ -1879,7 +1934,7 @@ func main() {
compatDeps.FileResolver = deps.FileRepo
}
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB)
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
// Construct auth service for jellycompat login.
userRepo := auth.NewUserRepository(deps.DB)
@@ -2379,7 +2434,7 @@ func mapFolderTypeToMediaType(t string) string {
// exposes Get) to the audiobooks.SettingsReader interface (which
// requires GetString). The two signatures are identical modulo name.
type audiobooksSettingsAdapter struct {
repo *catalog.ServerSettingsRepo
repo catalog.SettingsStore
}
func (a *audiobooksSettingsAdapter) GetString(ctx context.Context, key string) (string, error) {
+20
View File
@@ -0,0 +1,20 @@
services:
silo:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: ${NVIDIA_GPU_COUNT:-1}
capabilities: [gpu]
# Optional distributed-mode example. Most single-host installs should
# just use the integrated service above.
# silo-transcode:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: ${NVIDIA_GPU_COUNT:-1}
# capabilities: [gpu]
+9
View File
@@ -36,6 +36,11 @@ services:
restart: unless-stopped
environment:
MODE: integrated
# Master key for at-rest credential encryption. Required — the server
# refuses to start without it. Generate with: openssl rand -base64 48
# Back it up SEPARATELY from your database dumps; losing it makes encrypted
# secrets unrecoverable. See docs/architecture/secret-encryption.md.
SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
REDIS_URL: redis://redis:6379
SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
@@ -71,6 +76,8 @@ services:
# profiles: [proxy]
# environment:
# MODE: proxy
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# PORT: "8080"
@@ -95,6 +102,8 @@ services:
# profiles: [transcode]
# environment:
# MODE: transcode
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
+142
View File
@@ -0,0 +1,142 @@
# At-rest credential encryption
Silo encrypts third-party integration credentials and other server-owned secrets
at rest. Sonarr/Radarr API keys, S3 keys, watch-sync OAuth tokens, the
Audiobookshelf-compat signing key, and the sensitive `server_settings` entries
are stored as ciphertext in PostgreSQL and decrypted only in memory. The key that
protects them lives **outside** the database, so a full DB dump or compromise
does not expose the secrets.
## How it works
- **Cipher:** AES-256-GCM with a random 12-byte nonce per value. Implemented in
`internal/secret`.
- **Key derivation:** the data key is HKDF-SHA256–derived from the `SECRET_KEY`
environment variable with a versioned domain label. `SECRET_KEY` is the
encryption root; it is never stored in the database.
- **Envelope:** every ciphertext is stored as `enc:v1:<base64url(nonce‖sealed)>`.
The `enc:v1:` prefix is the version marker (see *Key rotation* below).
- **Row binding (AAD):** each ciphertext is GCM-bound to its logical row via
additional-authenticated data — `table:column:<pk>` for per-row columns,
`server_settings:<key>` for settings. A DB-write attacker therefore cannot move
a credential blob from one row/column to another: decrypting under a different
binding fails authentication.
- **Read path:** an empty value stays empty; a non-`enc:v1:` value is treated as
legacy plaintext and passed through unchanged (so a not-yet-migrated row keeps
working); an `enc:v1:` value is decrypted and **any** failure (wrong key,
tampering, truncation) is surfaced as an error — it is never silently used as a
credential.
## SECRET_KEY
`SECRET_KEY` is **required**. The server refuses to start without it (it fatals
in `config.LoadBootstrap`), and it must be at least 32 characters.
Generate one with:
```bash
openssl rand -base64 48
```
Set it in the deployment environment (or in `.env` for source/dev runs, which
`godotenv` loads at startup). See `.env.example`.
### Back it up — separately from the database
Treat `SECRET_KEY` like a CA private key:
- **Store it outside your database backups.** A DB dump contains only ciphertext;
the value of at-rest encryption is entirely lost if the key travels with the
dump.
- Keep it in a secrets manager / sealed secret / password manager, not in the
repo or in the same bucket as your Postgres backups.
- Rotating the *deployment* (new host, restored DB) requires the **same**
`SECRET_KEY`. Restoring a database backup onto a node with a different
`SECRET_KEY` makes every encrypted secret unreadable.
### Key loss
If `SECRET_KEY` is lost, the encrypted secrets are **unrecoverable** — that is the
point of keeping the key out of the database. Recovery means re-entering the
affected credentials:
- Re-enter Sonarr/Radarr and Autoscan API keys, S3 keys, watch-sync connections,
history-import tokens, and subtitle provider credentials.
- `auth.jwt_secret` becomes unreadable, so all existing sessions are invalid and
users must log in again (a new secret is generated on next boot if the row is
cleared — see below).
## Rollout (automatic backfill)
On the first boot after deploying this change, the primary (migration-running)
node runs an idempotent, best-effort startup backfill that encrypts any
remaining plaintext in place:
1. sensitive `server_settings` keys,
2. the per-table credential columns (subtitles, watch-sync, webhook-sync,
history-import, including temporary server-list credentials stored in
session JSON),
3. the two arr `api_key_ref` columns — these are **resolved-then-encrypted**: a
legacy row that held a `server_settings` reference (e.g.
`requests.radarr.api_key`) is collapsed to the real credential before being
encrypted.
The backfill is safe to run repeatedly: already-encrypted values are skipped, and
a per-row guard makes concurrent multi-node boots converge without
double-encrypting. A failed row is logged and left as plaintext (no new exposure)
rather than blocking boot. Proxy/transcode nodes skip the backfill; they read
whatever the primary encrypted. No manual steps are required.
## Rollback / downgrade hazard
Downgrading to a binary that predates this change is **not** safe while secrets
are encrypted, because the old binary has no read path: it would read
`enc:v1:auth.jwt_secret` as a literal JWT secret (invalidating all sessions) and
read `enc:v1:`-prefixed integration keys as garbage credentials.
To downgrade safely you must first return the affected values to plaintext, for
example:
- Decrypt the sensitive `server_settings` back to plaintext (run a one-off that
reads each via the encrypting repo and writes the plaintext via the raw repo),
**or**
- At minimum, clear `auth.jwt_secret` so the older binary regenerates a fresh
plaintext one (this still logs everyone out), and re-enter any integration
credentials as plaintext.
There is no automatic "decrypt everything" downgrade path in this release; plan
downgrades accordingly.
## Key rotation (future)
The `enc:v1:` envelope and the HKDF domain label are versioned. A future rotation
would introduce `enc:v2:` with a new derived key (and `Decrypt` already dispatches
on the version, returning an explicit error for an unknown version). Rotation is
out of scope for this release; the versioning exists so it can be added without a
data migration of the envelope format.
## Scope and known gaps
Covered: arr (Requests + Autoscan) API keys, S3 keys, all sensitive
`server_settings`, watch-sync tokens, webhook-sync `access_token`,
history-import admin/session tokens and temporary server-list credentials,
subtitle provider `api_key`/`password`,
the Jellyfin-compat session's bridged Silo access/refresh tokens
(`jellycompat_sessions.streamapp_access_token` / `streamapp_refresh_token`), and
the ABS signing key.
Deliberately **not** encrypted (tracked as follow-ups):
- **Equality-looked-up secrets** — these are matched by exact value
(`WHERE … = $1`), and AES-GCM is randomized, so encrypting them would break the
lookup. They need a deterministic **blind-index hash** column instead:
`api_keys.api_key`, `webhook_sync_connections.webhook_secret`,
`jellycompat_sessions.token`, and `watch_together_rooms.join_token`.
- **Plugin runtime config** — `plugin_runtime_configs.config_value` is opaque
plugin-defined JSONB whose secret fields are manifest-declared and whose runtime
lives in a separate repo; encrypting it correctly needs a coordinated,
manifest-aware design.
Excluded (not a gap): `plex_sync_connections.*` is a dead table (zero Go
references); `oauth_completion.token_ciphertext` is already AES-GCM;
`users.password_hash` and the `*_hash` columns are already hashed.
+145
View File
@@ -0,0 +1,145 @@
# First-Party Audio API
This API is the Silo-native contract for audiobook playback. Clients should use
it for Silo first-party audio experiences instead of the Audiobookshelf
compatibility API. Music can reuse the same session, track, and progress
concepts when the music UX lands.
## Authentication
All endpoints require normal Silo user authentication. Playback start, sync,
close, and bookmark writes also require the active profile headers used by the
first-party clients.
Track stream URLs are returned from the start response and are scoped to the
created audio session. The server validates that the session still belongs to
the authenticated user, and to the active profile when a profile is attached.
## Start Playback
`POST /api/v1/audio/playback/start`
Request:
```json
{
"content_id": "128814341867700228",
"start_position": 120,
"restart": false
}
```
Response:
```json
{
"session_id": "uuid",
"content_id": "128814341867700228",
"type": "audiobook",
"title": "Book Title",
"subtitle": "Author or series subtitle",
"poster_url": "https://server/api/v1/images/...",
"backdrop_url": "https://server/api/v1/images/...",
"total_duration_seconds": 8746,
"resume_position_seconds": 120,
"tracks": [
{
"index": 0,
"file_id": 123,
"file_name": "Part 1.mp3",
"duration_seconds": 3600,
"start_offset_seconds": 0,
"stream_url": "/api/v1/audio/playback/{session_id}/tracks/0",
"stream_type": "progressive",
"play_method": "direct",
"codec": "mp3",
"container": "mp3",
"bitrate": 128000,
"chapters": []
}
],
"chapters": [
{
"index": 0,
"title": "Chapter 1",
"start_seconds": 0,
"end_seconds": 1800,
"track_index": 0
}
],
"bookmarks": []
}
```
`total_duration_seconds`, `resume_position_seconds`, global chapter times, and
sync positions are cumulative book seconds, not per-file seconds. Clients map
global time to the active track by finding the last track whose
`start_offset_seconds` is less than or equal to the target time.
Tracks are ordered by presentation part index when available, then by path and
file id. The server direct-plays AVFoundation-safe formats and returns
`play_method: "aac"` for fallback streams that are remuxed/transcoded to an
Apple-safe AAC container.
## Stream Track
`GET /api/v1/audio/playback/{session_id}/tracks/{track_index}`
The response is the media stream for the selected track. Clients should request
the track for the current global time and seek within that track using:
`local_seconds = global_seconds - track.start_offset_seconds`
## Sync Progress
`PATCH /api/v1/audio/playback/{session_id}/sync`
Request:
```json
{
"position": 1234,
"duration": 8746,
"is_paused": false
}
```
The server persists `position` as the audiobook progress position. The client
should send cumulative book seconds on a throttle while playing, on manual
seeks, and once more before closing the session.
## Close Playback
`POST /api/v1/audio/playback/{session_id}/close`
Request:
```json
{
"position": 1234,
"duration": 8746,
"is_paused": true
}
```
Close performs a final progress sync and expires the in-memory audio session.
## Bookmarks
`GET /api/v1/audio/items/{content_id}/bookmarks`
`POST /api/v1/audio/items/{content_id}/bookmarks`
`DELETE /api/v1/audio/items/{content_id}/bookmarks/{time_seconds}`
Create request:
```json
{
"time_seconds": 1234,
"title": "Chapter note"
}
```
Bookmark positions are cumulative book seconds so they remain stable across
multi-file audiobooks and client platforms.
+6 -10
View File
@@ -20,12 +20,13 @@ type autoscanQueuer = autoscan.Queuer
// RequestIntegrationLookup adapts the Requests repository to the autoscan
// connection resolver's RequestIntegrationLookup: it resolves a soft-linked
// Requests integration to its base URL and (write-only) api-key ref.
// Requests integration to its base URL and api key (the repo decrypts the key
// on read, so this returns plaintext).
type RequestIntegrationLookup struct {
Repo *mediarequests.Repository
}
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error) {
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error) {
integration, err := l.Repo.GetIntegration(ctx, integrationID)
if err != nil {
return "", "", err
@@ -125,27 +126,22 @@ func scanSourceDisplayName(pluginID string, c *plugins.Capability) string {
}
}
// AutoscanSecretResolver resolves an encrypted api-key reference to plaintext.
// It is satisfied by the server settings repo (Get(ctx, key) (string, error)).
type AutoscanSecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
}
// BuildAutoscanService wires the v2 autoscan engine from its concrete
// dependencies. Both the HTTP router (manual trigger) and the background poll
// task share this constructor so the adapter wiring lives in exactly one place.
// Credentials are now decrypted inline by the autoscan/requests repos, so there
// is no separate secret resolver to thread.
func BuildAutoscanService(
repo *autoscan.Repository,
pluginService *plugins.Service,
installationStore *plugins.InstallationStore,
requestsRepo *mediarequests.Repository,
secrets AutoscanSecretResolver,
folderRepo *catalog.FolderRepository,
queue autoscanQueuer,
redisClient *redis.Client,
) *autoscan.Service {
provider := autoscan.NewPluginProvider(PluginScanSourceAdapter{pluginService})
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo}, secrets)
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo})
svc := autoscan.NewService(
repo,
provider,
+4 -23
View File
@@ -1137,29 +1137,10 @@ func (h *AdminHandler) HandleUpdateItemMetadata(w http.ResponseWriter, r *http.R
// --- Server Settings endpoints ---
var sensitiveSettingKeys = map[string]bool{
"auth.jwt_secret": true,
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"redis.url": true,
"recommendations.openai_api_key": true,
"recommendations.embedding_auth_token": true,
"tmdb.api_key": true,
"introdb.api_key": true,
"mdblist.api_key": true,
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
"subtitle_ai.api_key": true,
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
}
// sensitiveSettingKeys is the audited allowlist of secret-bearing settings
// keys, shared with the at-rest encryption decorator so redaction and
// encryption can never drift apart. See catalog.SensitiveSettingKeys.
var sensitiveSettingKeys = catalog.SensitiveSettingKeys
// HandleGetSettings handles GET /admin/settings.
func (h *AdminHandler) HandleGetSettings(w http.ResponseWriter, r *http.Request) {
+44 -8
View File
@@ -22,7 +22,9 @@ type AdminStats struct {
TotalFiles int `json:"total_files"`
TotalUsers int `json:"total_users"`
TotalMovies int `json:"total_movies"`
TotalMovieFiles int `json:"total_movie_files"`
TotalShows int `json:"total_shows"`
TotalShowFiles int `json:"total_show_files"`
ActiveStreams int `json:"active_streams"`
TotalStorageBytes int64 `json:"total_storage_bytes"`
WatchProviderActivity WatchProviderActivity `json:"watch_provider_activity"`
@@ -127,13 +129,15 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
}
var (
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalShows int64
activeStreams int64
totalStorage int64
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalMovieFiles int64
totalShows int64
totalShowFiles int64
activeStreams int64
totalStorage int64
)
row := pool.QueryRow(ctx, `
@@ -151,8 +155,34 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
file_stats AS (
SELECT
COUNT(*)::bigint AS total_files,
COUNT(*) FILTER (WHERE file_kind = 'movie')::bigint AS total_movie_files,
COUNT(*) FILTER (WHERE file_kind = 'series')::bigint AS total_show_files,
COALESCE(SUM(file_size), 0)::bigint AS total_storage_bytes
FROM media_files
FROM (
SELECT
media_files.file_size,
CASE
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'movie'
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'series'
THEN 'series'
WHEN episodes.content_id IS NOT NULL
THEN 'series'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
WHEN lower(trim(media_folders.type)) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(media_folders.type)) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
ELSE ''
END AS file_kind
FROM media_files
JOIN media_folders ON media_folders.id = media_files.media_folder_id
LEFT JOIN media_items ON media_items.content_id = media_files.content_id
LEFT JOIN episodes ON episodes.content_id = media_files.episode_id
) classified_files
),
session_stats AS (
SELECT COUNT(*)::bigint AS active_streams
@@ -163,7 +193,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
item_stats.total_items,
file_stats.total_files,
item_stats.total_movies,
file_stats.total_movie_files,
item_stats.total_shows,
file_stats.total_show_files,
session_stats.active_streams,
file_stats.total_storage_bytes
FROM user_stats
@@ -176,7 +208,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
&totalItems,
&totalFiles,
&totalMovies,
&totalMovieFiles,
&totalShows,
&totalShowFiles,
&activeStreams,
&totalStorage,
); err != nil {
@@ -194,7 +228,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
TotalItems: int(totalItems),
TotalFiles: int(totalFiles),
TotalMovies: int(totalMovies),
TotalMovieFiles: int(totalMovieFiles),
TotalShows: int(totalShows),
TotalShowFiles: int(totalShowFiles),
ActiveStreams: int(activeStreams),
TotalStorageBytes: totalStorage,
WatchProviderActivity: activity,
+757
View File
@@ -0,0 +1,757 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"fmt"
"math"
"net/http"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/oklog/ulid/v2"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/userstore"
)
const audioPlaybackSessionTTL = 12 * time.Hour
type AudioDetailService interface {
GetItemDetail(ctx context.Context, contentID string, filter catalog.AccessFilter) (*catalog.ItemDetail, error)
}
type AudioFileResolver interface {
GetByContentID(ctx context.Context, contentID string) ([]*models.MediaFile, error)
GetByID(ctx context.Context, id int) (*models.MediaFile, error)
}
type AudioHandler struct {
detailSvc AudioDetailService
files AudioFileResolver
storeProvider userstore.UserStoreProvider
bookmarkStore *PGAudioBookmarkStore
mu sync.Mutex
sessions map[string]audioPlaybackSession
now func() time.Time
}
func NewAudioHandler(
detailSvc AudioDetailService,
files AudioFileResolver,
storeProvider userstore.UserStoreProvider,
bookmarkStore *PGAudioBookmarkStore,
) *AudioHandler {
return &AudioHandler{
detailSvc: detailSvc,
files: files,
storeProvider: storeProvider,
bookmarkStore: bookmarkStore,
sessions: make(map[string]audioPlaybackSession),
now: func() time.Time { return time.Now().UTC() },
}
}
type audioPlaybackSession struct {
ID string
UserID int
ProfileID string
ContentID string
TotalSeconds float64
Files []audioSessionFile
CreatedAt time.Time
LastSeenAt time.Time
ExpiresAt time.Time
}
type audioSessionFile struct {
FileID int
FilePath string
Start float64
Duration float64
Mode string
TrackIdx int
}
type audioStartPlaybackRequest struct {
ContentID string `json:"content_id"`
StartPosition *float64 `json:"start_position,omitempty"`
Restart bool `json:"restart,omitempty"`
}
type audioProgressRequest struct {
Position float64 `json:"position"`
Duration float64 `json:"duration,omitempty"`
IsPaused bool `json:"is_paused"`
}
type audioPlaybackResponse struct {
SessionID string `json:"session_id"`
ContentID string `json:"content_id"`
Type string `json:"type"`
Title string `json:"title"`
Subtitle string `json:"subtitle,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
BackdropURL string `json:"backdrop_url,omitempty"`
TotalDurationSeconds float64 `json:"total_duration_seconds"`
ResumePositionSeconds float64 `json:"resume_position_seconds"`
Tracks []audioTrackResponse `json:"tracks"`
Chapters []audioChapter `json:"chapters"`
Bookmarks []audioBookmark `json:"bookmarks"`
}
type audioTrackResponse struct {
Index int `json:"index"`
FileID int `json:"file_id"`
FileName string `json:"file_name,omitempty"`
DurationSeconds float64 `json:"duration_seconds"`
StartOffsetSeconds float64 `json:"start_offset_seconds"`
StreamURL string `json:"stream_url"`
StreamType string `json:"stream_type"`
PlayMethod string `json:"play_method"`
Codec string `json:"codec,omitempty"`
Container string `json:"container,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
Chapters []audioChapter `json:"chapters"`
}
type audioChapter struct {
Index int `json:"index"`
Title string `json:"title,omitempty"`
StartSeconds float64 `json:"start_seconds"`
EndSeconds float64 `json:"end_seconds,omitempty"`
TrackIndex int `json:"track_index,omitempty"`
}
type audioBookmark struct {
ID string `json:"id"`
ContentID string `json:"content_id"`
TimeSeconds float64 `json:"time_seconds"`
Title string `json:"title"`
CreatedAt string `json:"created_at,omitempty"`
UpdatedAt string `json:"updated_at,omitempty"`
}
type audioBookmarkRequest struct {
TimeSeconds *float64 `json:"time_seconds"`
Title string `json:"title"`
}
func (h *AudioHandler) HandleStartPlayback(w http.ResponseWriter, r *http.Request) {
if h == nil || h.detailSvc == nil || h.files == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Audio playback is not configured")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if userID == 0 || profileID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Profile is required")
return
}
var req audioStartPlaybackRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
req.ContentID = strings.TrimSpace(req.ContentID)
if req.ContentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return
}
filter := requestAccessFilter(r)
detail, err := h.detailSvc.GetItemDetail(r.Context(), req.ContentID, filter)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load item detail")
return
}
if detail == nil || detail.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "Audiobook not found")
return
}
files, err := h.files.GetByContentID(r.Context(), req.ContentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load audio files")
return
}
files = catalog.FilterMediaFilesByAccess(files, filter)
sortAudioFiles(files)
if len(files) == 0 {
writeError(w, http.StatusNotFound, "not_found", "No playable audio files found")
return
}
sessionID := ulid.Make().String()
tracks, sessionFiles, chapters, totalDuration := buildAudioTrackResponses(sessionID, files)
if detail.Audiobook != nil && detail.Audiobook.TotalDurationSeconds > 0 {
totalDuration = float64(detail.Audiobook.TotalDurationSeconds)
}
resume := resolveAudioResumePosition(r.Context(), h.storeProvider, userID, profileID, req.ContentID)
if req.Restart {
resume = 0
} else if req.StartPosition != nil && isFiniteNonNegative(*req.StartPosition) {
resume = clampFloat(*req.StartPosition, 0, totalDuration)
}
bookmarks, _ := h.listBookmarks(r.Context(), userID, profileID, req.ContentID)
session := audioPlaybackSession{
ID: sessionID,
UserID: userID,
ProfileID: profileID,
ContentID: req.ContentID,
TotalSeconds: totalDuration,
Files: sessionFiles,
CreatedAt: h.now(),
LastSeenAt: h.now(),
ExpiresAt: h.now().Add(audioPlaybackSessionTTL),
}
h.putSession(session)
writeJSON(w, http.StatusOK, audioPlaybackResponse{
SessionID: sessionID,
ContentID: detail.ContentID,
Type: detail.Type,
Title: detail.Title,
Subtitle: audioSubtitle(detail),
PosterURL: detail.PosterURL,
BackdropURL: detail.BackdropURL,
TotalDurationSeconds: totalDuration,
ResumePositionSeconds: resume,
Tracks: tracks,
Chapters: chapters,
Bookmarks: bookmarks,
})
}
func (h *AudioHandler) HandleSyncPlayback(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
var req audioProgressRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
duration := req.Duration
if duration <= 0 {
duration = session.TotalSeconds
}
position := clampFloat(req.Position, 0, duration)
if err := h.persistProgress(r.Context(), session.UserID, session.ProfileID, session.ContentID, position, duration); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync audio progress")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"session_id": session.ID,
"position": position,
"duration": duration,
"is_paused": req.IsPaused,
})
}
func (h *AudioHandler) HandleClosePlayback(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
var req audioProgressRequest
_ = json.NewDecoder(r.Body).Decode(&req)
if isFiniteNonNegative(req.Position) {
duration := req.Duration
if duration <= 0 {
duration = session.TotalSeconds
}
position := clampFloat(req.Position, 0, duration)
_ = h.persistProgress(r.Context(), session.UserID, session.ProfileID, session.ContentID, position, duration)
}
h.deleteSession(session.ID)
w.WriteHeader(http.StatusNoContent)
}
func (h *AudioHandler) HandleStreamTrack(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
rawIndex := chi.URLParam(r, "track_index")
index, err := strconv.Atoi(rawIndex)
if err != nil || index < 0 || index >= len(session.Files) {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid track index")
return
}
file := session.Files[index]
mode := r.URL.Query().Get("mode")
if mode == "" {
mode = file.Mode
}
if mode == "aac" {
seek := 0.0
if raw := r.URL.Query().Get("seek"); raw != "" {
if parsed, err := strconv.ParseFloat(raw, 64); err == nil && parsed >= 0 {
seek = parsed
}
}
if err := playback.ServeRemux(w, r, file.FilePath, "mp4", seek, true, 0); err != nil {
return
}
return
}
_ = playback.ServeDirectPlay(w, r, file.FilePath)
}
func (h *AudioHandler) HandleListBookmarks(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
bookmarks, err := h.listBookmarks(r.Context(), userID, profileID, contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list bookmarks")
return
}
writeJSON(w, http.StatusOK, map[string]any{"bookmarks": bookmarks})
}
func (h *AudioHandler) HandleCreateBookmark(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
if h.bookmarkStore == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Bookmarks are not configured")
return
}
var req audioBookmarkRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
if req.TimeSeconds == nil || !isFiniteNonNegative(*req.TimeSeconds) {
writeError(w, http.StatusBadRequest, "bad_request", "time_seconds is required")
return
}
filter := requestAccessFilter(r)
detail, err := h.detailSvc.GetItemDetail(r.Context(), contentID, filter)
if err != nil || detail == nil || detail.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "Audiobook not found")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
bookmark, err := h.bookmarkStore.Upsert(r.Context(), userID, profileID, contentID, *req.TimeSeconds, req.Title)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save bookmark")
return
}
writeJSON(w, http.StatusOK, bookmark)
}
func (h *AudioHandler) HandleDeleteBookmark(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
if h.bookmarkStore == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Bookmarks are not configured")
return
}
rawTime := chi.URLParam(r, "time_seconds")
timeSeconds, err := strconv.ParseFloat(rawTime, 64)
if err != nil || !isFiniteNonNegative(timeSeconds) {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid bookmark time")
return
}
if err := h.bookmarkStore.Delete(r.Context(), apimw.GetUserID(r.Context()), apimw.GetProfileID(r.Context()), contentID, timeSeconds); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to delete bookmark")
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *AudioHandler) contentIDParam(w http.ResponseWriter, r *http.Request) (string, bool) {
contentID := strings.TrimSpace(chi.URLParam(r, "content_id"))
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return "", false
}
return contentID, true
}
func (h *AudioHandler) sessionForRequest(w http.ResponseWriter, r *http.Request) (audioPlaybackSession, bool) {
sessionID := chi.URLParam(r, "session_id")
if sessionID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Session ID is required")
return audioPlaybackSession{}, false
}
session, ok := h.getSession(sessionID)
if !ok {
writeError(w, http.StatusNotFound, "not_found", "Audio playback session not found")
return audioPlaybackSession{}, false
}
if session.UserID != apimw.GetUserID(r.Context()) {
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
return audioPlaybackSession{}, false
}
profileID := apimw.GetProfileID(r.Context())
if profileID != "" && session.ProfileID != profileID {
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another profile")
return audioPlaybackSession{}, false
}
return session, true
}
func (h *AudioHandler) putSession(session audioPlaybackSession) {
h.mu.Lock()
defer h.mu.Unlock()
h.pruneExpiredLocked(h.now())
h.sessions[session.ID] = session
}
func (h *AudioHandler) getSession(sessionID string) (audioPlaybackSession, bool) {
h.mu.Lock()
defer h.mu.Unlock()
now := h.now()
h.pruneExpiredLocked(now)
session, ok := h.sessions[sessionID]
if !ok {
return audioPlaybackSession{}, false
}
if now.After(session.ExpiresAt) {
delete(h.sessions, sessionID)
return audioPlaybackSession{}, false
}
session.LastSeenAt = now
h.sessions[sessionID] = session
return session, true
}
func (h *AudioHandler) deleteSession(sessionID string) {
h.mu.Lock()
defer h.mu.Unlock()
delete(h.sessions, sessionID)
}
func (h *AudioHandler) pruneExpiredLocked(now time.Time) {
for id, session := range h.sessions {
if now.After(session.ExpiresAt) {
delete(h.sessions, id)
}
}
}
func (h *AudioHandler) persistProgress(ctx context.Context, userID int, profileID, contentID string, position, duration float64) error {
if h.storeProvider == nil {
return nil
}
store, err := h.storeProvider.ForUser(ctx, userID)
if err != nil {
return err
}
return store.SetProgress(ctx, profileID, contentID, position, duration, userstore.ProgressThresholds{})
}
func (h *AudioHandler) listBookmarks(ctx context.Context, userID int, profileID, contentID string) ([]audioBookmark, error) {
if h.bookmarkStore == nil {
return []audioBookmark{}, nil
}
return h.bookmarkStore.List(ctx, userID, profileID, contentID)
}
func buildAudioTrackResponses(sessionID string, files []*models.MediaFile) ([]audioTrackResponse, []audioSessionFile, []audioChapter, float64) {
tracks := make([]audioTrackResponse, 0, len(files))
sessionFiles := make([]audioSessionFile, 0, len(files))
allChapters := []audioChapter{}
offset := 0.0
for index, file := range files {
duration := float64(file.Duration)
if duration < 0 {
duration = 0
}
mode := audioDeliveryMode(file)
streamURL := fmt.Sprintf("/api/v1/audio/playback/%s/tracks/%d", sessionID, index)
if mode == "aac" {
streamURL += "?mode=aac"
}
chapters := audioChaptersForTrack(index, offset, file.Chapters)
allChapters = append(allChapters, chapters...)
tracks = append(tracks, audioTrackResponse{
Index: index,
FileID: file.ID,
FileName: filepath.Base(file.FilePath),
DurationSeconds: duration,
StartOffsetSeconds: offset,
StreamURL: streamURL,
StreamType: "progressive",
PlayMethod: mode,
Codec: file.CodecAudio,
Container: file.Container,
Bitrate: file.Bitrate,
Chapters: chapters,
})
sessionFiles = append(sessionFiles, audioSessionFile{
FileID: file.ID,
FilePath: file.FilePath,
Start: offset,
Duration: duration,
Mode: mode,
TrackIdx: index,
})
offset += duration
}
return tracks, sessionFiles, allChapters, offset
}
func audioChaptersForTrack(trackIndex int, offset float64, chapters []models.MediaChapter) []audioChapter {
out := make([]audioChapter, 0, len(chapters))
for _, chapter := range chapters {
start := offset + maxFloat(0, chapter.StartSeconds)
end := 0.0
if chapter.EndSeconds > 0 {
end = offset + chapter.EndSeconds
}
out = append(out, audioChapter{
Index: len(out),
Title: chapter.Title,
StartSeconds: start,
EndSeconds: end,
TrackIndex: trackIndex,
})
}
return out
}
func sortAudioFiles(files []*models.MediaFile) {
sort.SliceStable(files, func(i, j int) bool {
left, right := files[i], files[j]
if left.PresentationPartIndex != right.PresentationPartIndex {
if left.PresentationPartIndex == 0 {
return false
}
if right.PresentationPartIndex == 0 {
return true
}
return left.PresentationPartIndex < right.PresentationPartIndex
}
if left.FilePath != right.FilePath {
return left.FilePath < right.FilePath
}
return left.ID < right.ID
})
}
func audioDeliveryMode(file *models.MediaFile) string {
if file == nil {
return "direct"
}
codec := strings.ToLower(strings.TrimSpace(file.CodecAudio))
container := strings.ToLower(strings.TrimPrefix(strings.TrimSpace(file.Container), "."))
ext := strings.ToLower(strings.TrimPrefix(filepath.Ext(file.FilePath), "."))
if container == "" {
container = ext
}
switch codec {
case "aac", "mp3", "alac", "flac":
return "direct"
case "":
switch container {
case "mp3", "m4a", "m4b", "mp4", "aac", "flac", "wav", "aiff", "aif":
return "direct"
}
}
return "aac"
}
func audioSubtitle(detail *catalog.ItemDetail) string {
if detail == nil || detail.Audiobook == nil {
return ""
}
parts := []string{}
if len(detail.Audiobook.Authors) > 0 {
names := make([]string, 0, len(detail.Audiobook.Authors))
for _, author := range detail.Audiobook.Authors {
if strings.TrimSpace(author.Name) != "" {
names = append(names, author.Name)
}
}
if len(names) > 0 {
parts = append(parts, strings.Join(names, ", "))
}
}
if len(detail.Audiobook.Narrators) > 0 {
names := make([]string, 0, len(detail.Audiobook.Narrators))
for _, narrator := range detail.Audiobook.Narrators {
if strings.TrimSpace(narrator.Name) != "" {
names = append(names, narrator.Name)
}
}
if len(names) > 0 {
parts = append(parts, "Narrated by "+strings.Join(names, ", "))
}
}
return strings.Join(parts, " • ")
}
func resolveAudioResumePosition(ctx context.Context, provider userstore.UserStoreProvider, userID int, profileID, contentID string) float64 {
if provider == nil || userID == 0 || profileID == "" {
return 0
}
store, err := provider.ForUser(ctx, userID)
if err != nil {
return 0
}
progress, err := store.GetProgress(ctx, profileID, contentID)
if err != nil || progress == nil || progress.Completed {
return 0
}
if progress.PositionSeconds < 30 {
return 0
}
if progress.DurationSeconds > 0 && progress.PositionSeconds >= progress.DurationSeconds-5 {
return 0
}
return progress.PositionSeconds
}
func isFiniteNonNegative(value float64) bool {
return !math.IsNaN(value) && !math.IsInf(value, 0) && value >= 0
}
func clampFloat(value, lower, upper float64) float64 {
if !isFiniteNonNegative(value) {
return lower
}
if value < lower {
return lower
}
if upper > lower && value > upper {
return upper
}
return value
}
func maxFloat(a, b float64) float64 {
if a > b {
return a
}
return b
}
type PGAudioBookmarkStore struct {
Pool *pgxpool.Pool
}
func NewPGAudioBookmarkStore(pool *pgxpool.Pool) *PGAudioBookmarkStore {
if pool == nil {
return nil
}
return &PGAudioBookmarkStore{Pool: pool}
}
func (s *PGAudioBookmarkStore) List(ctx context.Context, userID int, profileID, contentID string) ([]audioBookmark, error) {
if s == nil || s.Pool == nil {
return []audioBookmark{}, nil
}
rows, err := s.Pool.Query(ctx, `
SELECT id, library_item_id, time_seconds, title, created_at, updated_at
FROM abs_bookmarks
WHERE user_id = $1
AND COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid)
= COALESCE($2::uuid, '00000000-0000-0000-0000-000000000000'::uuid)
AND library_item_id = $3
ORDER BY time_seconds ASC`,
userID, profileArg(profileID), contentID,
)
if errors.Is(err, pgx.ErrNoRows) {
return []audioBookmark{}, nil
}
if err != nil {
return nil, err
}
defer rows.Close()
bookmarks := []audioBookmark{}
for rows.Next() {
var createdAt, updatedAt time.Time
var bookmark audioBookmark
if err := rows.Scan(&bookmark.ID, &bookmark.ContentID, &bookmark.TimeSeconds, &bookmark.Title, &createdAt, &updatedAt); err != nil {
return nil, err
}
bookmark.CreatedAt = createdAt.UTC().Format(time.RFC3339)
bookmark.UpdatedAt = updatedAt.UTC().Format(time.RFC3339)
bookmarks = append(bookmarks, bookmark)
}
if err := rows.Err(); err != nil {
return nil, err
}
return bookmarks, nil
}
func (s *PGAudioBookmarkStore) Upsert(ctx context.Context, userID int, profileID, contentID string, timeSeconds float64, title string) (audioBookmark, error) {
if s == nil || s.Pool == nil {
return audioBookmark{}, fmt.Errorf("bookmark store unavailable")
}
id := ulid.Make().String()
var createdAt, updatedAt time.Time
var bookmark audioBookmark
err := s.Pool.QueryRow(ctx, `
INSERT INTO abs_bookmarks
(id, user_id, profile_id, library_item_id, time_seconds, title)
VALUES ($1, $2, $3::uuid, $4, $5, $6)
ON CONFLICT (
user_id,
COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid),
library_item_id,
time_seconds
) DO UPDATE
SET title = EXCLUDED.title,
updated_at = now()
RETURNING id, library_item_id, time_seconds, title, created_at, updated_at`,
id, userID, profileArg(profileID), contentID, timeSeconds, strings.TrimSpace(title),
).Scan(&bookmark.ID, &bookmark.ContentID, &bookmark.TimeSeconds, &bookmark.Title, &createdAt, &updatedAt)
if err != nil {
return audioBookmark{}, err
}
bookmark.CreatedAt = createdAt.UTC().Format(time.RFC3339)
bookmark.UpdatedAt = updatedAt.UTC().Format(time.RFC3339)
return bookmark, nil
}
func (s *PGAudioBookmarkStore) Delete(ctx context.Context, userID int, profileID, contentID string, timeSeconds float64) error {
if s == nil || s.Pool == nil {
return fmt.Errorf("bookmark store unavailable")
}
_, err := s.Pool.Exec(ctx, `
DELETE FROM abs_bookmarks
WHERE user_id = $1
AND COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid)
= COALESCE($2::uuid, '00000000-0000-0000-0000-000000000000'::uuid)
AND library_item_id = $3
AND time_seconds = $4`,
userID, profileArg(profileID), contentID, timeSeconds,
)
return err
}
func profileArg(profileID string) any {
if strings.TrimSpace(profileID) == "" {
return nil
}
return profileID
}
+201
View File
@@ -0,0 +1,201 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
type stubAudioDetailService struct {
detail *catalog.ItemDetail
}
func (s stubAudioDetailService) GetItemDetail(context.Context, string, catalog.AccessFilter) (*catalog.ItemDetail, error) {
return s.detail, nil
}
type stubAudioFiles struct {
files []*models.MediaFile
}
func (s stubAudioFiles) GetByContentID(context.Context, string) ([]*models.MediaFile, error) {
return append([]*models.MediaFile(nil), s.files...), nil
}
func (s stubAudioFiles) GetByID(_ context.Context, id int) (*models.MediaFile, error) {
for _, file := range s.files {
if file.ID == id {
return file, nil
}
}
return nil, nil
}
func TestAudioStartPlaybackOrdersTracksAndBuildsGlobalChapters(t *testing.T) {
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{
testAudioFile(3, "part-03.mp3", 30, 30),
testAudioFile(1, "part-01.mp3", 10, 10),
testAudioFile(2, "part-02.mp3", 20, 20),
}},
testUserStoreProvider{store: newPlaybackTestStore(t)},
nil,
)
req := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
req = req.WithContext(newAuthorizedPlaybackContext())
rec := httptest.NewRecorder()
handler.HandleStartPlayback(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", rec.Code, rec.Body.String())
}
var resp audioPlaybackResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if len(resp.Tracks) != 3 {
t.Fatalf("tracks = %d, want 3", len(resp.Tracks))
}
if resp.Tracks[0].FileID != 1 || resp.Tracks[1].FileID != 2 || resp.Tracks[2].FileID != 3 {
t.Fatalf("track order = [%d %d %d], want [1 2 3]", resp.Tracks[0].FileID, resp.Tracks[1].FileID, resp.Tracks[2].FileID)
}
if resp.Tracks[1].StartOffsetSeconds != 10 || resp.Tracks[2].StartOffsetSeconds != 30 {
t.Fatalf("offsets = %.0f %.0f, want 10 30", resp.Tracks[1].StartOffsetSeconds, resp.Tracks[2].StartOffsetSeconds)
}
if len(resp.Chapters) != 3 || resp.Chapters[2].StartSeconds != 30 {
t.Fatalf("global chapters = %+v, want third chapter at 30s", resp.Chapters)
}
}
func TestAudioSyncPersistsCumulativeProgressAndEnforcesOwner(t *testing.T) {
store := newPlaybackTestStore(t)
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{
testAudioFile(1, "part-01.mp3", 10, 10),
testAudioFile(2, "part-02.mp3", 20, 20),
}},
testUserStoreProvider{store: store},
nil,
)
startReq := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
startReq = startReq.WithContext(newAuthorizedPlaybackContext())
startRec := httptest.NewRecorder()
handler.HandleStartPlayback(startRec, startReq)
if startRec.Code != http.StatusOK {
t.Fatalf("start status = %d body=%s", startRec.Code, startRec.Body.String())
}
var start audioPlaybackResponse
if err := json.Unmarshal(startRec.Body.Bytes(), &start); err != nil {
t.Fatalf("decode start: %v", err)
}
otherCtx := apimw.SetClaims(context.Background(), &auth.Claims{UserID: 2, Role: "user", TokenType: auth.TokenTypeAccess})
otherCtx = apimw.SetProfileID(otherCtx, "profile-1")
denied := httptest.NewRequest(http.MethodPatch, "/api/v1/audio/playback/"+start.SessionID+"/sync", strings.NewReader(`{"position":12,"duration":30}`))
denied = withAudioRouteParam(denied.WithContext(otherCtx), "session_id", start.SessionID)
deniedRec := httptest.NewRecorder()
handler.HandleSyncPlayback(deniedRec, denied)
if deniedRec.Code != http.StatusForbidden {
t.Fatalf("foreign sync status = %d, want 403", deniedRec.Code)
}
syncReq := httptest.NewRequest(http.MethodPatch, "/api/v1/audio/playback/"+start.SessionID+"/sync", strings.NewReader(`{"position":18,"duration":30}`))
syncReq = withAudioRouteParam(syncReq.WithContext(newAuthorizedPlaybackContext()), "session_id", start.SessionID)
syncRec := httptest.NewRecorder()
handler.HandleSyncPlayback(syncRec, syncReq)
if syncRec.Code != http.StatusOK {
t.Fatalf("sync status = %d body=%s", syncRec.Code, syncRec.Body.String())
}
progress, err := store.GetProgress(context.Background(), "profile-1", "book-1")
if err != nil {
t.Fatalf("get progress: %v", err)
}
if progress == nil || progress.PositionSeconds != 18 || progress.DurationSeconds != 30 {
t.Fatalf("progress = %+v, want 18/30", progress)
}
}
func TestAudioSessionExpires(t *testing.T) {
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{testAudioFile(1, "part-01.mp3", 10, 10)}},
testUserStoreProvider{store: newPlaybackTestStore(t)},
nil,
)
now := time.Date(2026, 6, 8, 12, 0, 0, 0, time.UTC)
handler.now = func() time.Time { return now }
startReq := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
startReq = startReq.WithContext(newAuthorizedPlaybackContext())
startRec := httptest.NewRecorder()
handler.HandleStartPlayback(startRec, startReq)
var start audioPlaybackResponse
if err := json.Unmarshal(startRec.Body.Bytes(), &start); err != nil {
t.Fatalf("decode start: %v", err)
}
now = now.Add(audioPlaybackSessionTTL + time.Second)
streamReq := httptest.NewRequest(http.MethodHead, "/api/v1/audio/playback/"+start.SessionID+"/tracks/0", nil)
streamReq = withAudioRouteParam(streamReq.WithContext(newAuthorizedPlaybackContext()), "session_id", start.SessionID)
streamReq = withAudioRouteParam(streamReq, "track_index", "0")
streamRec := httptest.NewRecorder()
handler.HandleStreamTrack(streamRec, streamReq)
if streamRec.Code != http.StatusNotFound {
t.Fatalf("expired stream status = %d, want 404", streamRec.Code)
}
}
func testAudiobookDetail() *catalog.ItemDetail {
return &catalog.ItemDetail{
ContentID: "book-1",
Type: "audiobook",
Title: "Book One",
Audiobook: &catalog.AudiobookDetailExtension{
Authors: []catalog.AudiobookPerson{{Name: "Author One"}},
},
}
}
func testAudioFile(id int, name string, partIndex int, duration int) *models.MediaFile {
return &models.MediaFile{
ID: id,
ContentID: "book-1",
FilePath: "/media/" + name,
CodecAudio: "mp3",
Container: "mp3",
Duration: duration,
PresentationPartIndex: partIndex,
Chapters: []models.MediaChapter{{
Index: 0,
Title: name,
StartSeconds: 0,
EndSeconds: float64(duration),
}},
}
}
func withAudioRouteParam(req *http.Request, key, value string) *http.Request {
routeCtx := chi.RouteContext(req.Context())
if routeCtx == nil {
routeCtx = chi.NewRouteContext()
}
routeCtx.URLParams.Add(key, value)
return req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, routeCtx))
}
+1 -1
View File
@@ -32,7 +32,7 @@ type SectionHandler struct {
StoreProvider userstore.UserStoreProvider
UserRepo *auth.UserRepository
DetailSvc *catalog.DetailService
Settings *catalog.ServerSettingsRepo
Settings catalog.SettingsStore
CollectionRepo *catalog.LibraryCollectionRepository
}
+1 -1
View File
@@ -13,7 +13,7 @@ const SectionsAllowProfileCustomSettingKey = "sections.allow_profile_custom_sect
// SectionSettingsHandler exposes GET/PUT for the sections-related server setting.
type SectionSettingsHandler struct {
Settings *catalog.ServerSettingsRepo
Settings catalog.SettingsStore
}
type sectionsSettingResponse struct {
+4 -2
View File
@@ -17,14 +17,16 @@ import (
type SystemHandler struct {
transcodePool *nodepool.TranscodePool
jwtSecret string
ffmpegPath string
buildInfo buildinfo.Info
}
// NewSystemHandler creates a SystemHandler.
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string) *SystemHandler {
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string, ffmpegPath string) *SystemHandler {
return &SystemHandler{
transcodePool: transcodePool,
jwtSecret: jwtSecret,
ffmpegPath: ffmpegPath,
buildInfo: buildinfo.Current(),
}
}
@@ -45,7 +47,7 @@ func (h *SystemHandler) HandleHWAccel(w http.ResponseWriter, r *http.Request) {
}
}
info := playback.DetectHWAccel()
info := playback.DetectHWAccelWithFFmpeg(h.ffmpegPath)
writeJSON(w, http.StatusOK, info)
}
+40 -10
View File
@@ -53,6 +53,7 @@ import (
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/subtitles"
subtitleai "github.com/Silo-Server/silo-server/internal/subtitles/ai"
@@ -76,6 +77,7 @@ type Dependencies struct {
BootstrapSensitiveValues map[string]string
AppContext context.Context
DB *pgxpool.Pool
SecretCipher *secret.Cipher // at-rest credential cipher (required when DB is set)
FrontendFS fs.FS
S3Public *s3client.Client // public assets bucket client (may be nil)
S3Private *s3client.Client // private internal bucket client (may be nil)
@@ -223,9 +225,11 @@ func NewRouter(deps Dependencies) chi.Router {
healthHandler := handlers.NewHealthHandler(healthServerName, healthServerID)
// Build server settings repo if DB is available (needed by auth and admin).
var settingsRepo *catalog.ServerSettingsRepo
// Wrap it in the encrypting decorator so sensitive keys rest as ciphertext
// and every consumer transparently reads plaintext.
var settingsRepo catalog.SettingsStore
if deps.DB != nil {
settingsRepo = catalog.NewServerSettingsRepo(deps.DB)
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(deps.DB), deps.SecretCipher)
}
// Build auth handler and auth middleware if DB and config are available.
@@ -431,27 +435,25 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.Config != nil {
tmdbAPIKey = deps.Config.TMDBAPIKey
}
requestsRepo := mediarequests.NewRepository(deps.DB)
requestsRepo := mediarequests.NewRepository(deps.DB, deps.SecretCipher)
requestSvc := mediarequests.NewService(
requestsRepo,
tmdb.NewClient(tmdbAPIKey, 40),
mediarequests.NewCatalogPresence(itemRepo, providerIDRepo),
)
requestSvc.SetSecretResolver(settingsRepo)
requestSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if viewerResolver != nil {
requestSvc.SetEntitlementResolver(mediarequests.NewAccessEntitlements(viewerResolver))
}
requestHandler = handlers.NewRequestsHandler(requestSvc)
autoscanRepo := autoscan.NewRepository(deps.DB)
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && deps.PluginService != nil {
autoscanSvc := BuildAutoscanService(
autoscanRepo,
deps.PluginService,
plugins.NewInstallationStore(deps.DB),
requestsRepo,
settingsRepo,
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -566,7 +568,7 @@ func NewRouter(deps Dependencies) chi.Router {
// Create subtitleRepo early — only needs DB, shared with playback handler and subtitle search handler.
var subtitleRepo *subtitles.PgRepository
if deps.DB != nil {
subtitleRepo = subtitles.NewPgRepository(deps.DB)
subtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
}
// Notifier that pushes "subtitle ready" events to active sessions when an AI
@@ -580,6 +582,7 @@ func NewRouter(deps Dependencies) chi.Router {
var playbackCommandDispatcher *playback.CommandDispatcher
var streamHandler *handlers.StreamHandler
var watchTogetherHandler *handlers.WatchTogetherHandler
var audioHandler *handlers.AudioHandler
if deps.SessionMgr != nil {
var playbackAdminStore handlers.PlaybackAdminStore
if deps.DB != nil {
@@ -689,6 +692,14 @@ func NewRouter(deps Dependencies) chi.Router {
)
}
}
if detailSvc != nil && deps.FileRepo != nil && deps.UserStoreProvider != nil {
audioHandler = handlers.NewAudioHandler(
detailSvc,
deps.FileRepo,
deps.UserStoreProvider,
handlers.NewPGAudioBookmarkStore(deps.DB),
)
}
// Wire subtitle repo and S3 client onto streamHandler for S3-stored subtitle serving.
if streamHandler != nil && subtitleRepo != nil && deps.S3Public != nil {
@@ -1000,6 +1011,7 @@ func NewRouter(deps Dependencies) chi.Router {
libraryCollectionService.TraktTokenResolver = &traktCollectionTokenResolver{
pool: deps.DB,
settings: settingsRepo,
cipher: deps.SecretCipher,
provider: watchtrakt.NewProvider(nil, ""),
}
}
@@ -1159,7 +1171,7 @@ func NewRouter(deps Dependencies) chi.Router {
var historyImportHandler *handlers.HistoryImportHandler
var historyImportSvc *historyimport.Service
if deps.DB != nil {
historyRepo := historyimport.NewRepository(deps.DB)
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyImportSvc = historyimport.NewService(deps.AppContext, historyRepo, deps.UserStoreProvider)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, providerIDRepo)
historyImportSvc.SetStableIdentityResolver(historyIdentity)
@@ -1168,7 +1180,7 @@ func NewRouter(deps Dependencies) chi.Router {
}
historyImportHandler = handlers.NewHistoryImportHandler(historyImportSvc)
if deps.UserStoreProvider != nil {
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB), historyRepo, deps.UserStoreProvider)
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB, deps.SecretCipher), historyRepo, deps.UserStoreProvider)
webhookSyncSvc.SetStableIdentityResolver(historyIdentity)
webhookSyncHandler = handlers.NewWebhookSyncHandler(webhookSyncSvc)
}
@@ -1767,6 +1779,22 @@ func NewRouter(deps Dependencies) chi.Router {
})
}
if audioHandler != nil {
r.Route("/audio", func(r chi.Router) {
r.Group(func(r chi.Router) {
r.Use(apimw.RequireProfile)
r.Post("/playback/start", audioHandler.HandleStartPlayback)
r.Patch("/playback/{session_id}/sync", audioHandler.HandleSyncPlayback)
r.Post("/playback/{session_id}/close", audioHandler.HandleClosePlayback)
r.Get("/items/{content_id}/bookmarks", audioHandler.HandleListBookmarks)
r.Post("/items/{content_id}/bookmarks", audioHandler.HandleCreateBookmark)
r.Delete("/items/{content_id}/bookmarks/{time_seconds}", audioHandler.HandleDeleteBookmark)
})
r.Get("/playback/{session_id}/tracks/{track_index}", audioHandler.HandleStreamTrack)
r.Head("/playback/{session_id}/tracks/{track_index}", audioHandler.HandleStreamTrack)
})
}
if watchTogetherHandler != nil {
r.Route("/watch-together", func(r chi.Router) {
r.Get("/rooms/{room_id}/ws", watchTogetherHandler.HandleRoomWebSocket)
@@ -2102,10 +2130,12 @@ func NewRouter(deps Dependencies) chi.Router {
// System inspection.
{
sysJWTSecret := ""
sysFFmpegPath := ""
if deps.Config != nil {
sysJWTSecret = deps.Config.Auth.JWTSecret
sysFFmpegPath = deps.Config.Playback.FFmpegPath
}
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret)
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret, sysFFmpegPath)
r.Route("/system", func(r chi.Router) {
r.Get("/build", systemHandler.HandleBuildInfo)
r.Get("/hw-accel", systemHandler.HandleHWAccel)
@@ -11,13 +11,15 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/watchsync"
watchtrakt "github.com/Silo-Server/silo-server/internal/watchsync/providers/trakt"
)
type traktCollectionTokenResolver struct {
pool *pgxpool.Pool
settings *catalog.ServerSettingsRepo
settings catalog.SettingsStore // encrypting decorator: decrypts trakt client_secret
cipher *secret.Cipher // decrypts watch_provider_connections tokens read via raw SQL
provider *watchtrakt.Provider
}
@@ -97,7 +99,7 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
LIMIT 1
`, profileID)
var conn watchsync.Connection
if err := row.Scan(
err := row.Scan(
&conn.ID,
&conn.Provider,
&conn.UserID,
@@ -107,24 +109,45 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
&conn.AccessToken,
&conn.RefreshToken,
&conn.TokenExpiresAt,
); err != nil {
)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return watchsync.Connection{}, errors.New("trakt connection not found for profile")
}
return watchsync.Connection{}, fmt.Errorf("load trakt connection: %w", err)
}
// This resolver reads watch_provider_connections directly (bypassing the
// watchsync repo), so it must apply the same at-rest decryption the repo
// does, bound to the same connection identity (watchsync.TokenAAD).
if conn.AccessToken, err = r.cipher.DecryptIfEncrypted(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt access token: %w", err)
}
if conn.RefreshToken, err = r.cipher.DecryptIfEncrypted(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt refresh token: %w", err)
}
return conn, nil
}
func (r *traktCollectionTokenResolver) updateTokens(ctx context.Context, conn watchsync.Connection) error {
_, err := r.pool.Exec(ctx, `
// Encrypt the refreshed tokens inline, bound to the connection identity, so
// the raw-SQL write matches what the watchsync repo (and this resolver's read
// path) store.
accessToken, err := r.cipher.Encrypt(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt refresh token: %w", err)
}
_, err = r.pool.Exec(ctx, `
UPDATE watch_provider_connections
SET access_token = $2,
refresh_token = $3,
token_expires_at = $4,
updated_at = now()
WHERE id = $1::uuid
`, conn.ID, conn.AccessToken, conn.RefreshToken, conn.TokenExpiresAt)
`, conn.ID, accessToken, refreshToken, conn.TokenExpiresAt)
if err != nil {
return fmt.Errorf("update refreshed trakt connection tokens: %w", err)
}
+8 -4
View File
@@ -13,16 +13,20 @@ import (
)
const (
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
)
// ABSConfigProvider implements abs.ConfigProvider using silo's server_settings
// table. The ABS JWT secret is generated once on first read and persisted for
// the lifetime of the deployment.
type ABSConfigProvider struct {
Settings *catalog.ServerSettingsRepo
// Settings is the encrypting settings decorator in production, so the ABS
// JWT secret (a SensitiveSettingKey) rests as ciphertext and is transparently
// decrypted here. Typed as the interface so both the raw repo and the
// decorator satisfy it.
Settings catalog.SettingsStore
// secretCache holds the generated secret so we only hit the DB once per
// process lifetime. Protected by mu.
+2 -2
View File
@@ -42,8 +42,8 @@ type ABSHandlerDeps struct {
Pool *pgxpool.Pool
Items *catalog.ItemRepository
Files *scanner.FileRepository
Settings *catalog.ServerSettingsRepo
Auth absAuthAdapter // see BuildABSHandler below
Settings catalog.SettingsStore // encrypting decorator in production
Auth absAuthAdapter // see BuildABSHandler below
AccessResolver abs.AccessResolver
// Recs is the recommendations repository used to power the
// /items/{id}/similar endpoint via embedding nearest-neighbor search.
+13 -27
View File
@@ -13,31 +13,27 @@ type ResolvedConnection struct {
}
// RequestIntegrationLookup resolves a soft-linked Requests integration to its
// base URL and encrypted api-key reference.
// base URL and (already-decrypted) api key.
type RequestIntegrationLookup interface {
Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error)
}
// SecretResolver resolves an encrypted api-key reference to its plaintext value.
type SecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error)
}
type ConnectionResolver struct {
requests RequestIntegrationLookup
secrets SecretResolver
}
func NewConnectionResolver(r RequestIntegrationLookup, s SecretResolver) *ConnectionResolver {
return &ConnectionResolver{requests: r, secrets: s}
func NewConnectionResolver(r RequestIntegrationLookup) *ConnectionResolver {
return &ConnectionResolver{requests: r}
}
// Resolve turns a stored Connection into concrete credentials. When the
// connection is linked to a Requests integration the live base URL + key ref are
// read from there; otherwise the connection's own fields are used. The api-key
// ref is then resolved to plaintext via the secrets resolver.
// connection is linked to a Requests integration the live base URL + key are
// read from there (the Requests repo decrypts it); otherwise the connection's
// own fields are used (the autoscan repo already decrypted api_key_ref on read).
// Either way the returned key is plaintext — there is no longer a separate
// secret-resolution step.
func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (ResolvedConnection, error) {
baseURL, apiKeyRef := c.BaseURL, c.APIKeyRef
baseURL, apiKey := c.BaseURL, c.APIKeyRef
// A pointer-to-empty-string request_integration_id is NOT a live link (it can
// arise from a both-NULL orphan or a stripped link). Treat empty/whitespace as
// "no link" so we don't call requests.Get("") and use the connection's own
@@ -46,21 +42,11 @@ func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (Resolv
if cr.requests == nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: no lookup configured", *c.RequestIntegrationID)
}
u, ref, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
u, key, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: %w", *c.RequestIntegrationID, err)
}
baseURL, apiKeyRef = u, ref
baseURL, apiKey = u, key
}
apiKey := strings.TrimSpace(apiKeyRef)
if cr.secrets != nil && apiKey != "" {
resolved, err := cr.secrets.Get(ctx, apiKey)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: resolve api key: %w", err)
}
if resolved = strings.TrimSpace(resolved); resolved != "" {
apiKey = resolved
}
}
return ResolvedConnection{BaseURL: baseURL, APIKey: apiKey}, nil
return ResolvedConnection{BaseURL: baseURL, APIKey: strings.TrimSpace(apiKey)}, nil
}
+20 -36
View File
@@ -6,9 +6,10 @@ import (
"testing"
)
// fakeRequestLookup implements RequestIntegrationLookup from a static map.
// fakeRequestLookup implements RequestIntegrationLookup from a static map. The
// returned apiKey is already plaintext (the Requests repo decrypts on read).
type fakeRequestLookup struct {
entries map[string]struct{ baseURL, apiKeyRef string }
entries map[string]struct{ baseURL, apiKey string }
err error
}
@@ -20,30 +21,17 @@ func (f fakeRequestLookup) Get(_ context.Context, id string) (string, string, er
if !ok {
return "", "", errors.New("integration not found: " + id)
}
return e.baseURL, e.apiKeyRef, nil
}
// fakeSecrets implements SecretResolver from a static map; unknown refs resolve
// to empty (so the ref itself is used).
type fakeSecrets struct {
values map[string]string
err error
}
func (f fakeSecrets) Get(_ context.Context, ref string) (string, error) {
if f.err != nil {
return "", f.err
}
return f.values[ref], nil
return e.baseURL, e.apiKey, nil
}
func TestResolveConnectionOwnCredentials(t *testing.T) {
secrets := fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
// The autoscan repo already decrypted api_key_ref, so the connection carries
// the literal key; Resolve returns it verbatim (trimmed).
r := NewConnectionResolver(fakeRequestLookup{})
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "ownref",
APIKeyRef: "OWNKEY",
})
if err != nil {
t.Fatalf("Resolve own: %v", err)
@@ -54,11 +42,10 @@ func TestResolveConnectionOwnCredentials(t *testing.T) {
}
func TestResolveConnectionLinked(t *testing.T) {
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKeyRef string }{
"req-1": {baseURL: "http://req:7878", apiKeyRef: "rk"},
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKey string }{
"req-1": {baseURL: "http://req:7878", apiKey: "REQKEY"},
}}
secrets := fakeSecrets{values: map[string]string{"rk": "REQKEY"}}
r := NewConnectionResolver(lookup, secrets)
r := NewConnectionResolver(lookup)
id := "req-1"
got, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id})
@@ -71,7 +58,7 @@ func TestResolveConnectionLinked(t *testing.T) {
}
func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{}, fakeSecrets{})
r := NewConnectionResolver(fakeRequestLookup{})
id := "gone"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when linked Requests integration is missing")
@@ -79,7 +66,7 @@ func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
}
func TestResolveConnectionLinkedLookupErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")}, fakeSecrets{})
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")})
id := "req-1"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when the lookup itself fails")
@@ -92,13 +79,12 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
// requests.Get(""). The lookup here errors on any call, so reaching it fails.
r := NewConnectionResolver(
fakeRequestLookup{err: errors.New("lookup must not be called")},
fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}},
)
for _, empty := range []string{"", " "} {
id := empty
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "ownref",
APIKeyRef: "OWNKEY",
RequestIntegrationID: &id,
})
if err != nil {
@@ -110,20 +96,18 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
}
}
func TestResolveConnectionTrimsAndFallsBackOnEmptySecret(t *testing.T) {
// Whitespace-padded ref; secret resolves to whitespace-only -> fall back to
// the trimmed ref (parity with requests.resolveAPIKey).
secrets := fakeSecrets{values: map[string]string{"ownref": " "}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
func TestResolveConnectionTrimsKey(t *testing.T) {
// Resolve trims surrounding whitespace from the (decrypted) key.
r := NewConnectionResolver(fakeRequestLookup{})
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: " ownref ",
APIKeyRef: " OWNKEY ",
})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if got.APIKey != "ownref" {
t.Fatalf("expected trimmed ref fallback, got %q", got.APIKey)
if got.APIKey != "OWNKEY" {
t.Fatalf("expected trimmed key, got %q", got.APIKey)
}
}
+57 -11
View File
@@ -9,14 +9,38 @@ import (
"time"
"unicode/utf8"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
type Repository struct{ pool *pgxpool.Pool }
type Repository struct {
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository { return &Repository{pool: pool} }
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// connectionAPIKeyAAD binds an autoscan_connections api_key_ref ciphertext to
// its row id.
func connectionAPIKeyAAD(id string) string {
return secret.RowAAD("autoscan_connections", "api_key_ref", id)
}
// encryptAPIKey encrypts a non-empty, trimmed key bound to the connection id;
// an empty key returns "" so NULL/keep-existing semantics are preserved.
func (r *Repository) encryptAPIKey(id, apiKey string) (string, error) {
apiKey = strings.TrimSpace(apiKey)
if apiKey == "" {
return "", nil
}
return r.cipher.Encrypt(apiKey, connectionAPIKeyAAD(id))
}
// --- Settings ---
@@ -64,7 +88,7 @@ func (r *Repository) UpdateSettings(ctx context.Context, s Settings) (Settings,
const connectionColumns = `id, name, kind, base_url, api_key_ref, request_integration_id`
func scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
func (r *Repository) scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
var c Connection
var baseURL, apiKeyRef, reqIntegrationID *string
if err := row.Scan(&c.ID, &c.Name, &c.Kind, &baseURL, &apiKeyRef, &reqIntegrationID); err != nil {
@@ -77,6 +101,13 @@ func scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
c.APIKeyRef = *apiKeyRef
}
c.RequestIntegrationID = reqIntegrationID
// Decrypt the stored key (read-path contract): legacy plaintext passes
// through, enc:v1: decrypts, corrupt ciphertext errors.
apiKey, err := r.cipher.DecryptIfEncrypted(c.APIKeyRef, connectionAPIKeyAAD(c.ID))
if err != nil {
return Connection{}, fmt.Errorf("decrypt autoscan connection %s api key: %w", c.ID, err)
}
c.APIKeyRef = apiKey
return c, nil
}
@@ -89,12 +120,22 @@ func nullable(s string) any {
}
func (r *Repository) CreateConnection(ctx context.Context, c Connection) (Connection, error) {
// Generate the id in Go (rather than the DB default) so the api_key_ref
// ciphertext can be AAD-bound to the row id at insert time.
id := strings.TrimSpace(c.ID)
if id == "" {
id = uuid.NewString()
}
apiKeyRef, err := r.encryptAPIKey(id, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_connections (name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5)
INSERT INTO autoscan_connections (id, name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING `+connectionColumns,
c.Name, c.Kind, nullable(c.BaseURL), nullable(c.APIKeyRef), c.RequestIntegrationID)
out, err := scanConnection(row)
id, c.Name, c.Kind, nullable(c.BaseURL), nullable(apiKeyRef), c.RequestIntegrationID)
out, err := r.scanConnection(row)
if err != nil {
return Connection{}, fmt.Errorf("create autoscan connection: %w", err)
}
@@ -108,6 +149,11 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
// next poll. Mirrors requests.UpdateIntegration's CASE-WHEN keep-semantics.
// Pass the raw trimmed string (not nullable()) so the empty-string sentinel
// reaches the CASE.
// Encrypt the incoming key; an empty result preserves the keep-existing CASE.
apiKeyRef, err := r.encryptAPIKey(c.ID, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
UPDATE autoscan_connections
SET name = $2, kind = $3, base_url = $4,
@@ -115,8 +161,8 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
request_integration_id = $6, updated_at = now()
WHERE id = $1
RETURNING `+connectionColumns,
c.ID, c.Name, c.Kind, nullable(c.BaseURL), c.APIKeyRef, c.RequestIntegrationID)
out, err := scanConnection(row)
c.ID, c.Name, c.Kind, nullable(c.BaseURL), apiKeyRef, c.RequestIntegrationID)
out, err := r.scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, c.ID)
@@ -151,7 +197,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
defer rows.Close()
var out []Connection
for rows.Next() {
c, err := scanConnection(rows)
c, err := r.scanConnection(rows)
if err != nil {
return nil, err
}
@@ -163,7 +209,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
func (r *Repository) GetConnection(ctx context.Context, id string) (Connection, error) {
row := r.pool.QueryRow(ctx, `SELECT `+connectionColumns+`
FROM autoscan_connections WHERE id = $1`, id)
c, err := scanConnection(row)
c, err := r.scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, id)
+138 -136
View File
@@ -28,65 +28,35 @@ type RatingFilter struct {
Min float64
// Limit caps the number of rows returned. Zero or negative means no limit.
Limit int
// LibraryID, when non-nil, restricts results to items in that library via
// the media_item_libraries junction table.
// LibraryID, when non-nil, restricts results to items in that library.
LibraryID *int
// Filter carries viewer-level access constraints (content rating ceiling,
// allowed/disabled library sets).
Filter AccessFilter
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
// buildRatingThresholdQuery builds the SQL statement and bind args for ListByRatingThreshold.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold — NULL ratings are excluded implicitly by >=.
// IMDb rating threshold - NULL ratings are excluded implicitly by >=.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.Min)
argIdx++
// Optional single-library pin.
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id = $%d", argIdx))
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, f.LibraryID, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
@@ -95,6 +65,18 @@ func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f Ratin
args = append(args, f.Limit)
}
return query, args
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
query, args := buildRatingThresholdQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing items by rating threshold: %w", err)
@@ -122,6 +104,50 @@ type UnplayedFilter struct {
Filter AccessFilter
}
// buildUnplayedHighRatedQuery builds the SQL statement and bind args for ListUnplayedHighRated.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// Items the user has any watch history entry for are excluded.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
}
return query, args
}
// ListUnplayedHighRated returns high-rated items that the given user/profile has
// never started watching. "Never started" means no row exists in
// user_watch_history for (user_id, profile_id, media_item_id), regardless of
@@ -133,67 +159,9 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
return nil, fmt.Errorf("ListUnplayedHighRated: UserID and ProfileID are required")
}
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// LEFT JOIN exclusion: items the user has any watch history entry for are excluded.
// We use a NOT EXISTS subquery rather than an outer join so it composes cleanly
// with library access conditions on the main query.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
query, args := buildUnplayedHighRatedQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
@@ -225,13 +193,10 @@ type ForgottenFavoritesFilter struct {
Filter AccessFilter
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
// buildForgottenFavoritesQuery builds the SQL statement and bind args for ListForgottenFavorites.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
@@ -255,39 +220,15 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
@@ -296,6 +237,21 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
args = append(args, f.Limit)
}
return query, args
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
query, args := buildForgottenFavoritesQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing forgotten favorites: %w", err)
@@ -308,3 +264,49 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
}
return items, nil
}
func appendDiscoveryLibraryScope(
conditions *[]string,
args *[]any,
argIdx *int,
libraryID *int,
filter AccessFilter,
) bool {
if libraryID != nil {
whereSQL, scopeArgs, ok := buildLibraryScopeJoin([]int{*libraryID}, nil, *argIdx, "", "mi.content_id")
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
if filter.AllowedLibraryIDs != nil && len(filter.AllowedLibraryIDs) == 0 {
return false
}
// buildLibraryScopeJoin uses semi-joins: disabled libraries are an item-level
// exclusion, matching the rest of catalog access filtering and avoiding row fanout.
if filter.AllowedLibraryIDs == nil && len(filter.DisabledLibraryIDs) > 0 {
// In deny-only mode, require at least one library membership. Without this,
// stale or provisional media_items rows with no membership pass NOT EXISTS.
*conditions = append(*conditions,
"EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)",
)
}
whereSQL, scopeArgs, ok := buildLibraryScopeJoin(
filter.AllowedLibraryIDs,
filter.DisabledLibraryIDs,
*argIdx,
"",
"mi.content_id",
)
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
+146 -224
View File
@@ -4,139 +4,12 @@ package catalog
// discovery_repo.go using the same pure-unit approach used throughout this
// package: build a query string and assert that key fragments are present,
// without needing a live database.
//
// The helpers themselves execute against a real pool; integration coverage is
// provided by whichever recipe resolver exercises them in higher-level tests.
import (
"strings"
"testing"
)
// buildRatingThresholdQuery is the testable extraction of the SQL generation
// logic from ListByRatingThreshold so we can inspect it without a pool.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.Min)
argIdx++
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, "mil.media_folder_id = $2")
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// buildUnplayedHighRatedQuery mirrors the SQL-building portion of
// ListUnplayedHighRated for unit testing.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.MinRating)
argIdx++
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t)")
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// itoa is a local helper to avoid importing strconv.
func itoa(n int) string {
if n == 0 {
return "0"
}
buf := [20]byte{}
pos := len(buf)
for n > 0 {
pos--
buf[pos] = byte('0' + n%10)
n /= 10
}
return string(buf[pos:])
}
// ---------------------------------------------------------------------------
// ListByRatingThreshold SQL generation tests
// ---------------------------------------------------------------------------
@@ -147,6 +20,7 @@ func TestRatingThreshold_BasicQuery(t *testing.T) {
Limit: 20,
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -181,25 +55,24 @@ func TestRatingThreshold_NoLimitOmitsClause(t *testing.T) {
}
}
func TestRatingThreshold_LibraryIDJoin(t *testing.T) {
func TestRatingThreshold_LibraryIDUsesSemiJoin(t *testing.T) {
libID := 42
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
LibraryID: &libID,
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id = $2") {
t.Fatalf("expected media_folder_id predicate at $2, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected library id predicate at $2, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args (min + library id), got %d", len(args))
}
if args[1] != 42 {
t.Fatalf("expected library id 42 at args[1], got %v", args[1])
t.Fatalf("expected 2 args (min + library id slice), got %d", len(args))
}
assertIntSliceArg(t, args, 1, []int{42})
}
func TestRatingThreshold_AllowedLibraries(t *testing.T) {
@@ -210,18 +83,20 @@ func TestRatingThreshold_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($2, $3)") {
t.Fatalf("expected allowed library IN clause, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected allowed library ANY clause, got:\n%s", query)
}
if len(args) != 3 {
t.Fatalf("expected 3 args, got %d: %v", len(args), args)
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{1, 2})
}
func TestRatingThreshold_DisabledLibraries(t *testing.T) {
func TestRatingThreshold_DisabledLibrariesUseItemLevelExclusion(t *testing.T) {
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
@@ -229,19 +104,24 @@ func TestRatingThreshold_DisabledLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "mil.media_folder_id NOT IN ($2)") {
t.Fatalf("expected disabled library NOT IN clause, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_out.media_folder_id = ANY($2)") {
t.Fatalf("expected disabled library ANY clause, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{99})
}
func TestRatingThreshold_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
query, _ := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
AllowedLibraryIDs: []int{}, // empty slice → early return
AllowedLibraryIDs: []int{},
},
})
if query != "" {
@@ -261,6 +141,7 @@ func TestUnplayedHighRated_BasicQuery(t *testing.T) {
ProfileID: "profile-abc",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -301,14 +182,14 @@ func TestUnplayedHighRated_NoLimitOmitsClause(t *testing.T) {
}
}
func TestUnplayedHighRated_NoLibraryJoin(t *testing.T) {
func TestUnplayedHighRated_NoLibraryScope(t *testing.T) {
query, _ := buildUnplayedHighRatedQuery(UnplayedFilter{
MinRating: 7.0,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
}
}
@@ -322,15 +203,17 @@ func TestUnplayedHighRated_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
}
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestUnplayedHighRated_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -359,8 +242,6 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
t.Fatalf("expected content_rating = ANY filter, got:\n%s", query)
}
// args: minRating, userID, profileID, then content rating slice (one arg).
// Task 5.1 converted the rating ladder from IN ($N, $N+1, ...) to
// = ANY($N) with a single bound slice arg.
if len(args) != 4 {
t.Fatalf("expected exactly 4 args (minRating, userID, profileID, rating slice); got %d: %v", len(args), args)
}
@@ -370,62 +251,6 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
// ListForgottenFavorites SQL generation tests
// ---------------------------------------------------------------------------
// buildForgottenFavoritesQuery is the testable extraction of the SQL
// generation logic from ListForgottenFavorites.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= 7.0")
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t\t AND uwh.watched_at >= NOW() - ($"+itoa(argIdx+2)+" || ' days')::interval\n\t)")
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
func TestForgottenFavorites_BasicQuery(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
@@ -434,6 +259,7 @@ func TestForgottenFavorites_BasicQuery(t *testing.T) {
ProfileID: "profile-xyz",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= 7.0") {
t.Fatalf("expected 7.0 rating floor, got:\n%s", query)
}
@@ -477,14 +303,14 @@ func TestForgottenFavorites_NoLimitOmitsClause(t *testing.T) {
}
}
func TestForgottenFavorites_NoLibraryJoin(t *testing.T) {
func TestForgottenFavorites_NoLibraryScope(t *testing.T) {
query, _ := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
}
}
@@ -498,15 +324,17 @@ func TestForgottenFavorites_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
}
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -525,14 +353,13 @@ func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T)
func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 0, // should default to 365
LookbackDays: 0,
UserID: 1,
ProfileID: "p",
})
if !strings.Contains(query, "NOT EXISTS") {
t.Fatalf("expected NOT EXISTS subquery, got:\n%s", query)
}
// args[2] should be the defaulted lookback value (365).
if len(args) < 3 {
t.Fatalf("expected at least 3 args, got %d", len(args))
}
@@ -540,3 +367,98 @@ func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
t.Fatalf("expected lookback default 365, got %v", args[2])
}
}
func TestDiscoveryQueries_QualifyContentIDAndAvoidLibraryJoin(t *testing.T) {
restricted := AccessFilter{AllowedLibraryIDs: []int{1, 2}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: restricted}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: restricted}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: restricted}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertQualifiedItemSelect(t, tc.query)
assertNoLibraryJoin(t, tc.query)
if !strings.Contains(tc.query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", tc.query)
}
if strings.Contains(tc.query, "SELECT content_id,") {
t.Fatalf("SELECT list uses an unqualified content_id, got:\n%s", tc.query)
}
})
}
}
func TestDiscoveryQueries_DisabledLibrariesUseNotExists(t *testing.T) {
filter := AccessFilter{DisabledLibraryIDs: []int{9}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: filter}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: filter}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: filter}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertNoLibraryJoin(t, tc.query)
assertDenyOnlyRequiresLibraryMembership(t, tc.query)
if !strings.Contains(tc.query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", tc.query)
}
})
}
}
func assertQualifiedItemSelect(t *testing.T, query string) {
t.Helper()
if !strings.HasPrefix(query, "SELECT mi.content_id,") {
t.Fatalf("SELECT list is not qualified with the mi alias, got:\n%s", query)
}
}
func assertNoLibraryJoin(t *testing.T, query string) {
t.Helper()
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("plain library JOIN can fan out discovery rows, got:\n%s", query)
}
}
func assertDenyOnlyRequiresLibraryMembership(t *testing.T, query string) {
t.Helper()
membershipPredicate := "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)"
if !strings.Contains(query, membershipPredicate) {
t.Fatalf("deny-only library filters must require positive library membership, got:\n%s", query)
}
if strings.Contains(query, "mil_scope_any.media_folder_id") {
t.Fatalf("positive membership predicate should not bind a specific library, got:\n%s", query)
}
}
func assertIntSliceArg(t *testing.T, args []any, idx int, want []int) {
t.Helper()
if idx >= len(args) {
t.Fatalf("missing arg %d in %v", idx, args)
}
got, ok := args[idx].([]int)
if !ok {
t.Fatalf("arg %d has type %T, want []int", idx, args[idx])
}
if len(got) != len(want) {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
}
}
func mustQuery(query string, _ []any) string { return query }
+194
View File
@@ -0,0 +1,194 @@
package catalog
import (
"context"
"errors"
"fmt"
"sort"
"github.com/Silo-Server/silo-server/internal/secret"
)
// SettingsStore is the read/write surface over server_settings shared by the
// raw *ServerSettingsRepo and the *EncryptedSettingsRepo decorator. Consumers
// depend on this interface (rather than the concrete repo) so they
// transparently gain at-rest encryption for sensitive keys once the decorator
// is wired in.
type SettingsStore interface {
Get(ctx context.Context, key string) (string, error)
Set(ctx context.Context, key, value string) error
GetAll(ctx context.Context) (map[string]string, error)
}
// SensitiveSettingKeys is the single source of truth for which server_settings
// keys hold secrets. It drives BOTH at-rest encryption (the decorator encrypts
// these on write, decrypts them on read) AND admin-API redaction, so the two
// can never drift apart.
//
// It was audited from the config loader's real inputs
// (internal/config/db_loader.go), the admin redaction map, and the other known
// server_settings consumers — the Audiobookshelf-compat JWT secret
// (internal/audiobooks/config.go), the watch-sync OAuth client config
// (internal/watchsync/service.go), and the legacy single-instance arr keys —
// including the legacy aliases still read as fallbacks (s3.operational_*,
// recommendations.openai_api_key) and redis.sentinel_password, which the old
// redaction map omitted.
//
// Adding a key here is only safe when every reader of that key goes through a
// cipher-aware path (the decorator, or a repo that calls
// secret.Cipher.DecryptIfEncrypted with secret.SettingsAAD). A raw SQL reader
// would otherwise see ciphertext after the backfill.
var SensitiveSettingKeys = map[string]bool{
// Core auth signing secret — no longer the encryption root (SECRET_KEY is),
// and now itself encrypted at rest under SECRET_KEY.
"auth.jwt_secret": true,
// Audiobookshelf-compat HMAC signing key (generated + persisted as hex).
"audiobooks.abs.jwt_secret": true,
// S3 — public assets, private internal, user DB, plus the legacy
// "operational" aliases db_loader still reads as fallbacks (L155-192;
// migration 086 copied but did not delete them).
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"s3.operational_access_key": true,
"s3.operational_secret_key": true,
"s3.operational_token_secret": true,
// Redis — url may embed credentials (redis://:pass@host); sentinel_password
// is read at db_loader L334 and was MISSING from the old redaction map.
"redis.url": true,
"redis.sentinel_password": true,
// Metadata / list-provider API keys.
"tmdb.api_key": true,
"mdblist.api_key": true,
"introdb.api_key": true,
// Subtitle AI (on-demand translation) API key.
"subtitle_ai.api_key": true,
// Recommendations embedding auth (+ legacy openai alias at db_loader L409).
"recommendations.embedding_auth_token": true,
"recommendations.openai_api_key": true,
// Watch-sync OAuth client credentials. The client_id entries are low-value
// but kept as a harmless safe superset and to match existing redaction.
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
// Legacy single-instance arr keys (superseded by per-integration rows, but
// still referenced by older request_integrations rows until backfilled).
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
}
// EncryptedSettingsRepo decorates a raw settings store, transparently
// encrypting the SensitiveSettingKeys on write and decrypting them on read.
// config.LoadFromDB and every other consumer keep seeing plaintext while the
// values rest as ciphertext in the database, GCM-bound to their key.
type EncryptedSettingsRepo struct {
// inner is the RAW store (no encryption). It must never be another
// EncryptedSettingsRepo, or writes would double-encrypt.
inner SettingsStore
cipher *secret.Cipher
}
// NewEncryptedSettingsRepo wraps a raw settings store with the cipher. Pass the
// concrete *ServerSettingsRepo (or an in-memory fake in tests) as inner.
func NewEncryptedSettingsRepo(inner SettingsStore, cipher *secret.Cipher) *EncryptedSettingsRepo {
return &EncryptedSettingsRepo{inner: inner, cipher: cipher}
}
var _ SettingsStore = (*EncryptedSettingsRepo)(nil)
// Set encrypts a sensitive, non-empty value (AAD-bound to its key) before
// delegating to the raw store; non-sensitive keys and empty values delegate
// unchanged. Encrypting an empty value is a no-op, so an absent secret stays an
// empty column rather than an envelope.
func (r *EncryptedSettingsRepo) Set(ctx context.Context, key, value string) error {
if SensitiveSettingKeys[key] && value != "" {
ct, err := r.cipher.Encrypt(value, secret.SettingsAAD(key))
if err != nil {
return fmt.Errorf("encrypt setting %q: %w", key, err)
}
value = ct
}
return r.inner.Set(ctx, key, value)
}
// Get reads a value and applies the read-path contract: legacy plaintext passes
// through, an enc:v1: value is decrypted, and a corrupt ciphertext errors.
func (r *EncryptedSettingsRepo) Get(ctx context.Context, key string) (string, error) {
value, err := r.inner.Get(ctx, key)
if err != nil {
return "", err
}
out, err := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if err != nil {
return "", fmt.Errorf("decrypt setting %q: %w", key, err)
}
return out, nil
}
// GetAll reads every setting and decrypts any enc:v1: value in place, so
// callers (notably config.LoadFromDB) receive a fully plaintext map.
func (r *EncryptedSettingsRepo) GetAll(ctx context.Context) (map[string]string, error) {
all, err := r.inner.GetAll(ctx)
if err != nil {
return nil, err
}
for key, value := range all {
out, derr := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if derr != nil {
return nil, fmt.Errorf("decrypt setting %q: %w", key, derr)
}
all[key] = out
}
return all, nil
}
// BackfillSensitiveSettings encrypts any plaintext value currently stored under
// a sensitive key. It is idempotent (already-encrypted and empty values are
// skipped) and best-effort: per-key failures are collected and returned but do
// not abort the sweep, so one unreadable row cannot block startup. Returns the
// number of keys newly encrypted.
func (r *EncryptedSettingsRepo) BackfillSensitiveSettings(ctx context.Context) (int, error) {
keys := make([]string, 0, len(SensitiveSettingKeys))
for key := range SensitiveSettingKeys {
keys = append(keys, key)
}
sort.Strings(keys) // deterministic order for logs and tests
encrypted := 0
var errs []error
for _, key := range keys {
// Read the RAW inner value (not through this decorator) so we can tell
// plaintext from an already-encrypted envelope.
raw, err := r.inner.Get(ctx, key)
if err != nil {
errs = append(errs, fmt.Errorf("read %q: %w", key, err))
continue
}
if raw == "" || secret.IsEncrypted(raw) {
continue
}
ct, err := r.cipher.Encrypt(raw, secret.SettingsAAD(key))
if err != nil {
errs = append(errs, fmt.Errorf("encrypt %q: %w", key, err))
continue
}
if err := r.inner.Set(ctx, key, ct); err != nil {
errs = append(errs, fmt.Errorf("write %q: %w", key, err))
continue
}
encrypted++
}
return encrypted, errors.Join(errs...)
}
@@ -0,0 +1,210 @@
package catalog
import (
"context"
"crypto/rand"
"testing"
"github.com/Silo-Server/silo-server/internal/secret"
)
// memSettings is an in-memory raw SettingsStore for DB-free decorator tests.
type memSettings struct{ m map[string]string }
func newMemSettings() *memSettings { return &memSettings{m: map[string]string{}} }
func (s *memSettings) Get(_ context.Context, key string) (string, error) { return s.m[key], nil }
func (s *memSettings) Set(_ context.Context, key, value string) error {
s.m[key] = value
return nil
}
func (s *memSettings) GetAll(_ context.Context) (map[string]string, error) {
out := make(map[string]string, len(s.m))
for k, v := range s.m {
out[k] = v
}
return out, nil
}
func newCipher(t *testing.T) *secret.Cipher {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return c
}
func pickSensitiveKey() string {
for k := range SensitiveSettingKeys {
return k
}
return "auth.jwt_secret"
}
func TestEncryptedSettings_SensitiveRoundTrip(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "tmdb.api_key"
const plain = "tmdb-secret-key-value"
if err := dec.Set(ctx, key, plain); err != nil {
t.Fatalf("Set: %v", err)
}
// Raw store must hold ciphertext.
if got := raw.m[key]; !secret.IsEncrypted(got) {
t.Fatalf("raw store value = %q, want enc:v1: ciphertext", got)
}
// Decorator Get must return plaintext.
got, err := dec.Get(ctx, key)
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != plain {
t.Fatalf("Get = %q, want %q", got, plain)
}
}
func TestEncryptedSettings_NonSensitivePassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "server.log_level"
if SensitiveSettingKeys[key] {
t.Fatalf("precondition: %q must be non-sensitive", key)
}
if err := dec.Set(ctx, key, "debug"); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m[key]; got != "debug" {
t.Fatalf("non-sensitive raw value = %q, want plaintext 'debug'", got)
}
}
func TestEncryptedSettings_LegacyPlaintextPassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
// Simulate a pre-encryption row: a sensitive key holding raw plaintext.
raw.m["tmdb.api_key"] = "legacy-plaintext"
got, err := dec.Get(ctx, "tmdb.api_key")
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != "legacy-plaintext" {
t.Fatalf("legacy plaintext Get = %q, want pass-through", got)
}
}
func TestEncryptedSettings_EmptySensitiveStaysEmpty(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", ""); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m["tmdb.api_key"]; got != "" {
t.Fatalf("empty sensitive value stored as %q, want empty (no envelope)", got)
}
}
func TestEncryptedSettings_GetAllDecrypts(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", "k1"); err != nil {
t.Fatalf("Set: %v", err)
}
if err := dec.Set(ctx, "server.mode", "integrated"); err != nil {
t.Fatalf("Set: %v", err)
}
all, err := dec.GetAll(ctx)
if err != nil {
t.Fatalf("GetAll: %v", err)
}
if all["tmdb.api_key"] != "k1" {
t.Fatalf("GetAll sensitive = %q, want decrypted 'k1'", all["tmdb.api_key"])
}
if all["server.mode"] != "integrated" {
t.Fatalf("GetAll non-sensitive = %q", all["server.mode"])
}
}
// TestSensitiveSettingKeys_Audited locks the audited allowlist: a dropped key is
// a plaintext leak (and breaks redaction), so the critical secrets must stay
// present, while values that are NOT secrets must stay absent (encrypting them
// would corrupt config reads).
func TestSensitiveSettingKeys_Audited(t *testing.T) {
mustHave := []string{
"auth.jwt_secret",
"audiobooks.abs.jwt_secret",
"s3.public_secret_key",
"s3.operational_secret_key", // legacy alias still read as a fallback
"redis.sentinel_password", // was missing from the old redaction map
"recommendations.openai_api_key",
"tmdb.api_key",
"requests.radarr.api_key",
"requests.sonarr.api_key",
"watchsync.trakt.client_secret",
}
for _, k := range mustHave {
if !SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys missing critical secret key %q", k)
}
}
mustNotHave := []string{
"server.mode",
"server.log_level",
"database.max_connections",
"jellyfin_compat.server_id",
}
for _, k := range mustNotHave {
if SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys must not contain non-secret key %q", k)
}
}
}
func TestEncryptedSettings_BackfillIdempotent(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
key := pickSensitiveKey()
raw.m[key] = "plaintext-secret" // legacy row
raw.m["server.mode"] = "integrated" // non-sensitive, must be left alone
n, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill: %v", err)
}
if n != 1 {
t.Fatalf("backfill encrypted %d, want 1", n)
}
if !secret.IsEncrypted(raw.m[key]) {
t.Fatalf("after backfill raw[%q] = %q, want ciphertext", key, raw.m[key])
}
if raw.m["server.mode"] != "integrated" {
t.Fatalf("backfill mutated non-sensitive key: %q", raw.m["server.mode"])
}
// Decryptable back to the original.
if got, _ := dec.Get(ctx, key); got != "plaintext-secret" {
t.Fatalf("post-backfill Get = %q, want original", got)
}
// Second run is a no-op.
n2, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill 2: %v", err)
}
if n2 != 0 {
t.Fatalf("second backfill encrypted %d, want 0 (no double-encrypt)", n2)
}
}
+13
View File
@@ -3,6 +3,7 @@ package catalog
import (
"context"
"errors"
"strings"
"testing"
"time"
@@ -136,3 +137,15 @@ func TestDeleteInBatchesReturnsError(t *testing.T) {
t.Fatalf("expected sentinel, got %v", err)
}
}
func TestLateOrphanCleanupUsesProvisionalSafetyPredicate(t *testing.T) {
conditions := normalizePredicateSQL(orphanedProvisionalMediaItemConditions)
for name, query := range map[string]string{
"collect": collectOrphanedProvisionalIDsByContentIDSQL,
"delete": deleteOrphanedProvisionalIDsByContentIDSQL,
} {
if !strings.Contains(normalizePredicateSQL(query), conditions) {
t.Fatalf("%s late orphan query does not use provisional safety predicate", name)
}
}
}
+142 -22
View File
@@ -563,16 +563,6 @@ func (r *FolderRepository) DeleteWithStats(
}
}
// Filter accumulated image dirs once, now that orphans are gone, against
// any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
}
stats.OrphanedImageDirs = filtered
}
// Phase 2: delete this folder's media_files (folder-tied, not item-tied).
if progress != nil {
progress(orphanTotal, orphanTotal, "Deleting media files")
@@ -592,23 +582,30 @@ func (r *FolderRepository) DeleteWithStats(
}
// Phase 3: delete remaining folder memberships (shared items kept; only the
// membership in this folder is removed).
// membership in this folder is removed). Deleting the memberships with
// RETURNING lets us catch skeleton items that a matcher created after the
// phase-1 orphan sweep but before the delete job reached this phase.
if progress != nil {
progress(orphanTotal, orphanTotal, "Removing library memberships")
}
if _, err := deleteInBatches(ctx, folderChildDeleteBatch, func(ctx context.Context) (int64, error) {
tag, e := r.pool.Exec(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)`, id, folderChildDeleteBatch)
if e != nil {
return 0, e
}
return tag.RowsAffected(), nil
}); err != nil {
lateOrphans, lateImageDirs, err := r.deleteFolderMemberships(ctx, id)
if err != nil {
return nil, fmt.Errorf("deleting media item links: %w", err)
}
stats.OrphanedItems += lateOrphans
for _, d := range lateImageDirs {
rawDirs[d] = struct{}{}
}
// Filter accumulated image dirs once, now that all item orphans are gone,
// against any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
}
stats.OrphanedImageDirs = filtered
}
// Phase 4: delete the now-lightweight folder row. Tolerate 0 rows so a
// resumed run that already removed it still succeeds.
@@ -656,6 +653,129 @@ func (r *FolderRepository) collectOrphanBatch(ctx context.Context, folderID, lim
return ids, nil
}
// deleteFolderMemberships removes this folder's remaining item memberships and
// deletes any content that becomes orphaned as a result. This is intentionally
// separate from the phase-1 orphan sweep because in-flight matchers can create
// new skeleton memberships after phase 1 has already drained.
func (r *FolderRepository) deleteFolderMemberships(ctx context.Context, folderID int) (int, []string, error) {
var orphaned int
var imageDirs []string
for {
var contentIDs []string
if err := retryOnDeadlock(ctx, func() error {
ids, err := r.deleteFolderMembershipBatch(ctx, folderID, folderChildDeleteBatch)
if err != nil {
return err
}
contentIDs = ids
return nil
}); err != nil {
return orphaned, imageDirs, err
}
if len(contentIDs) > 0 {
deleted, dirs, err := r.deleteOrphanedItemsByContentID(ctx, contentIDs)
if err != nil {
return orphaned, imageDirs, err
}
orphaned += deleted
imageDirs = append(imageDirs, dirs...)
}
if len(contentIDs) < folderChildDeleteBatch {
return orphaned, imageDirs, nil
}
}
}
func (r *FolderRepository) deleteFolderMembershipBatch(ctx context.Context, folderID, limit int) ([]string, error) {
rows, err := r.pool.Query(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)
RETURNING content_id`, folderID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var contentIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning deleted membership content_id: %w", err)
}
contentIDs = append(contentIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return contentIDs, nil
}
const collectOrphanedProvisionalIDsByContentIDSQL = `
SELECT mi.content_id
FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
const deleteOrphanedProvisionalIDsByContentIDSQL = `
DELETE FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
func (r *FolderRepository) deleteOrphanedItemsByContentID(ctx context.Context, contentIDs []string) (int, []string, error) {
if len(contentIDs) == 0 {
return 0, nil, nil
}
orphanIDs, err := r.collectOrphanedProvisionalIDsByContentID(ctx, contentIDs)
if err != nil {
return 0, nil, err
}
if len(orphanIDs) == 0 {
return 0, nil, nil
}
imageDirs, err := collectImageDirs(ctx, r.pool, orphanIDs)
if err != nil {
return 0, nil, err
}
var deleted int64
if err := retryOnDeadlock(ctx, func() error {
tag, err := r.pool.Exec(ctx, deleteOrphanedProvisionalIDsByContentIDSQL, orphanIDs)
if err != nil {
return err
}
deleted = tag.RowsAffected()
return nil
}); err != nil {
return 0, nil, err
}
return int(deleted), imageDirs, nil
}
func (r *FolderRepository) collectOrphanedProvisionalIDsByContentID(ctx context.Context, contentIDs []string) ([]string, error) {
rows, err := r.pool.Query(ctx, collectOrphanedProvisionalIDsByContentIDSQL, contentIDs)
if err != nil {
return nil, err
}
defer rows.Close()
var orphanIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning orphan content_id: %w", err)
}
orphanIDs = append(orphanIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return orphanIDs, nil
}
// dirSetToSlice returns the keys of set as a slice (nil if empty).
func dirSetToSlice(set map[string]struct{}) []string {
if len(set) == 0 {
+200
View File
@@ -0,0 +1,200 @@
package catalog
import (
"context"
"fmt"
"github.com/jackc/pgx/v5/pgxpool"
)
const defaultOrphanedProvisionalCleanupBatchSize = 1000
const orphanedMediaItemSafetyConditions = `NOT EXISTS (
SELECT 1 FROM public.media_item_libraries mil
WHERE mil.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.media_files mf
WHERE mf.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.episodes e
WHERE e.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.seasons s
WHERE s.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.library_collection_items lci
WHERE lci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_bookmarks ab
WHERE ab.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_playback_sessions aps
WHERE aps.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_rss_feeds arf
WHERE arf.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.downloads d
WHERE d.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.playback_history_admin pha
WHERE pha.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_bindings psib
WHERE psib.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_state psis
WHERE psis.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.podcast_feeds pf
WHERE pf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_favorites uf
WHERE uf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_downloads ud
WHERE ud.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_history_hidden_items uhhi
WHERE uhhi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid
WHERE uhid.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid_series
WHERE uhid_series.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_audio_preferences uap
WHERE uap.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_personal_collection_items upci
WHERE upci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_ratings ur
WHERE ur.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_series_playback_preferences uspp
WHERE uspp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_subtitle_preferences usp
WHERE usp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_history uwh
WHERE uwh.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_progress uwp
WHERE uwp.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watchlist uwl
WHERE uwl.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_favorite_items wpfi
WHERE wpfi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_history_exports wphe
WHERE wphe.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_scrobble_sessions wpss
WHERE wpss.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_rooms wtr
WHERE wtr.selected_content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_suggestions wts
WHERE wts.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.webhook_sync_item_state wsis
WHERE wsis.media_item_id = mi.content_id
)`
const orphanedProvisionalMediaItemConditions = `mi.status IN ('pending', 'unmatched', 'ambiguous')
AND ` + orphanedMediaItemSafetyConditions
const orphanedProvisionalMediaItemPredicate = `
WHERE ` + orphanedProvisionalMediaItemConditions
type OrphanedProvisionalCleanupStats struct {
Candidates int
Deleted int
}
type OrphanedProvisionalCleaner struct {
pool *pgxpool.Pool
}
func NewOrphanedProvisionalCleaner(pool *pgxpool.Pool) *OrphanedProvisionalCleaner {
return &OrphanedProvisionalCleaner{pool: pool}
}
func (c *OrphanedProvisionalCleaner) Cleanup(ctx context.Context, batchSize int) (OrphanedProvisionalCleanupStats, error) {
var stats OrphanedProvisionalCleanupStats
if c == nil || c.pool == nil {
return stats, fmt.Errorf("orphaned provisional cleanup is not configured")
}
if batchSize <= 0 {
batchSize = defaultOrphanedProvisionalCleanupBatchSize
}
if err := c.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM public.media_items mi `+orphanedProvisionalMediaItemPredicate,
).Scan(&stats.Candidates); err != nil {
return stats, fmt.Errorf("counting orphaned provisional media items: %w", err)
}
for {
tag, err := c.pool.Exec(ctx, `
WITH candidates AS (
SELECT mi.content_id
FROM public.media_items mi
`+orphanedProvisionalMediaItemPredicate+`
ORDER BY mi.content_id ASC
LIMIT $1
)
DELETE FROM public.media_items mi
USING candidates c
WHERE mi.content_id = c.content_id
`, batchSize)
if err != nil {
return stats, fmt.Errorf("deleting orphaned provisional media items: %w", err)
}
deleted := int(tag.RowsAffected())
stats.Deleted += deleted
if deleted < batchSize {
break
}
}
return stats, nil
}
+41
View File
@@ -0,0 +1,41 @@
package catalog
import (
"strings"
"testing"
)
func normalizePredicateSQL(s string) string {
return strings.Join(strings.Fields(s), " ")
}
func TestOrphanedProvisionalPredicatePreservesDurableMediaItemReferences(t *testing.T) {
predicate := normalizePredicateSQL(orphanedProvisionalMediaItemPredicate)
for _, want := range []string{
"public.abs_bookmarks ab WHERE ab.library_item_id = mi.content_id",
"public.abs_playback_sessions aps WHERE aps.content_id = mi.content_id",
"public.abs_rss_feeds arf WHERE arf.library_item_id = mi.content_id",
"public.episodes e WHERE e.series_id = mi.content_id",
"public.podcast_feeds pf WHERE pf.media_item_id = mi.content_id",
"public.seasons s WHERE s.series_id = mi.content_id",
"public.user_audio_preferences uap WHERE uap.series_id = mi.content_id",
"public.user_home_item_dismissals uhid WHERE uhid.media_item_id = mi.content_id",
"public.user_home_item_dismissals uhid_series WHERE uhid_series.series_id = mi.content_id",
"public.user_personal_collection_items upci WHERE upci.media_item_id = mi.content_id",
"public.user_series_playback_preferences uspp WHERE uspp.series_id = mi.content_id",
"public.user_subtitle_preferences usp WHERE usp.series_id = mi.content_id",
} {
if !strings.Contains(predicate, normalizePredicateSQL(want)) {
t.Fatalf("cleanup predicate missing durable reference guard %q", want)
}
}
for _, droppedTable := range []string{
"abs_collection_items",
"abs_playlist_items",
"abs_playlists",
} {
if strings.Contains(predicate, droppedTable) {
t.Fatalf("cleanup predicate must not reference dropped legacy table %q", droppedTable)
}
}
}
+1 -1
View File
@@ -32,7 +32,7 @@ func ExtractFrame(ctx context.Context, opts FrameExtractOptions) ([]byte, string
runExtract = runFFmpegFrameExtract
}
resolvedAccel := playback.ResolveHWAccel(opts.HWAccel)
resolvedAccel := playback.ResolveHWAccelWithFFmpeg(opts.HWAccel, ffmpegPath)
resolvedDevice := opts.HWDevice
if resolvedDevice == "" && (resolvedAccel == "qsv" || resolvedAccel == "vaapi") {
resolvedDevice = playback.PickRenderDevice("")
+1 -1
View File
@@ -652,7 +652,7 @@ func (s *Service) extractFrameLocal(ctx context.Context, inputPath string, seekS
func (s *Service) resolveHWConfig() (string, string) {
s.hwResolveOnce.Do(func() {
s.resolvedHWAccel = playback.ResolveHWAccel(s.hwAccel)
s.resolvedHWAccel = playback.ResolveHWAccelWithFFmpeg(s.hwAccel, s.ffmpegPath)
s.resolvedHWDevice = s.hwDevice
if s.resolvedHWDevice == "" && (s.resolvedHWAccel == "qsv" || s.resolvedHWAccel == "vaapi") {
s.resolvedHWDevice = playback.PickRenderDevice("")
+19
View File
@@ -7,6 +7,11 @@ import (
"github.com/joho/godotenv"
)
// minSecretKeyLen is the minimum acceptable SECRET_KEY length (in characters).
// It mirrors secret.MinMasterKeyLen; the package is not imported here to keep
// the bootstrap loader dependency-free, so the two constants must stay in sync.
const minSecretKeyLen = 32
// BootstrapConfig holds the minimal config needed before database connection.
type BootstrapConfig struct {
DatabaseURL string
@@ -14,6 +19,10 @@ type BootstrapConfig struct {
Listen string
JFListen string
Mode string
// SecretKey is the master key (raw SECRET_KEY env value) from which the
// at-rest credential cipher derives its data key. It lives outside Postgres
// so encrypted secrets survive a full database compromise/dump.
SecretKey []byte
}
// LoadBootstrap loads bootstrap configuration from a .env file (if it exists)
@@ -28,6 +37,15 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
return nil, fmt.Errorf("DATABASE_URL is required (set in .env or environment)")
}
// SECRET_KEY is the at-rest encryption master key. It is required: the server
// must never fall back to a zero key or a key derived from another value, or
// encrypted credentials would not survive a database dump. godotenv has
// already loaded .env above, so dev sets it there.
secretKey := os.Getenv("SECRET_KEY")
if len(secretKey) < minSecretKeyLen {
return nil, fmt.Errorf("SECRET_KEY is required (>=%d chars); generate one with: openssl rand -base64 48", minSecretKeyLen)
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
@@ -51,5 +69,6 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
Listen: ":" + port,
JFListen: ":" + jfPort,
Mode: mode,
SecretKey: []byte(secretKey),
}, nil
}
+331 -20
View File
@@ -6,11 +6,14 @@ import (
"errors"
"fmt"
"sort"
"strconv"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
var (
@@ -26,11 +29,135 @@ var (
)
type Repository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// AAD builders binding each history-import secret column to its row id.
func sourceAdminTokenAAD(sourceID int) string {
return secret.RowAAD("history_import_sources", "admin_token", strconv.Itoa(sourceID))
}
func connectServerAccessKeyAAD(sessionID string, index int, server ConnectServer) string {
return secret.RowAAD("history_import_connect_sessions", "servers_json.access_key", sessionID+":"+strconv.Itoa(index)+":"+server.ID)
}
func connectSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_connect_sessions", "connect_access_token", sessionID)
}
func plexServerAccessTokenAAD(sessionID string, index int, server PlexServer) string {
return secret.RowAAD("history_import_plex_sessions", "servers_json.access_token", sessionID+":"+strconv.Itoa(index)+":"+server.ClientIdentifier)
}
func plexSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_plex_sessions", "auth_token", sessionID)
}
func (r *Repository) encryptSourceAdminToken(sourceID int, token string) (string, error) {
return r.cipher.Encrypt(token, sourceAdminTokenAAD(sourceID))
}
func (r *Repository) encryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
if out[i].AccessKey == "" {
continue
}
accessKey, err := r.cipher.Encrypt(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) encryptPlaintextConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, bool, error) {
out := append([]ConnectServer(nil), servers...)
changed := false
for i := range out {
accessKey, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt connect server access key: %w", err)
}
if didChange {
out[i].AccessKey = accessKey
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
accessKey, err := r.cipher.DecryptIfEncrypted(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) marshalConnectServersForWrite(sessionID string, servers []ConnectServer) ([]byte, error) {
encrypted, err := r.encryptConnectServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
}
func (r *Repository) encryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
if out[i].AccessToken == "" {
continue
}
accessToken, err := r.cipher.Encrypt(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) encryptPlaintextPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, bool, error) {
out := append([]PlexServer(nil), servers...)
changed := false
for i := range out {
accessToken, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt plex server access token: %w", err)
}
if didChange {
out[i].AccessToken = accessToken
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
accessToken, err := r.cipher.DecryptIfEncrypted(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) marshalPlexServersForWrite(sessionID string, servers []PlexServer) ([]byte, error) {
encrypted, err := r.encryptPlexServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
}
func (r *Repository) ListEnabledSources(ctx context.Context) ([]Source, error) {
@@ -80,20 +207,44 @@ func (r *Repository) GetSourceByID(ctx context.Context, id int) (*Source, error)
}
func (r *Repository) CreateSource(ctx context.Context, input CreateSourceInput) (*Source, error) {
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order,
admin_token)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6, NULLIF($7, ''))
tx, err := r.pool.Begin(ctx)
if err != nil {
return nil, fmt.Errorf("begin creating history import source: %w", err)
}
defer tx.Rollback(ctx)
row := tx.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6)
RETURNING id, name, source_type, base_url, COALESCE(system_id, ''), enabled, sort_order,
(admin_token IS NOT NULL) AS has_admin_token,
created_at, updated_at`,
input.Name, input.SourceType, input.BaseURL, input.SystemID, input.Enabled, input.SortOrder,
input.AdminToken,
)
source, err := scanSource(row)
if err != nil {
return nil, fmt.Errorf("creating history import source: %w", err)
}
if input.AdminToken != "" {
encryptedToken, err := r.encryptSourceAdminToken(source.ID, input.AdminToken)
if err != nil {
return nil, fmt.Errorf("encrypt admin token for source %d: %w", source.ID, err)
}
result, err := tx.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, source.ID, encryptedToken)
if err != nil {
return nil, fmt.Errorf("setting admin token for source %d: %w", source.ID, err)
}
if result.RowsAffected() == 0 {
return nil, ErrSourceNotFound
}
source.HasAdminToken = true
}
if err := tx.Commit(ctx); err != nil {
return nil, fmt.Errorf("commit creating history import source: %w", err)
}
return source, nil
}
@@ -135,18 +286,22 @@ func (r *Repository) DeleteSource(ctx context.Context, id int) error {
}
func (r *Repository) CreateConnectSession(ctx context.Context, session ConnectSession) (*ConnectSession, error) {
serversJSON, err := json.Marshal(session.Servers)
serversJSON, err := r.marshalConnectServersForWrite(session.ID, session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling connect servers: %w", err)
}
accessToken, err := r.cipher.Encrypt(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt connect access token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_connect_sessions (
id, user_id, connect_user_id, connect_access_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.ConnectUserID, session.ConnectAccessToken, serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.ConnectUserID, accessToken, serversJSON, session.ExpiresAt,
)
stored, err := scanConnectSession(row)
stored, err := r.scanConnectSession(row)
if err != nil {
return nil, fmt.Errorf("creating connect session: %w", err)
}
@@ -158,7 +313,7 @@ func (r *Repository) GetConnectSession(ctx context.Context, userID int, sessionI
SELECT id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_connect_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := scanConnectSession(row)
session, err := r.scanConnectSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrConnectSessionNotFound
}
@@ -199,18 +354,22 @@ func (r *Repository) DeleteExpiredConnectSessions(ctx context.Context) error {
}
func (r *Repository) CreatePlexSession(ctx context.Context, session PlexSession) (*PlexSession, error) {
serversJSON, err := json.Marshal(session.Servers)
serversJSON, err := r.marshalPlexServersForWrite(session.ID, session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling plex servers: %w", err)
}
authToken, err := r.cipher.Encrypt(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt plex auth token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_plex_sessions (
id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(session.AuthToken), serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(authToken), serversJSON, session.ExpiresAt,
)
stored, err := scanPlexSession(row)
stored, err := r.scanPlexSession(row)
if err != nil {
return nil, fmt.Errorf("creating plex session: %w", err)
}
@@ -222,7 +381,7 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
SELECT id, user_id, pin_id, pin_code, COALESCE(auth_token, ''), servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_plex_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := scanPlexSession(row)
session, err := r.scanPlexSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrPlexSessionNotFound
}
@@ -239,14 +398,18 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
}
func (r *Repository) UpdatePlexSessionAuth(ctx context.Context, sessionID, authToken string, servers []PlexServer) error {
serversJSON, err := json.Marshal(servers)
serversJSON, err := r.marshalPlexServersForWrite(sessionID, servers)
if err != nil {
return fmt.Errorf("marshaling plex servers: %w", err)
}
encryptedToken, err := r.cipher.Encrypt(authToken, plexSessionTokenAAD(sessionID))
if err != nil {
return fmt.Errorf("encrypt plex auth token: %w", err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET auth_token = $2, servers_json = $3, updated_at = NOW()
WHERE id = $1`, sessionID, authToken, serversJSON)
WHERE id = $1`, sessionID, encryptedToken, serversJSON)
if err != nil {
return fmt.Errorf("updating plex session auth %s: %w", sessionID, err)
}
@@ -280,6 +443,136 @@ func (r *Repository) DeleteExpiredPlexSessions(ctx context.Context) error {
return nil
}
// BackfillSessionServerSecrets encrypts credentials nested inside history-import
// session server JSON. The top-level session token columns are handled by the
// generic column backfill, but Emby Connect access keys and Plex per-server
// access tokens live inside servers_json and need a typed JSON pass.
func (r *Repository) BackfillSessionServerSecrets(ctx context.Context) (int, error) {
connectN, connectErr := r.backfillConnectSessionServerSecrets(ctx)
plexN, plexErr := r.backfillPlexSessionServerSecrets(ctx)
return connectN + plexN, errors.Join(connectErr, plexErr)
}
func (r *Repository) backfillConnectSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_connect_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select connect sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan connect session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate connect sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []ConnectServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode connect session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextConnectServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt connect session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode connect session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_connect_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update connect session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) backfillPlexSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_plex_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select plex sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan plex session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate plex sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []PlexServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode plex session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextPlexServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt plex session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode plex session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update plex session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) ProfileExistsForUser(ctx context.Context, userID int, profileID string) (bool, error) {
var exists bool
if err := r.pool.QueryRow(ctx,
@@ -770,7 +1063,7 @@ func scanSources(rows pgx.Rows) ([]Source, error) {
return sources, rows.Err()
}
func scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
func (r *Repository) scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
var session ConnectSession
var serversJSON []byte
if err := scanner.Scan(
@@ -779,10 +1072,19 @@ func scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectS
); err != nil {
return nil, err
}
token, err := r.cipher.DecryptIfEncrypted(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt connect access token: %w", err)
}
session.ConnectAccessToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling connect session servers: %w", err)
}
session.Servers, err = r.decryptConnectServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
@@ -921,7 +1223,7 @@ func nonNilUnmatchedSamples(values []UnmatchedSample) []UnmatchedSample {
return values
}
func scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
func (r *Repository) scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
var session PlexSession
var authToken *string
var serversJSON []byte
@@ -935,10 +1237,19 @@ func scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession
if authToken != nil {
session.AuthToken = *authToken
}
token, err := r.cipher.DecryptIfEncrypted(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt plex auth token: %w", err)
}
session.AuthToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling plex session servers: %w", err)
}
session.Servers, err = r.decryptPlexServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
+9 -1
View File
@@ -21,10 +21,14 @@ var (
// SetSourceAdminToken stores an admin token for the given source.
func (r *Repository) SetSourceAdminToken(ctx context.Context, sourceID int, token string) error {
encryptedToken, err := r.encryptSourceAdminToken(sourceID, token)
if err != nil {
return fmt.Errorf("encrypt admin token for source %d: %w", sourceID, err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, sourceID, token)
WHERE id = $1`, sourceID, encryptedToken)
if err != nil {
return fmt.Errorf("setting admin token for source %d: %w", sourceID, err)
}
@@ -73,6 +77,10 @@ func (r *Repository) GetSourceWithAdminToken(ctx context.Context, sourceID int)
if err != nil {
return nil, "", fmt.Errorf("getting source %d with admin token: %w", sourceID, err)
}
adminToken, err = r.cipher.DecryptIfEncrypted(adminToken, sourceAdminTokenAAD(sourceID))
if err != nil {
return nil, "", fmt.Errorf("decrypt admin token for source %d: %w", sourceID, err)
}
return &s, adminToken, nil
}
+194 -6
View File
@@ -1,15 +1,34 @@
package historyimport
import (
"bytes"
"crypto/rand"
"encoding/json"
"fmt"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/secret"
)
// newTestRepo builds a Repository with a real cipher for scan-method tests.
func newTestRepo(t *testing.T) *Repository {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return &Repository{cipher: c}
}
type plexSessionScannerStub struct {
authToken *string
servers []PlexServer
authToken *string
servers []PlexServer
serversRaw []byte
}
func (s plexSessionScannerStub) Scan(dest ...any) error {
@@ -52,9 +71,13 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
return fmt.Errorf("dest[4] type = %T, want *string or **string", dest[4])
}
serversJSON, err := json.Marshal(s.servers)
if err != nil {
return err
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
}
serversDest, ok := dest[5].(*[]byte)
if !ok {
@@ -92,7 +115,7 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
t.Parallel()
session, err := scanPlexSession(plexSessionScannerStub{
session, err := newTestRepo(t).scanPlexSession(plexSessionScannerStub{
authToken: nil,
servers: []PlexServer{{
Name: "Plex Server",
@@ -118,4 +141,169 @@ func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
if session.Servers[0].ClientIdentifier != "server-1" {
t.Fatalf("session.Servers[0].ClientIdentifier = %q, want server-1", session.Servers[0].ClientIdentifier)
}
if session.Servers[0].AccessToken != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", session.Servers[0].AccessToken)
}
}
type connectSessionScannerStub struct {
accessToken string
servers []ConnectServer
serversRaw []byte
}
func (s connectSessionScannerStub) Scan(dest ...any) error {
if len(dest) != 9 {
return fmt.Errorf("unexpected scan dest count: %d", len(dest))
}
id, ok := dest[0].(*string)
if !ok {
return fmt.Errorf("dest[0] type = %T, want *string", dest[0])
}
*id = "connect-session-1"
userID, ok := dest[1].(*int)
if !ok {
return fmt.Errorf("dest[1] type = %T, want *int", dest[1])
}
*userID = 42
connectUserID, ok := dest[2].(*string)
if !ok {
return fmt.Errorf("dest[2] type = %T, want *string", dest[2])
}
*connectUserID = "connect-user-1"
accessToken, ok := dest[3].(*string)
if !ok {
return fmt.Errorf("dest[3] type = %T, want *string", dest[3])
}
*accessToken = s.accessToken
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
}
serversDest, ok := dest[4].(*[]byte)
if !ok {
return fmt.Errorf("dest[4] type = %T, want *[]byte", dest[4])
}
*serversDest = serversJSON
expiresAt, ok := dest[5].(*time.Time)
if !ok {
return fmt.Errorf("dest[5] type = %T, want *time.Time", dest[5])
}
*expiresAt = time.Date(2026, time.April, 1, 13, 0, 0, 0, time.UTC)
consumedAt, ok := dest[6].(**time.Time)
if !ok {
return fmt.Errorf("dest[6] type = %T, want **time.Time", dest[6])
}
*consumedAt = nil
createdAt, ok := dest[7].(*time.Time)
if !ok {
return fmt.Errorf("dest[7] type = %T, want *time.Time", dest[7])
}
*createdAt = time.Date(2026, time.April, 1, 12, 0, 0, 0, time.UTC)
updatedAt, ok := dest[8].(*time.Time)
if !ok {
return fmt.Errorf("dest[8] type = %T, want *time.Time", dest[8])
}
*updatedAt = time.Date(2026, time.April, 1, 12, 30, 0, 0, time.UTC)
return nil
}
func TestConnectSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []ConnectServer{{
ID: "emby-server-1",
Name: "Emby Server",
AccessKey: "server-access-key",
URL: "https://emby.example",
HasRemoteURL: true,
}}
raw, err := repo.marshalConnectServersForWrite("connect-session-1", servers)
if err != nil {
t.Fatalf("marshalConnectServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-access-key")) {
t.Fatalf("stored connect servers JSON leaked access key: %s", raw)
}
session, err := repo.scanConnectSession(connectSessionScannerStub{
accessToken: "connect-token",
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanConnectSession: %v", err)
}
if got := session.Servers[0].AccessKey; got != "server-access-key" {
t.Fatalf("session.Servers[0].AccessKey = %q, want server-access-key", got)
}
}
func TestPlexSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []PlexServer{{
Name: "Plex Server",
ClientIdentifier: "server-1",
AccessToken: "server-token",
RemoteURL: "https://plex.example",
Owned: true,
HasRemoteURL: true,
}}
raw, err := repo.marshalPlexServersForWrite("plex-session-1", servers)
if err != nil {
t.Fatalf("marshalPlexServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-token")) {
t.Fatalf("stored plex servers JSON leaked access token: %s", raw)
}
session, err := repo.scanPlexSession(plexSessionScannerStub{
authToken: nil,
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanPlexSession: %v", err)
}
if got := session.Servers[0].AccessToken; got != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", got)
}
}
func TestSourceAdminTokenEncryptionUsesSourceAAD(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
ciphertext, err := repo.encryptSourceAdminToken(42, "admin-token")
if err != nil {
t.Fatalf("encryptSourceAdminToken: %v", err)
}
if ciphertext == "admin-token" || bytes.Contains([]byte(ciphertext), []byte("admin-token")) {
t.Fatalf("encrypted admin token leaked plaintext: %q", ciphertext)
}
plaintext, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(42))
if err != nil {
t.Fatalf("decrypt with matching source AAD: %v", err)
}
if plaintext != "admin-token" {
t.Fatalf("decrypted admin token = %q, want admin-token", plaintext)
}
if _, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(43)); err == nil {
t.Fatalf("decrypt with different source AAD succeeded, want authentication failure")
}
}
+38 -12
View File
@@ -128,8 +128,7 @@ func (a *Authenticator) RequireSession(next http.Handler) http.Handler {
}
}
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
serveWithSession(next, w, r, session)
})
}
@@ -141,28 +140,55 @@ func safeTokenPrefix(token string) string {
return token[:8] + "..."
}
// serveWithSession injects the resolved compat session into the request context
// and continues the handler chain.
func serveWithSession(next http.Handler, w http.ResponseWriter, r *http.Request, session *Session) {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
}
// resolveCompatToken resolves a token to a compat session: a session-store token
// (normal login) or, matching Jellyfin, an sa_ admin API key (synthesized
// session bound to the key user's primary profile). Returns false when the token
// matches neither. keyAuth may be nil (resolveSession handles a nil receiver).
func resolveCompatToken(ctx context.Context, sessions *SessionStore, keyAuth *AdminAPIKeyAuthenticator, token string) (*Session, bool) {
if token == "" {
return nil, false
}
if session, ok := sessions.Get(token); ok {
return session, true
}
if strings.HasPrefix(token, "sa_") {
if session, _, _ := keyAuth.resolveSession(ctx, token); session != nil {
return session, true
}
}
return nil, false
}
// PlaybackSessionAuth creates middleware that falls back to playback session
// authentication for media stream endpoints where external players (e.g. libmpv)
// don't forward auth headers or query parameters.
func PlaybackSessionAuth(sessions *SessionStore, playbackStore *PlaybackSessionStore) func(next http.Handler) http.Handler {
func PlaybackSessionAuth(sessions *SessionStore, playbackStore *PlaybackSessionStore, keyAuth *AdminAPIKeyAuthenticator) func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Try standard token auth first.
// Try standard token auth first — a compat session token or an sa_
// admin key (synthesized session).
if token, ok := ExtractToken(r); ok {
if session, sessionOK := sessions.Get(token); sessionOK {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
if session, ok := resolveCompatToken(r.Context(), sessions, keyAuth, token); ok {
serveWithSession(next, w, r, session)
return
}
}
// Prefer the exact negotiated playback session when the client
// includes PlaySessionId on follow-up stream requests.
// Follow-up HLS requests (master/segment) carry only PlaySessionId,
// no auth header or api_key. Resolve the negotiated session's
// CompatToken — which for an API-key stream is itself the sa_ key,
// so it must go through the same session-or-API-key resolution.
if playSessionID := firstNonEmpty(r.URL.Query().Get("PlaySessionId"), r.URL.Query().Get("PlaySessionID")); playSessionID != "" {
if playSession, found := playbackStore.Get(playSessionID); found {
if session, ok := sessions.Get(playSession.CompatToken); ok {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
if session, ok := resolveCompatToken(r.Context(), sessions, keyAuth, playSession.CompatToken); ok {
serveWithSession(next, w, r, session)
return
}
writeError(w, http.StatusUnauthorized, "Unauthorized", "Session expired")
+254 -39
View File
@@ -2,12 +2,15 @@ package jellycompat
import (
"context"
"fmt"
"log/slog"
"net/http"
"strings"
"sync"
"time"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type adminAPIKeyContextKey string
@@ -24,8 +27,17 @@ type apiKeyUserLoader interface {
}
type AdminAPIKeyAuthenticator struct {
keys apiKeyValidator
users apiKeyUserLoader
keys apiKeyValidator
users apiKeyUserLoader
provider userstore.UserStoreProvider
now func() time.Time
// profiles caches only the primary-profile lookup per user — never an
// authorization decision, which is re-checked on every request.
profiles *apiKeyProfileCache
lastUsedMu sync.Mutex
lastUsedAt map[int64]time.Time
}
type adminAPIKeyAuthResult struct {
@@ -36,11 +48,26 @@ type adminAPIKeyAuthResult struct {
ok bool
}
func NewAdminAPIKeyAuthenticator(keys apiKeyValidator, users apiKeyUserLoader) *AdminAPIKeyAuthenticator {
// NewAdminAPIKeyAuthenticator builds the authenticator. keys and users are
// required (a nil return disables API-key auth entirely). provider is optional:
// when present, an admin key can synthesize a compat session (see
// resolveSession); when nil, only the admin-bool path (RequireAdminAPIKey /
// RequireSessionOrAdminAPIKey) is available.
func NewAdminAPIKeyAuthenticator(keys apiKeyValidator, users apiKeyUserLoader, provider userstore.UserStoreProvider, now func() time.Time) *AdminAPIKeyAuthenticator {
if keys == nil || users == nil {
return nil
}
return &AdminAPIKeyAuthenticator{keys: keys, users: users}
if now == nil {
now = time.Now
}
return &AdminAPIKeyAuthenticator{
keys: keys,
users: users,
provider: provider,
now: now,
profiles: newAPIKeyProfileCache(),
lastUsedAt: make(map[int64]time.Time),
}
}
func AdminAPIKeyFromContext(ctx context.Context) bool {
@@ -77,46 +104,234 @@ func RequireSessionOrAdminAPIKey(sessionAuth *Authenticator, keyAuth *AdminAPIKe
}
}
// RequireSessionOrAPIKeySession authenticates either a compat session token or
// an sa_ admin API key. For an API key it injects a session synthesized for the
// key user's primary profile, so ordinary handlers (which read
// SessionFromContext) work unchanged for both auth modes — matching Jellyfin,
// where an API key authorizes the same endpoints a user token does. When
// API-key auth is unavailable (nil authenticator or no UserStoreProvider) an
// sa_ token falls through to session-token auth and is rejected there.
func RequireSessionOrAPIKeySession(sessionAuth *Authenticator, keyAuth *AdminAPIKeyAuthenticator) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if token, ok := ExtractToken(r); ok && strings.HasPrefix(token, "sa_") {
if session, result, handled := keyAuth.resolveSession(r.Context(), token); handled {
if !result.ok {
writeError(w, result.status, result.code, result.message)
return
}
serveWithSession(next, w, r, session)
return
}
}
sessionAuth.RequireSession(next).ServeHTTP(w, r)
})
}
}
// authenticate validates an admin API key and, on success, returns a result
// carrying the admin-API-key marker in context. Used by the server-to-server
// admin routes that act on the admin identity rather than a compat session.
func (a *AdminAPIKeyAuthenticator) authenticate(r *http.Request) adminAPIKeyAuthResult {
unauthorized := adminAPIKeyAuthResult{
ctx: r.Context(),
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Invalid API key",
token, _ := ExtractToken(r)
apiKey, _, res := a.validate(r.Context(), token)
if !res.ok {
return res
}
if a == nil || a.keys == nil || a.users == nil {
return unauthorized
}
token, ok := ExtractToken(r)
if !ok || !strings.HasPrefix(token, "sa_") {
return unauthorized
}
apiKey, err := a.keys.GetByKey(r.Context(), token)
if err != nil || apiKey == nil {
return unauthorized
}
user, err := a.users.GetByID(r.Context(), apiKey.UserID)
if err != nil || user == nil || !user.Enabled {
return unauthorized
}
if user.Role != "admin" {
return adminAPIKeyAuthResult{
ctx: r.Context(),
status: http.StatusForbidden,
code: "Forbidden",
message: "Admin access required",
}
}
go func(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := a.keys.UpdateLastUsed(ctx, id); err != nil {
slog.Debug("jellycompat api key last-used update failed", "id", id, "error", err)
}
}(apiKey.ID)
a.touchLastUsed(apiKey.ID)
return adminAPIKeyAuthResult{
ctx: context.WithValue(r.Context(), adminAPIKeyKey, true),
status: http.StatusOK,
ok: true,
}
}
// validate authenticates an admin API key by token and returns the key plus its
// owning admin user. The full authorization check — key lookup, user load,
// enabled, admin role — runs on EVERY call, so revocation (key deletion, user
// disable, role change) takes effect on the very next request. On failure the
// result carries the HTTP status/code/message: 401 for a missing/unknown/
// disabled key, 403 for a non-admin key.
func (a *AdminAPIKeyAuthenticator) validate(ctx context.Context, token string) (*models.APIKey, *models.User, adminAPIKeyAuthResult) {
unauthorized := adminAPIKeyAuthResult{
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Invalid API key",
}
if a == nil || a.keys == nil || a.users == nil {
return nil, nil, unauthorized
}
if !strings.HasPrefix(token, "sa_") {
return nil, nil, unauthorized
}
apiKey, err := a.keys.GetByKey(ctx, token)
if err != nil || apiKey == nil {
return nil, nil, unauthorized
}
user, err := a.users.GetByID(ctx, apiKey.UserID)
if err != nil || user == nil || !user.Enabled {
return nil, nil, unauthorized
}
if user.Role != "admin" {
return nil, nil, adminAPIKeyAuthResult{
status: http.StatusForbidden,
code: "Forbidden",
message: "Admin access required",
}
}
return apiKey, user, adminAPIKeyAuthResult{ok: true}
}
// touchLastUsed records key usage without blocking the request, throttled to at
// most once per apiKeyLastUsedInterval per key so per-request validation on the
// hot path (e.g. HLS segments) does not issue a DB write each time.
func (a *AdminAPIKeyAuthenticator) touchLastUsed(id int64) {
now := a.now()
a.lastUsedMu.Lock()
if last, ok := a.lastUsedAt[id]; ok && now.Sub(last) < apiKeyLastUsedInterval {
a.lastUsedMu.Unlock()
return
}
a.lastUsedAt[id] = now
a.lastUsedMu.Unlock()
go func(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := a.keys.UpdateLastUsed(ctx, id); err != nil {
slog.Debug("jellycompat api key last-used update failed", "id", id, "error", err)
}
}(id)
}
// resolveSession authenticates an sa_ API key and returns a compat session
// synthesized for the key user's primary profile, so ordinary browse/stream
// handlers (which read SessionFromContext) work unchanged for an API key.
//
// handled reports whether this authenticator can serve API-key session auth at
// all: false when the authenticator is nil or has no UserStoreProvider wired, in
// which case the caller must fall through to session-token auth. When handled is
// true, result.ok reports success; on failure result carries what to return.
//
// The key and owning user are re-validated on EVERY call (no cached
// authorization), so revocation is immediate. Only the primary-profile lookup is
// cached (see primaryProfile), sparing the hot path the ListProfiles round-trip.
func (a *AdminAPIKeyAuthenticator) resolveSession(ctx context.Context, token string) (*Session, adminAPIKeyAuthResult, bool) {
if a == nil || a.provider == nil {
return nil, adminAPIKeyAuthResult{}, false
}
apiKey, user, res := a.validate(ctx, token)
if !res.ok {
return nil, res, true
}
a.touchLastUsed(apiKey.ID)
profile, err := a.primaryProfile(ctx, user.ID)
if err != nil {
slog.Warn("jellycompat api key session synthesis failed", "user_id", user.ID, "error", err)
return nil, adminAPIKeyAuthResult{
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Unable to resolve a profile for this API key",
}, true
}
// Upstream Silo token fields are left empty and the expiry zero: an API key
// has no refreshable Silo token, and a zero expiry makes RequireSession's
// refresh path a no-op. Downstream services scope by StreamAppUserID/ProfileID.
return &Session{
Token: token,
Username: user.Username,
AccountUsername: user.Username,
ProfileID: profile.ID,
ProfileName: profile.Name,
PseudoUserID: PseudoUserID(user.ID, profile.ID),
StreamAppUserID: user.ID,
CreatedAt: a.now(),
}, adminAPIKeyAuthResult{ok: true}, true
}
// primaryProfile returns the account's primary profile, caching the result per
// user for apiKeyProfileCacheTTL. This caches profile DATA only — never an
// authorization decision — so it is safe against revocation (validated
// separately on every request).
func (a *AdminAPIKeyAuthenticator) primaryProfile(ctx context.Context, userID int) (userstore.Profile, error) {
if profile, ok := a.profiles.get(userID, a.now()); ok {
return profile, nil
}
store, err := a.provider.ForUser(ctx, userID)
if err != nil {
return userstore.Profile{}, fmt.Errorf("open user store for user %d: %w", userID, err)
}
profiles, err := store.ListProfiles(ctx)
if err != nil {
return userstore.Profile{}, fmt.Errorf("list profiles for user %d: %w", userID, err)
}
profile, err := selectPrimaryProfile(userID, profiles)
if err != nil {
return userstore.Profile{}, err
}
a.profiles.put(userID, profile, a.now().Add(apiKeyProfileCacheTTL))
return profile, nil
}
// selectPrimaryProfile returns the account's primary profile, falling back to
// the first profile (with a warning) when none is flagged primary, and erroring
// when the account has no profiles.
func selectPrimaryProfile(userID int, profiles []userstore.Profile) (userstore.Profile, error) {
if len(profiles) == 0 {
return userstore.Profile{}, fmt.Errorf("user %d has no profiles", userID)
}
for _, p := range profiles {
if p.IsPrimary {
return p, nil
}
}
slog.Warn("jellycompat api key: no primary profile found, using first",
"user_id", userID, "profile_id", profiles[0].ID)
return profiles[0], nil
}
const (
// apiKeyProfileCacheTTL bounds how long a primary-profile lookup is reused.
// Profiles are stable, so this is generous; it only affects how quickly a
// reassigned/renamed primary profile is picked up, never authorization
// (the key and user are re-validated on every request).
apiKeyProfileCacheTTL = 5 * time.Minute
// apiKeyLastUsedInterval throttles api_keys.last_used_at writes so that
// per-request validation on the hot path does not write each time.
apiKeyLastUsedInterval = time.Minute
)
type apiKeyProfileCacheEntry struct {
profile userstore.Profile
expiresAt time.Time
}
// apiKeyProfileCache caches the primary-profile lookup per user so the hot path
// (e.g. HLS segment requests) skips the ListProfiles round-trip. It caches only
// profile data, never an authorization decision.
type apiKeyProfileCache struct {
mu sync.RWMutex
entries map[int]apiKeyProfileCacheEntry
}
func newAPIKeyProfileCache() *apiKeyProfileCache {
return &apiKeyProfileCache{entries: make(map[int]apiKeyProfileCacheEntry)}
}
func (c *apiKeyProfileCache) get(userID int, now time.Time) (userstore.Profile, bool) {
c.mu.RLock()
entry, ok := c.entries[userID]
c.mu.RUnlock()
if !ok || !entry.expiresAt.After(now) {
return userstore.Profile{}, false
}
return entry.profile, true
}
func (c *apiKeyProfileCache) put(userID int, profile userstore.Profile, expiresAt time.Time) {
c.mu.Lock()
c.entries[userID] = apiKeyProfileCacheEntry{profile: profile, expiresAt: expiresAt}
c.mu.Unlock()
}
@@ -0,0 +1,365 @@
package jellycompat
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// fakeUserStore embeds userstore.UserStore so only ListProfiles needs an
// implementation; any other method call panics (the synthesis path must not
// touch them).
type fakeUserStore struct {
userstore.UserStore
profiles []userstore.Profile
listErr error
}
func (s *fakeUserStore) ListProfiles(context.Context) ([]userstore.Profile, error) {
return s.profiles, s.listErr
}
type fakeUserStoreProvider struct {
store *fakeUserStore
forErr error
forCalls int
}
func (p *fakeUserStoreProvider) ForUser(context.Context, int) (userstore.UserStore, error) {
p.forCalls++
if p.forErr != nil {
return nil, p.forErr
}
return p.store, nil
}
func (p *fakeUserStoreProvider) Close() error { return nil }
// adminKeyAuthWithPrimary builds an authenticator backed by an admin key whose
// account has a non-primary and a primary profile (in that order, so the test
// proves IsPrimary selection is order-independent).
func adminKeyAuthWithPrimary(t *testing.T, clock func() time.Time) (*AdminAPIKeyAuthenticator, *fakeUserStoreProvider, *fakeAPIKeyValidator) {
t.Helper()
validator := &fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}}
users := &fakeAPIKeyUserLoader{user: &models.User{ID: 2, Username: "admin", Role: "admin", Enabled: true}}
provider := &fakeUserStoreProvider{store: &fakeUserStore{profiles: []userstore.Profile{
{ID: "kids", Name: "Kids", IsPrimary: false},
{ID: "p1", Name: "Parent", IsPrimary: true},
}}}
keyAuth := NewAdminAPIKeyAuthenticator(validator, users, provider, clock)
if keyAuth == nil {
t.Fatal("expected non-nil authenticator")
}
return keyAuth, provider, validator
}
func apiKeyRequest() *http.Request {
req := httptest.NewRequest(http.MethodGet, "/Items", nil)
req.Header.Set("X-Emby-Token", "sa_test")
return req
}
func TestRequireSessionOrAPIKeySession_SynthesizesPrimaryProfile(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, provider, _ := adminKeyAuthWithPrimary(t, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
var got *Session
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = SessionFromContext(r.Context())
w.WriteHeader(http.StatusOK)
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest())
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if got == nil {
t.Fatal("expected synthesized session in context")
}
if got.StreamAppUserID != 2 {
t.Errorf("StreamAppUserID = %d, want 2", got.StreamAppUserID)
}
if got.ProfileID != "p1" || got.ProfileName != "Parent" {
t.Errorf("profile = %q/%q, want p1/Parent", got.ProfileID, got.ProfileName)
}
if want := PseudoUserID(2, "p1"); got.PseudoUserID != want {
t.Errorf("PseudoUserID = %s, want %s", got.PseudoUserID, want)
}
if got.Username != "admin" {
t.Errorf("Username = %q, want admin", got.Username)
}
// Refresh-skip invariants: no upstream Silo token, zero expiry.
if got.StreamAppAccessToken != "" || got.StreamAppRefreshToken != "" {
t.Errorf("expected empty upstream tokens, got access=%q refresh=%q", got.StreamAppAccessToken, got.StreamAppRefreshToken)
}
if !got.StreamAppTokenExpiry.IsZero() {
t.Error("expected zero StreamAppTokenExpiry so RequireSession skips refresh")
}
if provider.forCalls != 1 {
t.Errorf("ForUser calls = %d, want 1", provider.forCalls)
}
}
func TestRequireSessionOrAPIKeySession_RejectsNonAdminKey(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
validator := &fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}}
users := &fakeAPIKeyUserLoader{user: &models.User{ID: 2, Username: "bob", Role: "user", Enabled: true}}
provider := &fakeUserStoreProvider{store: &fakeUserStore{profiles: []userstore.Profile{{ID: "p1", IsPrimary: true}}}}
keyAuth := NewAdminAPIKeyAuthenticator(validator, users, provider, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for a non-admin key")
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest())
if rec.Code != http.StatusForbidden {
t.Fatalf("status = %d, want 403", rec.Code)
}
// The non-admin account's profiles must never be consulted.
if provider.forCalls != 0 {
t.Errorf("ForUser calls = %d, want 0 (rejected before synthesis)", provider.forCalls)
}
}
func TestRequireSessionOrAPIKeySession_RejectsUnknownKey(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, _ := adminKeyAuthWithPrimary(t, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for an unknown key")
}))
req := httptest.NewRequest(http.MethodGet, "/Items", nil)
req.Header.Set("X-Emby-Token", "sa_doesnotmatch")
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
// TestRequireSessionOrAPIKeySession_FallsThroughWhenSynthesisUnavailable guards
// the regression where the middleware fronts the whole browse group: a nil
// authenticator or one without a UserStoreProvider must fall through to session
// auth (clean 401) rather than panic.
func TestRequireSessionOrAPIKeySession_FallsThroughWhenSynthesisUnavailable(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
cases := map[string]*AdminAPIKeyAuthenticator{
"nil authenticator": nil,
"nil provider": NewAdminAPIKeyAuthenticator(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
nil, clock,
),
}
for name, keyAuth := range cases {
t.Run(name, func(t *testing.T) {
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run without a valid session")
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest()) // must not panic
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
})
}
}
// TestResolveSession_RevalidatesEachCallCachesProfile: the key + owning user are
// validated on every call (so revocation is immediate), while the primary-profile
// lookup is cached until apiKeyProfileCacheTTL.
func TestResolveSession_RevalidatesEachCallCachesProfile(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, provider, validator := adminKeyAuthWithPrimary(t, clock)
ctx := context.Background()
for i := 1; i <= 3; i++ {
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatalf("resolve %d should succeed", i)
}
}
if validator.getCalls != 3 {
t.Errorf("GetByKey calls = %d, want 3 (re-validated every request)", validator.getCalls)
}
if provider.forCalls != 1 {
t.Errorf("ForUser calls = %d, want 1 (profile cached)", provider.forCalls)
}
// After the profile TTL, the profile is re-fetched (auth still every call).
now = now.Add(apiKeyProfileCacheTTL + time.Second)
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatal("post-TTL resolve should succeed")
}
if validator.getCalls != 4 {
t.Errorf("GetByKey calls = %d, want 4", validator.getCalls)
}
if provider.forCalls != 2 {
t.Errorf("ForUser calls = %d, want 2 after profile TTL", provider.forCalls)
}
}
// TestResolveSession_RevocationIsImmediate: once a key is revoked, the very next
// request fails — no cached authorization decision survives (Finding 2).
func TestResolveSession_RevocationIsImmediate(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, validator := adminKeyAuthWithPrimary(t, clock)
ctx := context.Background()
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatal("initial resolve should succeed")
}
validator.key = nil // revoke: GetByKey now returns not-found
s, res, handled := keyAuth.resolveSession(ctx, "sa_test")
if !handled {
t.Fatal("expected handled=true for an sa_ token")
}
if s != nil || res.ok {
t.Fatalf("revoked key must not resolve a session; got session=%v ok=%v", s != nil, res.ok)
}
if res.status != http.StatusUnauthorized {
t.Errorf("status = %d, want 401", res.status)
}
}
// TestPlaybackSessionAuth_APIKeyViaPlaySessionId: HLS follow-up requests carry
// only PlaySessionId. When the negotiated session's CompatToken is an sa_ key,
// PlaybackSessionAuth must still resolve it (Finding 1).
func TestPlaybackSessionAuth_APIKeyViaPlaySessionId(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, _ := adminKeyAuthWithPrimary(t, clock)
sessions := NewSessionStore(time.Hour, clock)
playback := NewPlaybackSessionStore(time.Hour, clock)
playback.Put(PlaybackSession{ID: "ps1", CompatToken: "sa_test"})
var got *Session
h := PlaybackSessionAuth(sessions, playback, keyAuth)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = SessionFromContext(r.Context())
w.WriteHeader(http.StatusOK)
}))
// No token header / no api_key — only PlaySessionId, like an HLS segment URL.
req := httptest.NewRequest(http.MethodGet, "/Videos/x/hls/p/seg0.ts?PlaySessionId=ps1", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if got == nil || got.Token != "sa_test" || got.StreamAppUserID != 2 {
t.Fatalf("expected synthesized api-key session (token sa_test, user 2); got %+v", got)
}
}
// TestPlaybackSessionAuth_RevokedAPIKeyViaPlaySessionId: a revoked key fails even
// through the PlaySessionId fallback.
func TestPlaybackSessionAuth_RevokedAPIKeyViaPlaySessionId(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, validator := adminKeyAuthWithPrimary(t, clock)
validator.key = nil // revoked
sessions := NewSessionStore(time.Hour, clock)
playback := NewPlaybackSessionStore(time.Hour, clock)
playback.Put(PlaybackSession{ID: "ps1", CompatToken: "sa_test"})
h := PlaybackSessionAuth(sessions, playback, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for a revoked key")
}))
req := httptest.NewRequest(http.MethodGet, "/Videos/x/hls/p/seg0.ts?PlaySessionId=ps1", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
// TestValidatePseudoUser confirms the path-userId guard a synthesized session
// relies on: empty or matching ids pass; a mismatched id 404s.
func TestValidatePseudoUser(t *testing.T) {
session := &Session{PseudoUserID: PseudoUserID(2, "p1")}
if !validatePseudoUser(httptest.NewRecorder(), "", session) {
t.Error("empty userID should pass")
}
if !validatePseudoUser(httptest.NewRecorder(), session.PseudoUserID.String(), session) {
t.Error("matching userID should pass")
}
rec := httptest.NewRecorder()
if validatePseudoUser(rec, PseudoUserID(2, "other").String(), session) {
t.Error("mismatched userID should fail")
}
if rec.Code != http.StatusNotFound {
t.Errorf("mismatch status = %d, want 404", rec.Code)
}
}
func TestHandleUsers_ReturnsCallerAsSingleElement(t *testing.T) {
h := NewAuthHandler(&config.Config{}, nil, nil)
session := &Session{PseudoUserID: PseudoUserID(2, "p1"), Username: "admin"}
req := httptest.NewRequest(http.MethodGet, "/Users", nil)
req = req.WithContext(context.WithValue(req.Context(), compatSessionKey, session))
rec := httptest.NewRecorder()
h.HandleUsers(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
var users []map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &users); err != nil {
t.Fatalf("unmarshal: %v; body=%s", err, rec.Body.String())
}
if len(users) != 1 {
t.Fatalf("len = %d, want 1 (current profile only)", len(users))
}
if users[0]["Id"] != session.PseudoUserID.String() {
t.Errorf("Id = %v, want %s", users[0]["Id"], session.PseudoUserID.String())
}
if users[0]["Name"] != "admin" {
t.Errorf("Name = %v, want admin", users[0]["Name"])
}
}
func TestHandleUsers_RequiresSession(t *testing.T) {
h := NewAuthHandler(&config.Config{}, nil, nil)
rec := httptest.NewRecorder()
h.HandleUsers(rec, httptest.NewRequest(http.MethodGet, "/Users", nil))
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
func TestRouter_UsersRequiresAuth(t *testing.T) {
cfg, err := config.LoadFromDB(map[string]string{})
if err != nil {
t.Fatalf("LoadFromDB: %v", err)
}
router := NewRouter(Dependencies{Config: cfg})
rec := httptest.NewRecorder()
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/Users", nil))
if rec.Code != http.StatusUnauthorized {
t.Fatalf("GET /Users status = %d, want 401", rec.Code)
}
}
+12 -4
View File
@@ -120,7 +120,7 @@ func TestRequireSession_NoAuthService_PassesThroughExpiredStreamAppToken(t *test
}
func TestRequireAdminAPIKey_AcceptsAdminKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
)
@@ -141,7 +141,7 @@ func TestRequireAdminAPIKey_AcceptsAdminKey(t *testing.T) {
}
func TestRequireAdminAPIKey_RejectsNonAdminKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "user", Enabled: true}},
)
@@ -159,7 +159,7 @@ func TestRequireAdminAPIKey_RejectsNonAdminKey(t *testing.T) {
}
func TestRequireAdminAPIKey_RejectsNilAPIKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{returnNilWithoutError: true},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
)
@@ -178,7 +178,7 @@ func TestRequireAdminAPIKey_RejectsNilAPIKey(t *testing.T) {
func TestRequireAdminAPIKey_LastUsedUpdateHasDeadline(t *testing.T) {
called := make(chan bool, 1)
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{
key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"},
update: func(ctx context.Context, _ int64) error {
@@ -210,13 +210,21 @@ func TestRequireAdminAPIKey_LastUsedUpdateHasDeadline(t *testing.T) {
}
}
// newAdminAPIKeyAuthForTest builds an authenticator without a UserStoreProvider,
// exercising the admin-bool path only (no session synthesis).
func newAdminAPIKeyAuthForTest(keys apiKeyValidator, users apiKeyUserLoader) *AdminAPIKeyAuthenticator {
return NewAdminAPIKeyAuthenticator(keys, users, nil, nil)
}
type fakeAPIKeyValidator struct {
key *models.APIKey
returnNilWithoutError bool
getCalls int
update func(context.Context, int64) error
}
func (f *fakeAPIKeyValidator) GetByKey(_ context.Context, key string) (*models.APIKey, error) {
f.getCalls++
if f.returnNilWithoutError {
return nil, nil
}
+17
View File
@@ -176,6 +176,23 @@ func (h *AuthHandler) HandleCurrentUser(w http.ResponseWriter, r *http.Request)
writeJSON(w, http.StatusOK, h.userDTO(session))
}
// HandleUsers serves GET /Users.
//
// Jellyfin's user-list endpoint accepts any authenticated caller (including an
// API key) and is what Tunarr probes to verify an API-key connection (the key
// has no "me"). Silo separates login accounts from household profiles and is
// multi-account, so we return only the caller's own user as a single-element
// list rather than enumerating every account's profiles — listing all users
// would leak identities across households.
func (h *AuthHandler) HandleUsers(w http.ResponseWriter, r *http.Request) {
session := SessionFromContext(r.Context())
if session == nil {
writeError(w, http.StatusUnauthorized, "Unauthorized", "Missing authentication token")
return
}
writeJSON(w, http.StatusOK, []userDTOResponse{h.userDTO(session)})
}
// HandleUserByID serves GET /Users/{id}.
func (h *AuthHandler) HandleUserByID(w http.ResponseWriter, r *http.Request) {
session := SessionFromContext(r.Context())
+6 -5
View File
@@ -70,12 +70,12 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.SubtitleRepo != nil {
subtitleRepo = deps.SubtitleRepo
} else if deps.DB != nil {
subtitleRepo = subtitles.NewPgRepository(deps.DB)
subtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
}
itemsHandler := NewItemsHandler(deps.ContentService, deps.UserDataService, deps.IDCodec, deps.Config, deps.ImageCache, nextUpRepo, deps.BrowseRepo, deps.PersonRepo, deps.DetailSvc, deps.ItemRepo, deps.EpisodeRepo, deps.AccessFilterFn, subtitleRepo)
itemsHandler.recommender = deps.Recommender
autoscanHandler := NewAutoscanHandler(deps.FolderRepo, deps.ScanQueue, deps.IDCodec, itemsHandler)
adminAPIKeyAuth := NewAdminAPIKeyAuthenticator(deps.APIKeyValidator, deps.APIKeyUserLoader)
adminAPIKeyAuth := NewAdminAPIKeyAuthenticator(deps.APIKeyValidator, deps.APIKeyUserLoader, deps.UserStoreProvider, deps.Now)
autoscanVirtualFoldersRegistered := false
if deps.Authenticator != nil && adminAPIKeyAuth != nil && autoscanHandler != nil {
r.With(RequireSessionOrAdminAPIKey(deps.Authenticator, adminAPIKeyAuth)).
@@ -126,8 +126,9 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.Authenticator != nil {
r.Group(func(r chi.Router) {
r.Use(deps.Authenticator.RequireSession)
r.Use(RequireSessionOrAPIKeySession(deps.Authenticator, adminAPIKeyAuth))
r.Get("/Users/Me", authHandler.HandleCurrentUser)
r.Get("/Users", authHandler.HandleUsers)
r.Get("/Users/{id}", authHandler.HandleUserByID)
r.Get("/UserViews", itemsHandler.HandleViews)
r.Get("/UserViews/GroupingOptions", itemsHandler.HandleGroupingOptionsStub)
@@ -205,7 +206,7 @@ func NewRouter(deps Dependencies) chi.Router {
// Stream routes: use playback-session auth fallback for media players
// (e.g. libmpv) that don't forward auth headers or query parameters.
r.Group(func(r chi.Router) {
r.Use(PlaybackSessionAuth(deps.SessionStore, deps.PlaybackStore))
r.Use(PlaybackSessionAuth(deps.SessionStore, deps.PlaybackStore, adminAPIKeyAuth))
r.Method(http.MethodHead, "/Videos/{id}/stream", http.HandlerFunc(playbackHandler.HandleVideoStream))
r.Get("/Videos/{id}/stream", playbackHandler.HandleVideoStream)
r.Method(http.MethodHead, "/Videos/{id}/stream.{container}", http.HandlerFunc(playbackHandler.HandleVideoStream))
@@ -240,7 +241,7 @@ func withDefaults(deps Dependencies) Dependencies {
deps.SessionStore = NewPersistentSessionStore(
deps.Config.JellyfinCompat.SessionTTL,
deps.Now,
NewSessionRepository(deps.DB),
NewSessionRepository(deps.DB, deps.SecretCipher),
)
} else {
deps.SessionStore = NewSessionStore(deps.Config.JellyfinCompat.SessionTTL, deps.Now)
+3 -1
View File
@@ -16,6 +16,7 @@ import (
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/recommendations"
"github.com/Silo-Server/silo-server/internal/scantrigger"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/subtitles"
"github.com/Silo-Server/silo-server/internal/userstore"
)
@@ -24,6 +25,7 @@ import (
type Dependencies struct {
Config *config.Config
DB *pgxpool.Pool
SecretCipher *secret.Cipher // at-rest credential cipher (required when DB is set)
ClientIPResolver *clientip.Resolver
HTTPClient *http.Client
Now func() time.Time
@@ -132,7 +134,7 @@ func (s *Server) SessionStore() *SessionStore {
// Call this once after constructing the server; goroutines stop when ctx is cancelled.
func (s *Server) StartBackgroundTasks(ctx context.Context) {
if s.deps.DB != nil {
repo := NewSessionRepository(s.deps.DB)
repo := NewSessionRepository(s.deps.DB, s.deps.SecretCipher)
StartSessionCleanup(ctx, repo, 1*time.Hour)
}
}
+35 -8
View File
@@ -9,21 +9,32 @@ import (
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
const jellycompatSessionColumns = `token, username, account_username, profile_id, profile_name, pseudo_user_id, streamapp_user_id, streamapp_access_token, streamapp_refresh_token, streamapp_token_expiry, created_at, expires_at`
// SessionRepository persists compat sessions in PostgreSQL.
type SessionRepository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
// NewSessionRepository creates a new compat session repository.
func NewSessionRepository(pool *pgxpool.Pool) *SessionRepository {
return &SessionRepository{pool: pool}
func NewSessionRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *SessionRepository {
return &SessionRepository{pool: pool, cipher: cipher}
}
func scanCompatSession(row pgx.Row) (*Session, error) {
// jellycompatTokenAAD binds a streamapp_* token ciphertext to its session row,
// keyed by the session token (the table's primary key). The session token
// itself is not encrypted (it is matched by value on lookup), so it is a stable,
// known identifier on both the read and write paths.
func jellycompatTokenAAD(column, token string) string {
return secret.RowAAD("jellycompat_sessions", column, token)
}
func (r *SessionRepository) scanCompatSession(row pgx.Row) (*Session, error) {
var session Session
err := row.Scan(
&session.Token,
@@ -45,6 +56,13 @@ func scanCompatSession(row pgx.Row) (*Session, error) {
}
return nil, fmt.Errorf("scan compat session: %w", err)
}
// Decrypt the bridged Silo access/refresh tokens (read-path contract).
if session.StreamAppAccessToken, err = r.cipher.DecryptIfEncrypted(session.StreamAppAccessToken, jellycompatTokenAAD("streamapp_access_token", session.Token)); err != nil {
return nil, fmt.Errorf("decrypt streamapp access token: %w", err)
}
if session.StreamAppRefreshToken, err = r.cipher.DecryptIfEncrypted(session.StreamAppRefreshToken, jellycompatTokenAAD("streamapp_refresh_token", session.Token)); err != nil {
return nil, fmt.Errorf("decrypt streamapp refresh token: %w", err)
}
return &session, nil
}
@@ -54,7 +72,16 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
session.Token = uuid.NewString()
}
_, err := r.pool.Exec(ctx, `
accessToken, err := r.cipher.Encrypt(session.StreamAppAccessToken, jellycompatTokenAAD("streamapp_access_token", session.Token))
if err != nil {
return fmt.Errorf("encrypt streamapp access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(session.StreamAppRefreshToken, jellycompatTokenAAD("streamapp_refresh_token", session.Token))
if err != nil {
return fmt.Errorf("encrypt streamapp refresh token: %w", err)
}
_, err = r.pool.Exec(ctx, `
INSERT INTO jellycompat_sessions (
token, username, account_username, profile_id, profile_name, pseudo_user_id,
streamapp_user_id, streamapp_access_token, streamapp_refresh_token,
@@ -84,8 +111,8 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
session.ProfileName,
session.PseudoUserID,
session.StreamAppUserID,
session.StreamAppAccessToken,
session.StreamAppRefreshToken,
accessToken,
refreshToken,
session.StreamAppTokenExpiry,
session.CreatedAt,
session.ExpiresAt,
@@ -98,7 +125,7 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
// GetByToken loads an active compat session by token.
func (r *SessionRepository) GetByToken(ctx context.Context, token string, now time.Time) (*Session, error) {
return scanCompatSession(r.pool.QueryRow(ctx,
return r.scanCompatSession(r.pool.QueryRow(ctx,
`SELECT `+jellycompatSessionColumns+`
FROM jellycompat_sessions
WHERE token = $1 AND expires_at > $2`,
+98 -14
View File
@@ -70,6 +70,10 @@ type metadataItemRepo interface {
ListUnmatchedByFolderAndPathPrefix(ctx context.Context, folderID int, pathPrefix string, limit int) ([]string, error)
}
type metadataItemDeleteRepo interface {
Delete(ctx context.Context, contentID string) ([]string, error)
}
type metadataProviderIDRepo interface {
GetByContentID(ctx context.Context, contentID string) ([]*models.MediaItemProviderID, error)
ReplaceByContentID(ctx context.Context, contentID string, providerIDs map[string]string) error
@@ -3741,18 +3745,9 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
return s.hooks.createOrFindSkeleton(ctx, file, folderID)
}
libraryType := ""
if s != nil && s.folderRepo != nil && folderID > 0 {
folder, err := s.folderRepo.GetByID(ctx, folderID)
if err != nil {
slog.Warn("metadata: failed to load folder context for path classification",
"folder_id", folderID,
"file_path", file.FilePath,
"error", err,
)
} else if folder != nil {
libraryType = folder.Type
}
libraryType, err := s.folderTypeForSkeleton(ctx, folderID)
if err != nil {
return nil, err
}
contentRootPath := filepath.Dir(file.FilePath)
@@ -4027,7 +4022,12 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
// pre-confirmation ownership claims. Matching is driven by explicit IDs or
// later provider confirmation to avoid cross-root false merges.
// No existing item — create skeleton.
// No existing item — create skeleton. Re-check the folder state while still
// under the dedup lock so a library disabled for deletion cannot receive new
// skeleton items from a matcher that passed an older outer enabled check.
if _, err := s.folderTypeForSkeleton(ctx, folderID); err != nil {
return nil, err
}
contentID, err := generateContentID()
if err != nil {
return nil, fmt.Errorf("generate content id: %w", err)
@@ -4054,10 +4054,53 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
// Link file, claim root, and create library membership.
if err := s.fileRepo.UpdateContentID(ctx, file.ID, contentID); err != nil {
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up unlinked skeleton",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("linking file to skeleton: %w", err)
}
if _, err := s.folderTypeForSkeleton(ctx, folderID); err != nil {
if clearErr := s.fileRepo.UpdateContentID(ctx, file.ID, ""); clearErr != nil {
slog.Warn("metadata: failed to clear file link after disabled skeleton folder",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", clearErr,
)
}
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up skeleton for disabled folder",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("validating skeleton folder before membership: %w", err)
}
if err := s.upsertLibraryMembership(ctx, contentID, folderID); err != nil {
s.logLibraryMembershipError("upserting skeleton membership", contentID, folderID, err)
if clearErr := s.fileRepo.UpdateContentID(ctx, file.ID, ""); clearErr != nil {
slog.Warn("metadata: failed to clear file link after skeleton membership failure",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", clearErr,
)
}
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up membershipless skeleton",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("upserting skeleton membership: %w", err)
}
res.ContentID = contentID
@@ -4065,6 +4108,47 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
return res, nil
}
func (s *MetadataService) folderTypeForSkeleton(ctx context.Context, folderID int) (string, error) {
if s == nil || s.folderRepo == nil || folderID <= 0 {
return "", nil
}
folder, err := s.folderRepo.GetByID(ctx, folderID)
if err != nil {
return "", fmt.Errorf("loading folder context for skeleton: %w", err)
}
if folder == nil {
return "", fmt.Errorf("folder %d is unavailable for skeleton creation", folderID)
}
if !folder.Enabled {
return "", fmt.Errorf("folder %d is disabled for skeleton creation", folderID)
}
return folder.Type, nil
}
func (s *MetadataService) deleteCreatedSkeleton(ctx context.Context, contentID string) error {
if s == nil || strings.TrimSpace(contentID) == "" {
return nil
}
if repo, ok := s.itemRepo.(metadataItemDeleteRepo); ok {
if _, err := repo.Delete(ctx, contentID); err != nil && !errors.Is(err, catalog.ErrItemNotFound) {
return err
}
return nil
}
if s.dbPool == nil {
return nil
}
_, err := s.dbPool.Exec(ctx, `
DELETE FROM media_items
WHERE content_id = $1
AND NOT EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = media_items.content_id
)`, contentID)
return err
}
// claimRoot records a canonical root path claim for dedup. Failures are logged
// but not fatal — the root claim is an optimization, not a hard requirement.
func (s *MetadataService) claimRoot(ctx context.Context, folderID int, rootPath, contentID string) {
+126 -1
View File
@@ -11,6 +11,7 @@ import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
@@ -81,6 +82,16 @@ func (r *fakeItemRepo) Upsert(_ context.Context, item *models.MediaItem) error {
return nil
}
func (r *fakeItemRepo) Delete(_ context.Context, contentID string) ([]string, error) {
r.mu.Lock()
defer r.mu.Unlock()
if _, ok := r.items[contentID]; !ok {
return nil, catalog.ErrItemNotFound
}
delete(r.items, contentID)
return nil, nil
}
func (r *fakeItemRepo) IncrementRefreshFailure(_ context.Context, contentID string) error {
r.mu.Lock()
defer r.mu.Unlock()
@@ -259,6 +270,8 @@ type fakeFileRepo struct {
groupContent map[string]string // "folderID:version:key" -> contentID
groupFiles map[string][]*models.MediaFile
matchStamps map[int]time.Time // fileID -> match_attempted_at
updateErrors map[int]error
afterUpdate func(fileID int, contentID string)
claimUnmatchedCalls int
claimNonSeriesCalls int
claimMixedCalls int
@@ -273,13 +286,22 @@ func newFakeFileRepo() *fakeFileRepo {
groupContent: make(map[string]string),
groupFiles: make(map[string][]*models.MediaFile),
matchStamps: make(map[int]time.Time),
updateErrors: make(map[int]error),
}
}
func (r *fakeFileRepo) UpdateContentID(_ context.Context, fileID int, contentID string) error {
r.mu.Lock()
defer r.mu.Unlock()
if err := r.updateErrors[fileID]; err != nil {
r.mu.Unlock()
return err
}
r.contentIDs[fileID] = contentID
afterUpdate := r.afterUpdate
r.mu.Unlock()
if afterUpdate != nil {
afterUpdate(fileID, contentID)
}
return nil
}
@@ -1168,6 +1190,109 @@ func TestCreateOrFindSkeleton_NoFolderIDs(t *testing.T) {
}
}
func TestCreateOrFindSkeleton_DisabledFolderDoesNotCreateItem(t *testing.T) {
h := newTestHarness()
h.service.folderRepo = &fakeMetadataFolderRepo{
folders: map[int]*models.MediaFolder{
10: {ID: 10, Type: "movies", Enabled: false},
},
}
ctx := context.Background()
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected disabled folder error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !strings.Contains(err.Error(), "disabled") {
t.Fatalf("expected disabled error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected no items to be created, got %d", len(h.itemRepo.items))
}
if cid := h.fileRepo.contentIDs[file.ID]; cid != "" {
t.Fatalf("expected file to remain unlinked, got content_id %q", cid)
}
}
func TestCreateOrFindSkeleton_LinkFailureDeletesSkeleton(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
linkErr := errors.New("media file not found")
h.fileRepo.updateErrors[1] = linkErr
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected link error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !errors.Is(err, linkErr) {
t.Fatalf("expected wrapped link error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected skeleton item to be deleted, got %d items", len(h.itemRepo.items))
}
if len(h.libraryRepo.memberships) != 0 {
t.Fatalf("expected no library memberships, got %d", len(h.libraryRepo.memberships))
}
}
func TestCreateOrFindSkeleton_DisabledBeforeMembershipDeletesSkeleton(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
folder := &models.MediaFolder{ID: 10, Type: "movies", Enabled: true}
h.service.folderRepo = &fakeMetadataFolderRepo{
folders: map[int]*models.MediaFolder{10: folder},
}
h.fileRepo.afterUpdate = func(_ int, contentID string) {
if contentID != "" {
folder.Enabled = false
}
}
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected disabled folder error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !strings.Contains(err.Error(), "disabled") {
t.Fatalf("expected disabled error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected skeleton item to be deleted, got %d items", len(h.itemRepo.items))
}
if len(h.libraryRepo.memberships) != 0 {
t.Fatalf("expected no library memberships, got %d", len(h.libraryRepo.memberships))
}
if cid := h.fileRepo.contentIDs[file.ID]; cid != "" {
t.Fatalf("expected file link to be cleared, got content_id %q", cid)
}
}
// TestCreateOrFindSkeleton_PendingItemGetsLibraryMembership verifies that
// library membership is created immediately for pending items, not just for
// matched items.
+16 -3
View File
@@ -11,6 +11,7 @@ import (
"github.com/Silo-Server/silo-server/internal/cache"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/secret"
)
// BootstrapOverrides holds values from env/CLI that must survive config
@@ -29,16 +30,21 @@ type Watcher struct {
mu sync.RWMutex
cfg *config.Config
pool *pgxpool.Pool
cipher *secret.Cipher
eventBus cache.EventBus
bootstrap BootstrapOverrides
onChange []func(old, updated *config.Config)
reloadCh chan struct{} // buffered(1), event bus writes here
}
// NewWatcher creates a new config watcher. Call Start to begin watching.
func NewWatcher(pool *pgxpool.Pool, eventBus cache.EventBus, bootstrap BootstrapOverrides) *Watcher {
// NewWatcher creates a new config watcher. Call Start to begin watching. The
// cipher decrypts sensitive server_settings values (read here via raw SQL)
// before they reach config.LoadFromDB, so a hot reload never feeds ciphertext
// into the live config (which would, e.g., break JWT validation).
func NewWatcher(pool *pgxpool.Pool, cipher *secret.Cipher, eventBus cache.EventBus, bootstrap BootstrapOverrides) *Watcher {
return &Watcher{
pool: pool,
cipher: cipher,
eventBus: eventBus,
bootstrap: bootstrap,
reloadCh: make(chan struct{}, 1),
@@ -114,7 +120,14 @@ func (w *Watcher) reload(ctx context.Context) error {
if err := rows.Scan(&k, &v); err != nil {
return fmt.Errorf("scan server_settings row: %w", err)
}
m[k] = v
// Decrypt sensitive keys (read-path contract: legacy plaintext passes
// through, enc:v1: values decrypt, corrupt ciphertext errors) so
// LoadFromDB always sees plaintext.
decrypted, derr := w.cipher.DecryptIfEncrypted(v, secret.SettingsAAD(k))
if derr != nil {
return fmt.Errorf("decrypt server_settings %q: %w", k, derr)
}
m[k] = decrypted
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate server_settings: %w", err)
+206 -19
View File
@@ -1,15 +1,38 @@
package playback
import (
"context"
"log/slog"
"os"
"os/exec"
"path/filepath"
"runtime"
"sort"
"strings"
"sync"
"time"
)
const defaultDRIDir = "/dev/dri"
var (
defaultDRIDir = "/dev/dri"
defaultNVIDIAControlDevice = "/dev/nvidiactl"
defaultNVIDIADeviceGlob = "/dev/nvidia[0-9]*"
sysClassDRMDir = "/sys/class/drm"
currentGOOS = runtime.GOOS
nvencProbeCommandTimeout = 3 * time.Second
)
type nvencProbeResult struct {
available bool
reason string
}
var nvencProbeCache = struct {
sync.Mutex
byPath map[string]nvencProbeResult
}{
byPath: make(map[string]nvencProbeResult),
}
// HWAccelInfo describes the detected hardware acceleration capability.
type HWAccelInfo struct {
@@ -22,6 +45,11 @@ type HWAccelInfo struct {
// DetectHWAccel probes this host's GPU hardware and returns structured info.
func DetectHWAccel() HWAccelInfo {
return DetectHWAccelWithFFmpeg("")
}
// DetectHWAccelWithFFmpeg probes this host's GPU hardware and configured FFmpeg.
func DetectHWAccelWithFFmpeg(ffmpegPath string) HWAccelInfo {
devices := listRenderDevices(defaultDRIDir)
intel := false
for _, d := range devices {
@@ -31,7 +59,7 @@ func DetectHWAccel() HWAccelInfo {
}
}
return HWAccelInfo{
Resolved: ResolveHWAccel("auto"),
Resolved: ResolveHWAccelWithFFmpeg("auto", ffmpegPath),
RenderDevices: devices,
IntelDetected: intel,
Source: "local",
@@ -53,34 +81,163 @@ func PickRenderDevice(explicit string) string {
return dev
}
// ResolveHWAccel resolves "auto" into a concrete acceleration method by
// probing the system. Preference order: qsv > vaapi > none.
// Non-"auto" values are returned unchanged.
// ResolveHWAccel resolves "auto" using the default FFmpeg binary.
func ResolveHWAccel(hwAccel string) string {
return ResolveHWAccelWithFFmpeg(hwAccel, "")
}
// ResolveHWAccelWithFFmpeg resolves "auto" into a concrete acceleration method
// by probing the system and the configured FFmpeg binary.
// Preference order: nvenc > qsv > vaapi > none.
// Non-"auto" values are returned unchanged.
func ResolveHWAccelWithFFmpeg(hwAccel string, ffmpegPath string) string {
if hwAccel != "auto" {
return hwAccel
}
if runtime.GOOS != "linux" {
if currentGOOS != "linux" {
return "none"
}
devices := listRenderDevices(defaultDRIDir)
if len(devices) == 0 {
slog.Info("hw_accel=auto: no render devices found, using software encoding")
return "none"
}
// Check for Intel GPU — enables QSV (preferred).
var intelDevice string
var nvidiaDevice string
var vaapiDevice string
for _, dev := range devices {
if isIntelDevice(dev) {
slog.Info("hw_accel=auto: Intel GPU detected, using QSV", "device", dev)
return "qsv"
switch {
case isNVIDIADevice(dev):
if nvidiaDevice == "" {
nvidiaDevice = dev
}
case isIntelDevice(dev):
if intelDevice == "" {
intelDevice = dev
}
default:
if vaapiDevice == "" {
vaapiDevice = dev
}
}
}
// Any accessible render device supports VAAPI.
slog.Info("hw_accel=auto: non-Intel GPU detected, using VAAPI", "device", devices[0])
return "vaapi"
if nvidiaDevice != "" || hasNVIDIADevice() {
if ok, reason := ffmpegSupportsNVENC(ffmpegPath); ok {
if nvidiaDevice != "" {
slog.Info("hw_accel=auto: NVIDIA GPU detected, using NVENC", "device", nvidiaDevice)
} else {
slog.Info("hw_accel=auto: NVIDIA device detected, using NVENC")
}
return "nvenc"
} else {
slog.Warn("hw_accel=auto: NVIDIA device detected but FFmpeg NVENC probe failed",
"ffmpeg", normalizeFFmpegPath(ffmpegPath), "reason", reason)
}
}
if intelDevice != "" {
slog.Info("hw_accel=auto: Intel GPU detected, using QSV", "device", intelDevice)
return "qsv"
}
if vaapiDevice != "" {
slog.Info("hw_accel=auto: non-Intel GPU detected, using VAAPI", "device", vaapiDevice)
return "vaapi"
}
slog.Info("hw_accel=auto: no compatible GPU devices found, using software encoding")
return "none"
}
func ffmpegSupportsNVENC(ffmpegPath string) (bool, string) {
ffmpegPath = normalizeFFmpegPath(ffmpegPath)
nvencProbeCache.Lock()
if result, ok := nvencProbeCache.byPath[ffmpegPath]; ok {
nvencProbeCache.Unlock()
return result.available, result.reason
}
nvencProbeCache.Unlock()
result := probeFFmpegNVENC(ffmpegPath)
nvencProbeCache.Lock()
nvencProbeCache.byPath[ffmpegPath] = result
nvencProbeCache.Unlock()
return result.available, result.reason
}
func normalizeFFmpegPath(ffmpegPath string) string {
ffmpegPath = strings.TrimSpace(ffmpegPath)
if ffmpegPath == "" {
ffmpegPath = ffmpegBinary()
}
if strings.ContainsRune(ffmpegPath, os.PathSeparator) {
return filepath.Clean(ffmpegPath)
}
return ffmpegPath
}
func probeFFmpegNVENC(ffmpegPath string) nvencProbeResult {
if output, err := runFFmpegProbe(ffmpegPath, "-hide_banner", "-hwaccels"); err != nil {
return nvencProbeResult{reason: "hwaccels probe failed: " + probeFailure(err, output)}
} else if !ffmpegOutputHasToken(output, "cuda") {
return nvencProbeResult{reason: "cuda hwaccel unavailable"}
}
if output, err := runFFmpegProbe(ffmpegPath, "-hide_banner", "-encoders"); err != nil {
return nvencProbeResult{reason: "encoders probe failed: " + probeFailure(err, output)}
} else if !ffmpegOutputHasToken(output, "h264_nvenc") {
return nvencProbeResult{reason: "h264_nvenc encoder unavailable"}
} else if !ffmpegOutputHasToken(output, "hevc_nvenc") {
return nvencProbeResult{reason: "hevc_nvenc encoder unavailable"}
}
if output, err := runFFmpegProbe(ffmpegPath, "-hide_banner", "-filters"); err != nil {
return nvencProbeResult{reason: "filters probe failed: " + probeFailure(err, output)}
} else if !ffmpegOutputHasToken(output, "scale_cuda") {
return nvencProbeResult{reason: "scale_cuda filter unavailable"}
} else if !ffmpegOutputHasToken(output, "hwupload_cuda") {
return nvencProbeResult{reason: "hwupload_cuda filter unavailable"}
}
if output, err := runFFmpegProbe(ffmpegPath,
"-hide_banner",
"-loglevel", "error",
"-f", "lavfi",
"-i", "testsrc2=size=640x360:rate=1",
"-frames:v", "1",
"-an",
"-c:v", "h264_nvenc",
"-f", "null",
"-",
); err != nil {
return nvencProbeResult{reason: "h264_nvenc smoke encode failed: " + probeFailure(err, output)}
}
return nvencProbeResult{available: true}
}
func runFFmpegProbe(ffmpegPath string, args ...string) ([]byte, error) {
ctx, cancel := context.WithTimeout(context.Background(), nvencProbeCommandTimeout)
defer cancel()
return exec.CommandContext(ctx, ffmpegPath, args...).CombinedOutput()
}
func ffmpegOutputHasToken(output []byte, token string) bool {
for _, field := range strings.Fields(string(output)) {
if strings.EqualFold(field, token) {
return true
}
}
return false
}
func probeFailure(err error, output []byte) string {
message := strings.TrimSpace(err.Error())
if trimmed := strings.TrimSpace(string(output)); trimmed != "" {
if len(trimmed) > 240 {
trimmed = trimmed[:240] + "..."
}
message += ": " + trimmed
}
return message
}
// listRenderDevices returns all accessible /dev/dri/renderD* paths, sorted.
@@ -107,7 +264,7 @@ func listRenderDevices(driDir string) []string {
func isIntelDevice(renderDevPath string) bool {
// /dev/dri/renderD128 → card name "renderD128"
name := filepath.Base(renderDevPath)
vendorPath := filepath.Join("/sys/class/drm", name, "device", "vendor")
vendorPath := filepath.Join(sysClassDRMDir, name, "device", "vendor")
data, err := os.ReadFile(vendorPath)
if err != nil {
return false
@@ -115,6 +272,36 @@ func isIntelDevice(renderDevPath string) bool {
return strings.TrimSpace(string(data)) == "0x8086"
}
// isNVIDIADevice checks whether a render device belongs to an NVIDIA GPU by
// reading the PCI vendor ID from sysfs. NVIDIA vendor ID is 0x10de.
func isNVIDIADevice(renderDevPath string) bool {
name := filepath.Base(renderDevPath)
vendorPath := filepath.Join(sysClassDRMDir, name, "device", "vendor")
data, err := os.ReadFile(vendorPath)
if err != nil {
return false
}
return strings.TrimSpace(string(data)) == "0x10de"
}
func hasNVIDIADevice() bool {
if file, err := os.Open(defaultNVIDIAControlDevice); err == nil {
file.Close()
return true
}
matches, err := filepath.Glob(defaultNVIDIADeviceGlob)
if err != nil || len(matches) == 0 {
return false
}
for _, dev := range matches {
if file, err := os.Open(dev); err == nil {
file.Close()
return true
}
}
return false
}
// detectRenderDevice enumerates /dev/dri/renderD* and returns the first
// available device, or empty string if none found.
func detectRenderDevice(driDir string) string {
+322
View File
@@ -0,0 +1,322 @@
package playback
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
type hwAccelTestEnv struct {
devDir string
driDir string
sysDir string
}
type fakeFFmpegProbe struct {
cuda bool
h264NVENC bool
hevcNVENC bool
scaleCUDA bool
uploadCUDA bool
smokeOK bool
hang bool
}
type fakeFFmpegBinary struct {
path string
logPath string
}
func TestResolveHWAccelWithFFmpegAutoPrefersNVENCOverIntel(t *testing.T) {
env := setupHWAccelTest(t)
env.addRenderDevice(t, "renderD128", "0x8086")
env.addRenderDevice(t, "renderD129", "0x10de")
ffmpeg := writeFakeFFmpeg(t, successfulNVENCProbe())
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "nvenc" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want nvenc", got)
}
}
func TestResolveHWAccelWithFFmpegFallsBackToIntelWhenNVENCProbeFails(t *testing.T) {
env := setupHWAccelTest(t)
env.addRenderDevice(t, "renderD128", "0x8086")
env.addRenderDevice(t, "renderD129", "0x10de")
ffmpeg := writeFakeFFmpeg(t, fakeFFmpegProbe{})
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "qsv" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want qsv", got)
}
}
func TestResolveHWAccelWithFFmpegFallsBackToVAAPIWhenNVENCProbeFails(t *testing.T) {
env := setupHWAccelTest(t)
env.addRenderDevice(t, "renderD128", "0x10de")
env.addRenderDevice(t, "renderD129", "0x1002")
ffmpeg := writeFakeFFmpeg(t, fakeFFmpegProbe{})
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "vaapi" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want vaapi", got)
}
}
func TestResolveHWAccelWithFFmpegReturnsNoneWhenNVENCProbeFailsWithoutFallback(t *testing.T) {
env := setupHWAccelTest(t)
env.addRenderDevice(t, "renderD128", "0x10de")
ffmpeg := writeFakeFFmpeg(t, fakeFFmpegProbe{})
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "none" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want none", got)
}
}
func TestResolveHWAccelWithFFmpegUsesNVIDIADeviceNodesWithoutDRM(t *testing.T) {
env := setupHWAccelTest(t)
env.addNVIDIADevice(t, "nvidia0")
ffmpeg := writeFakeFFmpeg(t, successfulNVENCProbe())
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "nvenc" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want nvenc", got)
}
}
func TestExplicitNVENCBypassesFFmpegProbe(t *testing.T) {
setupHWAccelTest(t)
if got := ResolveHWAccelWithFFmpeg("nvenc", "/does/not/exist/ffmpeg"); got != "nvenc" {
t.Fatalf("ResolveHWAccelWithFFmpeg() = %q, want nvenc", got)
}
}
func TestFFmpegSupportsNVENCRequiresCUDAEncodersFiltersAndSmoke(t *testing.T) {
setupHWAccelTest(t)
tests := []struct {
name string
probe fakeFFmpegProbe
}{
{
name: "missing cuda hwaccel",
probe: fakeFFmpegProbe{
h264NVENC: true, hevcNVENC: true, scaleCUDA: true, uploadCUDA: true, smokeOK: true,
},
},
{
name: "missing h264 nvenc encoder",
probe: fakeFFmpegProbe{
cuda: true, hevcNVENC: true, scaleCUDA: true, uploadCUDA: true, smokeOK: true,
},
},
{
name: "missing hevc nvenc encoder",
probe: fakeFFmpegProbe{
cuda: true, h264NVENC: true, scaleCUDA: true, uploadCUDA: true, smokeOK: true,
},
},
{
name: "missing scale cuda filter",
probe: fakeFFmpegProbe{
cuda: true, h264NVENC: true, hevcNVENC: true, uploadCUDA: true, smokeOK: true,
},
},
{
name: "missing hwupload cuda filter",
probe: fakeFFmpegProbe{
cuda: true, h264NVENC: true, hevcNVENC: true, scaleCUDA: true, smokeOK: true,
},
},
{
name: "smoke encode failure",
probe: fakeFFmpegProbe{
cuda: true, h264NVENC: true, hevcNVENC: true, scaleCUDA: true, uploadCUDA: true,
},
},
{
name: "probe timeout",
probe: fakeFFmpegProbe{hang: true},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
resetNVENCProbeCacheForTest()
ffmpeg := writeFakeFFmpeg(t, tt.probe)
if ok, reason := ffmpegSupportsNVENC(ffmpeg.path); ok {
t.Fatalf("ffmpegSupportsNVENC() = true, want false")
} else if reason == "" {
t.Fatalf("ffmpegSupportsNVENC() reason is empty")
}
})
}
}
func TestFFmpegSupportsNVENCCachesByFFmpegPath(t *testing.T) {
env := setupHWAccelTest(t)
env.addRenderDevice(t, "renderD128", "0x10de")
ffmpeg := writeFakeFFmpeg(t, successfulNVENCProbe())
for i := 0; i < 2; i++ {
if got := ResolveHWAccelWithFFmpeg("auto", ffmpeg.path); got != "nvenc" {
t.Fatalf("ResolveHWAccelWithFFmpeg() call %d = %q, want nvenc", i+1, got)
}
}
logData, err := os.ReadFile(ffmpeg.logPath)
if err != nil {
t.Fatalf("read ffmpeg probe log: %v", err)
}
if got := strings.Count(string(logData), "\n"); got != 4 {
t.Fatalf("probe command count = %d, want 4; log:\n%s", got, logData)
}
}
func TestFFmpegSupportsNVENCSmokeProbeUsesSafeFrameDimensions(t *testing.T) {
setupHWAccelTest(t)
ffmpeg := writeFakeFFmpeg(t, successfulNVENCProbe())
if ok, reason := ffmpegSupportsNVENC(ffmpeg.path); !ok {
t.Fatalf("ffmpegSupportsNVENC() = false, want true (reason=%q)", reason)
}
logData, err := os.ReadFile(ffmpeg.logPath)
if err != nil {
t.Fatalf("read ffmpeg probe log: %v", err)
}
logText := string(logData)
if !strings.Contains(logText, "testsrc2=size=640x360:rate=1") {
t.Fatalf("smoke probe should use 640x360 input; log:\n%s", logText)
}
}
func successfulNVENCProbe() fakeFFmpegProbe {
return fakeFFmpegProbe{
cuda: true,
h264NVENC: true,
hevcNVENC: true,
scaleCUDA: true,
uploadCUDA: true,
smokeOK: true,
}
}
func setupHWAccelTest(t *testing.T) *hwAccelTestEnv {
t.Helper()
oldDRIDir := defaultDRIDir
oldNVIDIAControlDevice := defaultNVIDIAControlDevice
oldNVIDIADeviceGlob := defaultNVIDIADeviceGlob
oldSysClassDRMDir := sysClassDRMDir
oldGOOS := currentGOOS
oldProbeTimeout := nvencProbeCommandTimeout
resetNVENCProbeCacheForTest()
tmp := t.TempDir()
env := &hwAccelTestEnv{
devDir: filepath.Join(tmp, "dev"),
driDir: filepath.Join(tmp, "dev", "dri"),
sysDir: filepath.Join(tmp, "sys", "class", "drm"),
}
defaultDRIDir = env.driDir
defaultNVIDIAControlDevice = filepath.Join(env.devDir, "nvidiactl")
defaultNVIDIADeviceGlob = filepath.Join(env.devDir, "nvidia[0-9]*")
sysClassDRMDir = env.sysDir
currentGOOS = "linux"
nvencProbeCommandTimeout = 200 * time.Millisecond
if err := os.MkdirAll(env.driDir, 0o755); err != nil {
t.Fatalf("create test dri dir: %v", err)
}
if err := os.MkdirAll(env.devDir, 0o755); err != nil {
t.Fatalf("create test dev dir: %v", err)
}
t.Cleanup(func() {
defaultDRIDir = oldDRIDir
defaultNVIDIAControlDevice = oldNVIDIAControlDevice
defaultNVIDIADeviceGlob = oldNVIDIADeviceGlob
sysClassDRMDir = oldSysClassDRMDir
currentGOOS = oldGOOS
nvencProbeCommandTimeout = oldProbeTimeout
resetNVENCProbeCacheForTest()
})
return env
}
func (e *hwAccelTestEnv) addRenderDevice(t *testing.T, name string, vendor string) {
t.Helper()
if err := os.WriteFile(filepath.Join(e.driDir, name), []byte{}, 0o600); err != nil {
t.Fatalf("create render device: %v", err)
}
vendorPath := filepath.Join(e.sysDir, name, "device", "vendor")
if err := os.MkdirAll(filepath.Dir(vendorPath), 0o755); err != nil {
t.Fatalf("create vendor dir: %v", err)
}
if err := os.WriteFile(vendorPath, []byte(vendor+"\n"), 0o644); err != nil {
t.Fatalf("write vendor file: %v", err)
}
}
func (e *hwAccelTestEnv) addNVIDIADevice(t *testing.T, name string) {
t.Helper()
if err := os.WriteFile(filepath.Join(e.devDir, name), []byte{}, 0o600); err != nil {
t.Fatalf("create nvidia device: %v", err)
}
}
func writeFakeFFmpeg(t *testing.T, probe fakeFFmpegProbe) fakeFFmpegBinary {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "ffmpeg")
logPath := filepath.Join(dir, "probe.log")
script := "#!/bin/sh\n"
script += fmt.Sprintf("printf '%%s\\n' \"$*\" >> %q\n", logPath)
if probe.hang {
script += "sleep 5\n"
}
script += "case \"$*\" in\n"
script += " *-hwaccels*)\n"
script += " echo 'Hardware acceleration methods:'\n"
if probe.cuda {
script += " echo 'cuda'\n"
}
script += " exit 0 ;;\n"
script += " *-encoders*)\n"
if probe.h264NVENC {
script += " echo ' V..... h264_nvenc NVIDIA NVENC H.264 encoder'\n"
}
if probe.hevcNVENC {
script += " echo ' V..... hevc_nvenc NVIDIA NVENC hevc encoder'\n"
}
script += " exit 0 ;;\n"
script += " *-filters*)\n"
if probe.scaleCUDA {
script += " echo ' ... scale_cuda V->V GPU video scaling'\n"
}
if probe.uploadCUDA {
script += " echo ' ... hwupload_cuda V->V upload CUDA frames'\n"
}
script += " exit 0 ;;\n"
script += " *)\n"
if probe.smokeOK {
script += " exit 0 ;;\n"
} else {
script += " echo 'no capable devices found' >&2\n"
script += " exit 1 ;;\n"
}
script += "esac\n"
if err := os.WriteFile(path, []byte(script), 0o755); err != nil {
t.Fatalf("write fake ffmpeg: %v", err)
}
return fakeFFmpegBinary{path: path, logPath: logPath}
}
func resetNVENCProbeCacheForTest() {
nvencProbeCache.Lock()
defer nvencProbeCache.Unlock()
nvencProbeCache.byPath = make(map[string]nvencProbeResult)
}
+59 -4
View File
@@ -39,7 +39,7 @@ type TranscodeOpts struct {
SegmentDuration int // seconds, default 6
StartSegmentNumber int // -hls_segment_start_number, default 0
FFmpegPath string // optional explicit ffmpeg binary path
HWAccel string // auto, qsv, vaapi, none
HWAccel string // auto, qsv, vaapi, nvenc, none
HWDevice string // e.g., /dev/dri/renderD128 (default if empty)
SubtitleTrackIndex int // -1 = no subtitles
SubtitleBurnIn bool
@@ -294,6 +294,9 @@ func buildFFmpegArgs(opts TranscodeOpts) []string {
} else if opts.HWAccel == "vaapi" {
scale := vaapiScaleFilter(opts.TargetResolution)
args = append(args, "-vf", scale)
} else if opts.HWAccel == "nvenc" {
scale := nvencScaleFilter(opts.TargetResolution)
args = append(args, "-vf", scale)
} else if opts.TargetResolution != "" {
scale := resolutionToScale(opts.TargetResolution)
if scale != "" {
@@ -351,7 +354,7 @@ func buildFFmpegArgs(opts TranscodeOpts) []string {
}
func resolveEffectiveTranscodeHWAccel(opts TranscodeOpts) string {
hwAccel := ResolveHWAccel(opts.HWAccel)
hwAccel := ResolveHWAccelWithFFmpeg(opts.HWAccel, opts.FFmpegPath)
if hwAccel == "" {
return ""
}
@@ -422,12 +425,12 @@ func appendSegmentBoundaryArgs(args []string, opts TranscodeOpts) []string {
fmt.Sprintf("expr:gte(t,n_forced*%d)", opts.SegmentDuration))
}
// Hardware encoders (QSV, VAAPI) may not reliably honor
// Hardware encoders (QSV, VAAPI, NVENC) may not reliably honor
// force_key_frames expressions. Set explicit GOP size so segment
// boundaries always start with an IDR frame. We assume 30 fps as a
// safe ceiling — the GOP will be at most segmentDuration * 30 frames.
// Matches Jellyfin's approach for hardware encoders.
if opts.HWAccel == "qsv" || opts.HWAccel == "vaapi" {
if opts.HWAccel == "qsv" || opts.HWAccel == "vaapi" || opts.HWAccel == "nvenc" {
gopSize := fmt.Sprintf("%d", opts.SegmentDuration*30)
args = append(args, "-g", gopSize, "-keyint_min", gopSize)
}
@@ -465,6 +468,15 @@ func appendHWAccelArgs(args []string, opts TranscodeOpts) []string {
"-hwaccel", "vaapi",
"-hwaccel_output_format", "vaapi",
)
case "nvenc":
args = append(args,
"-hwaccel", "cuda",
"-hwaccel_output_format", "cuda",
"-noautorotate",
)
if hwDevice := strings.TrimSpace(opts.HWDevice); hwDevice != "" {
args = append(args, "-hwaccel_device", hwDevice)
}
}
return args
}
@@ -530,6 +542,26 @@ func appendVideoArgs(args []string, opts TranscodeOpts) []string {
"-maxrate", fmt.Sprintf("%dk", opts.TargetBitrateKbps),
"-bufsize", fmt.Sprintf("%dk", opts.TargetBitrateKbps*2))
}
case opts.HWAccel == "nvenc" && codec == "h264":
args = append(args, "-c:v", "h264_nvenc", "-rc:v", "vbr")
if hasBitrateCap {
args = append(args,
"-b:v", fmt.Sprintf("%dk", opts.TargetBitrateKbps),
"-maxrate", fmt.Sprintf("%dk", opts.TargetBitrateKbps),
"-bufsize", fmt.Sprintf("%dk", opts.TargetBitrateKbps*2))
} else {
args = append(args, "-cq:v", "23", "-b:v", "0")
}
case opts.HWAccel == "nvenc" && codec == "hevc":
args = append(args, "-c:v", "hevc_nvenc", "-rc:v", "vbr")
if hasBitrateCap {
args = append(args,
"-b:v", fmt.Sprintf("%dk", opts.TargetBitrateKbps),
"-maxrate", fmt.Sprintf("%dk", opts.TargetBitrateKbps),
"-bufsize", fmt.Sprintf("%dk", opts.TargetBitrateKbps*2))
} else {
args = append(args, "-cq:v", "28", "-b:v", "0")
}
default:
// CPU fallback — match Jellyfin's proven browser-compatible settings.
// Force yuv420p to ensure 8-bit output (10-bit sources produce High 10
@@ -609,6 +641,10 @@ func appendSubtitleBurnInArgs(args []string, opts TranscodeOpts) []string {
// VAAPI-only: download, apply CPU filters, convert to nv12, upload back.
vf := "hwdownload,format=yuv420p," + cpuFilters + ",format=nv12,hwupload"
args = append(args, "-vf", vf)
case "nvenc":
// NVENC/CUDA: download to CPU for subtitle rendering, then upload back.
vf := "hwdownload,format=yuv420p," + cpuFilters + ",format=nv12,hwupload_cuda"
args = append(args, "-vf", vf)
default:
// CPU encoding: filters run directly on decoded frames.
args = append(args, "-vf", cpuFilters)
@@ -679,6 +715,25 @@ func vaapiScaleFilter(res string) string {
}
}
func nvencScaleFilter(res string) string {
switch res {
case "2160p":
return "scale_cuda=w=-2:h=2160:format=nv12"
case "1080p":
return "scale_cuda=w=-2:h=1080:format=nv12"
case "720p":
return "scale_cuda=w=-2:h=720:format=nv12"
case "480p":
return "scale_cuda=w=-2:h=480:format=nv12"
case "420p":
return "scale_cuda=w=-2:h=420:format=nv12"
case "328p":
return "scale_cuda=w=-2:h=328:format=nv12"
default:
return "scale_cuda=format=nv12"
}
}
// filterPathReplacer escapes special characters in file paths for ffmpeg filter syntax.
var filterPathReplacer = strings.NewReplacer(
"'", "'\\''",
+39 -2
View File
@@ -113,7 +113,7 @@ func TestBuildFFmpegArgs_CopyVideoSeekPreservesCodecCopy(t *testing.T) {
t.Fatalf("copy-mode seek should preserve -c:v copy: %s", joined)
}
// Must not contain any video encoder.
for _, enc := range []string{"h264_qsv", "h264_vaapi", "libx264", "hevc_qsv"} {
for _, enc := range []string{"h264_qsv", "h264_vaapi", "h264_nvenc", "libx264", "hevc_qsv", "hevc_nvenc"} {
if strings.Contains(joined, enc) {
t.Fatalf("copy-mode seek should not use encoder %s: %s", enc, joined)
}
@@ -166,7 +166,7 @@ func TestBuildFFmpegArgs_MPEG2CopyVideoUsesMPEGTS(t *testing.T) {
if strings.Contains(joined, "movflags=+frag_discont") {
t.Fatalf("mpeg2 MPEG-TS copy-mode should not use fMP4 movflags: %s", joined)
}
for _, enc := range []string{"h264_qsv", "h264_vaapi", "libx264", "hevc_qsv", "libx265"} {
for _, enc := range []string{"h264_qsv", "h264_vaapi", "h264_nvenc", "libx264", "hevc_qsv", "hevc_nvenc", "libx265"} {
if strings.Contains(joined, enc) {
t.Fatalf("mpeg2 copy-mode should not use encoder %s: %s", enc, joined)
}
@@ -234,6 +234,11 @@ func TestResolveEffectiveTranscodeHWAccel(t *testing.T) {
opts: TranscodeOpts{HWAccel: "vaapi", SourceVideoCodec: "mpeg4", TargetCodecVideo: "h264"},
want: "none",
},
{
name: "nvenc passthrough",
opts: TranscodeOpts{HWAccel: "nvenc", SourceVideoCodec: "h264", TargetCodecVideo: "h264"},
want: "nvenc",
},
}
for _, tt := range tests {
@@ -247,6 +252,38 @@ func TestResolveEffectiveTranscodeHWAccel(t *testing.T) {
}
}
func TestBuildFFmpegArgs_NVENCH264UsesCudaPipeline(t *testing.T) {
args := buildFFmpegArgs(TranscodeOpts{
InputPath: "/media/movie.mkv",
OutputDir: "/tmp/out",
SessionID: "session-nvenc",
SourceVideoCodec: "h264",
TargetCodecVideo: "h264",
TargetCodecAudio: "aac",
SegmentDuration: 2,
HWAccel: "nvenc",
TargetResolution: "720p",
TargetBitrateKbps: 2000,
})
joined := strings.Join(args, " ")
if !strings.Contains(joined, "-hwaccel cuda") {
t.Fatalf("nvenc args should enable cuda hwaccel: %s", joined)
}
if !strings.Contains(joined, "-c:v h264_nvenc") {
t.Fatalf("nvenc args should use h264_nvenc encoder: %s", joined)
}
if !strings.Contains(joined, "-vf scale_cuda=w=-2:h=720:format=nv12") {
t.Fatalf("nvenc args should use scale_cuda, not software scale: %s", joined)
}
if strings.Contains(joined, "-vf scale=-2:720") {
t.Fatalf("nvenc args must not use software scale on cuda frames: %s", joined)
}
if !strings.Contains(joined, "-b:v 2000k -maxrate 2000k -bufsize 4000k") {
t.Fatalf("nvenc args should include bitrate cap controls: %s", joined)
}
}
func TestBuildFFmpegArgs_VAAPIScalingUsesHardwareFilter(t *testing.T) {
args := buildFFmpegArgs(TranscodeOpts{
InputPath: "/media/movie.mkv",
+47 -10
View File
@@ -13,14 +13,33 @@ import (
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
type Repository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// apiKeyAAD binds a request_integrations api_key_ref ciphertext to its row.
func apiKeyAAD(id string) string {
return secret.RowAAD("request_integrations", "api_key_ref", id)
}
// encryptAPIKey encrypts a non-empty, trimmed API key bound to the integration
// id. An empty key returns "" so the update path's keep-existing CASE sentinel
// still fires (and an absent key stays absent).
func (r *Repository) encryptAPIKey(id, apiKey string) (string, error) {
apiKey = strings.TrimSpace(apiKey)
if apiKey == "" {
return "", nil
}
return r.cipher.Encrypt(apiKey, apiKeyAAD(id))
}
func (r *Repository) GetSettings(ctx context.Context) (Settings, error) {
@@ -466,7 +485,7 @@ func (r *Repository) ListIntegrations(ctx context.Context) ([]Integration, error
var out []Integration
for rows.Next() {
integration, err := scanIntegration(rows)
integration, err := r.scanIntegration(rows)
if err != nil {
return nil, err
}
@@ -481,7 +500,7 @@ func (r *Repository) ListIntegrations(ctx context.Context) ([]Integration, error
func (r *Repository) GetIntegration(ctx context.Context, id string) (*Integration, error) {
row := r.pool.QueryRow(ctx, `SELECT `+integrationColumns+
` FROM request_integrations WHERE id = $1`, id)
i, err := scanIntegration(row)
i, err := r.scanIntegration(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
@@ -510,6 +529,10 @@ func (r *Repository) insertIntegration(ctx context.Context, exec requestExecutor
if err != nil {
return nil, fmt.Errorf("marshal options: %w", err)
}
apiKeyRef, err := r.encryptAPIKey(i.ID, i.APIKeyRef)
if err != nil {
return nil, fmt.Errorf("encrypt api key: %w", err)
}
row := exec.QueryRow(ctx, `
INSERT INTO request_integrations (
id, kind, name, enabled, base_url, api_key_ref, root_folder,
@@ -519,11 +542,11 @@ func (r *Repository) insertIntegration(ctx context.Context, exec requestExecutor
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17, now())
RETURNING `+integrationColumns,
i.ID, i.Kind, strings.TrimSpace(i.Name), i.Enabled, strings.TrimSpace(i.BaseURL),
strings.TrimSpace(i.APIKeyRef), strings.TrimSpace(i.RootFolder), i.QualityProfileID,
apiKeyRef, strings.TrimSpace(i.RootFolder), i.QualityProfileID,
int32Slice(i.Tags), i.Is4K, i.IsDefault, i.IsDefault4K, i.AnimeEnabled,
i.AnimeQualityProfileID, strings.TrimSpace(i.AnimeRootFolder), int32Slice(i.AnimeTags),
options)
out, err := scanIntegration(row)
out, err := r.scanIntegration(row)
if err != nil {
return nil, fmt.Errorf("create request integration: %w", err)
}
@@ -540,6 +563,12 @@ func (r *Repository) updateIntegration(ctx context.Context, exec requestExecutor
if err != nil {
return nil, fmt.Errorf("marshal options: %w", err)
}
// Encrypt the incoming key; an empty result preserves the keep-existing CASE
// (a blank edit leaves the stored key untouched).
apiKeyRef, err := r.encryptAPIKey(i.ID, i.APIKeyRef)
if err != nil {
return nil, fmt.Errorf("encrypt api key: %w", err)
}
row := exec.QueryRow(ctx, `
UPDATE request_integrations SET
name=$2, enabled=$3, base_url=$4,
@@ -551,11 +580,11 @@ func (r *Repository) updateIntegration(ctx context.Context, exec requestExecutor
WHERE id=$1
RETURNING `+integrationColumns,
i.ID, strings.TrimSpace(i.Name), i.Enabled, strings.TrimSpace(i.BaseURL),
strings.TrimSpace(i.APIKeyRef), strings.TrimSpace(i.RootFolder), i.QualityProfileID,
apiKeyRef, strings.TrimSpace(i.RootFolder), i.QualityProfileID,
int32Slice(i.Tags), i.Is4K, i.IsDefault, i.IsDefault4K, i.AnimeEnabled,
i.AnimeQualityProfileID, strings.TrimSpace(i.AnimeRootFolder), int32Slice(i.AnimeTags),
options)
out, err := scanIntegration(row)
out, err := r.scanIntegration(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
@@ -763,7 +792,7 @@ type integrationScanner interface {
Scan(dest ...any) error
}
func scanIntegration(row integrationScanner) (Integration, error) {
func (r *Repository) scanIntegration(row integrationScanner) (Integration, error) {
var i Integration
var quality, animeQuality sql.NullInt64
var tags, animeTags []int32
@@ -777,6 +806,14 @@ func scanIntegration(row integrationScanner) (Integration, error) {
); err != nil {
return Integration{}, err
}
// Decrypt the stored api key (read-path contract: legacy plaintext passes
// through, enc:v1: values decrypt, corrupt ciphertext errors). Callers
// receive the literal key — there is no longer a ref/literal ambiguity.
apiKey, err := r.cipher.DecryptIfEncrypted(i.APIKeyRef, apiKeyAAD(i.ID))
if err != nil {
return Integration{}, fmt.Errorf("decrypt request integration %s api key: %w", i.ID, err)
}
i.APIKeyRef = apiKey
if quality.Valid {
v := int(quality.Int64)
i.QualityProfileID = &v
+7 -41
View File
@@ -27,10 +27,6 @@ type TMDBExternalIDClient interface {
const externalIDHydrationConcurrency = 4
type SecretResolver interface {
Get(ctx context.Context, key string) (string, error)
}
type MovieFulfillmentAdapter interface {
SubmitMovie(ctx context.Context, req Request, integration Integration) (FulfillmentResult, error)
}
@@ -66,7 +62,6 @@ type Service struct {
store Store
tmdb TMDBClient
presence PresenceResolver
secrets SecretResolver
movieAdapter MovieFulfillmentAdapter
seriesAdapter SeriesFulfillmentAdapter
entitlements EntitlementResolver
@@ -91,10 +86,6 @@ func NewService(store Store, tmdbClient TMDBClient, presence PresenceResolver) *
}
}
func (s *Service) SetSecretResolver(resolver SecretResolver) {
s.secrets = resolver
}
func (s *Service) SetFulfillmentAdapters(movie MovieFulfillmentAdapter, series SeriesFulfillmentAdapter) {
s.movieAdapter = movie
s.seriesAdapter = series
@@ -858,12 +849,9 @@ func (s *Service) LoadIntegrationOptions(ctx context.Context, viewer Viewer, int
if strings.TrimSpace(normalized.APIKeyRef) == "" {
return nil, fmt.Errorf("%w: api_key_ref is required", ErrInvalidInput)
}
// APIKeyRef is already the literal key: a saved instance was decrypted by
// GetIntegration above; a new instance carries the key the admin just typed.
resolved := normalized
apiKey, err := s.resolveAPIKey(ctx, resolved)
if err != nil {
return nil, err
}
resolved.APIKeyRef = apiKey
switch resolved.Kind {
case "radarr":
@@ -1200,15 +1188,10 @@ func (s *Service) submitApprovedRequest(ctx context.Context, req Request, actor
// the request aggregate). Returns the latest request snapshot.
func (s *Service) submitPlannedTarget(ctx context.Context, req Request, pt plannedTarget, actor Viewer) (*Request, error) {
resolved := resolveInstance(pt)
apiKey, err := s.resolveAPIKey(ctx, resolved)
if err != nil || apiKey == "" {
msg := "missing api key"
if err != nil {
msg = err.Error()
}
return s.createFailedTarget(ctx, req, pt, resolved, msg, actor)
// The repo already decrypted the stored key; a blank one means unconfigured.
if strings.TrimSpace(resolved.APIKeyRef) == "" {
return s.createFailedTarget(ctx, req, pt, resolved, "missing api key", actor)
}
resolved.APIKeyRef = apiKey
target, err := s.store.CreateTarget(ctx, Target{
RequestID: req.ID, IntegrationID: resolved.ID, IntegrationKind: resolved.Kind,
@@ -1338,11 +1321,10 @@ func (s *Service) reconcileRequest(ctx context.Context, req Request) (reconcileC
if !ok {
continue
}
apiKey, err := s.resolveAPIKey(ctx, in)
if err != nil || apiKey == "" {
// in.APIKeyRef was decrypted by ListIntegrations; skip if unconfigured.
if strings.TrimSpace(in.APIKeyRef) == "" {
continue
}
in.APIKeyRef = apiKey
probe := req
probe.ExternalID = t.ExternalID
st, err := s.checkFulfillmentStatus(ctx, probe, in)
@@ -1431,22 +1413,6 @@ func (s *Service) checkFulfillmentStatus(ctx context.Context, req Request, resol
}
}
func (s *Service) resolveAPIKey(ctx context.Context, integration Integration) (string, error) {
value := strings.TrimSpace(integration.APIKeyRef)
if value == "" || s.secrets == nil {
return value, nil
}
resolved, err := s.secrets.Get(ctx, value)
if err != nil {
return "", err
}
resolved = strings.TrimSpace(resolved)
if resolved == "" {
return value, nil
}
return resolved, nil
}
func integrationIsConfigured(integration Integration) bool {
return integration.Enabled &&
strings.TrimSpace(integration.BaseURL) != "" &&
+4 -51
View File
@@ -177,7 +177,6 @@ func TestCreateRequestAutoApprovesWithConfiguredIntegration(t *testing.T) {
ExternalStatus: "queued",
}}
service := newTestService(store)
service.SetSecretResolver(fakeSecrets{"request.radarr.api_key": "radarr-key"})
service.SetFulfillmentAdapters(adapter, nil)
req, err := service.CreateRequest(context.Background(), testViewer(1), CreateRequestInput{
@@ -195,39 +194,6 @@ func TestCreateRequestAutoApprovesWithConfiguredIntegration(t *testing.T) {
}
}
func TestCreateRequestAutoApprovalSecretFailureMarksTargetFailed(t *testing.T) {
store := newFakeStore()
store.settings.RequestsEnabled = true
store.settings.GlobalAutoApprovalEnabled = true
qualityProfileID := 1
store.integrations = []Integration{{
Kind: "radarr",
Enabled: true,
IsDefault: true,
BaseURL: "http://radarr.local",
APIKeyRef: "requests.radarr.api_key",
RootFolder: "/movies",
QualityProfileID: &qualityProfileID,
}}
service := newTestService(store)
service.SetSecretResolver(fakeSecretError{err: errors.New("secret lookup unavailable")})
service.SetFulfillmentAdapters(&fakeMovieAdapter{}, nil)
req, err := service.CreateRequest(context.Background(), testViewer(1), CreateRequestInput{
MediaType: MediaTypeMovie,
TMDBID: 550,
Title: "Fight Club",
})
if err != nil {
t.Fatalf("CreateRequest returned error: %v", err)
}
// The auto-approval gate no longer resolves secrets; submission does. A
// secret-resolution failure now surfaces as a failed fulfillment target.
if req.Outcome != OutcomeFailed {
t.Fatalf("outcome = %q, want failed (secret resolution failed during submit)", req.Outcome)
}
}
func TestCreateRequestAutoApprovalSubmitsMovie(t *testing.T) {
store := newFakeStore()
store.settings.RequestsEnabled = true
@@ -238,7 +204,7 @@ func TestCreateRequestAutoApprovalSubmitsMovie(t *testing.T) {
Enabled: true,
IsDefault: true,
BaseURL: "http://radarr.local",
APIKeyRef: "requests.radarr.api_key",
APIKeyRef: "radarr-key",
RootFolder: "/movies",
QualityProfileID: &qualityProfileID,
}}
@@ -248,7 +214,6 @@ func TestCreateRequestAutoApprovalSubmitsMovie(t *testing.T) {
ExternalStatus: "queued",
}}
service := newTestService(store)
service.SetSecretResolver(fakeSecrets{"requests.radarr.api_key": "radarr-key"})
service.SetFulfillmentAdapters(adapter, nil)
req, err := service.CreateRequest(context.Background(), testViewer(1), CreateRequestInput{
@@ -265,8 +230,10 @@ func TestCreateRequestAutoApprovalSubmitsMovie(t *testing.T) {
if adapter.calls != 1 {
t.Fatalf("adapter calls = %d, want 1", adapter.calls)
}
// The repo decrypts api_key_ref on read, so the adapter receives the literal
// key verbatim — no resolver indirection.
if got := adapter.gotIntegration.APIKeyRef; got != "radarr-key" {
t.Fatalf("adapter api key = %q, want resolved key", got)
t.Fatalf("adapter api key = %q, want radarr-key", got)
}
}
@@ -1772,17 +1739,3 @@ func TestSubmitApprovedFansOutDualQuality(t *testing.T) {
t.Fatalf("expected 2 submissions (hd+uhd), got %d: %v", len(rec.ids), rec.ids)
}
}
type fakeSecrets map[string]string
func (f fakeSecrets) Get(_ context.Context, key string) (string, error) {
return f[key], nil
}
type fakeSecretError struct {
err error
}
func (f fakeSecretError) Get(context.Context, string) (string, error) {
return "", f.err
}
+8 -2
View File
@@ -2645,10 +2645,16 @@ func (r *FileRepository) ListByEpisodeIDs(ctx context.Context, episodeIDs []stri
// UpdateContentID sets the content_id on a media file, linking it to a matched
// media item. This is called by the matcher after a successful resolution.
func (r *FileRepository) UpdateContentID(ctx context.Context, fileID int, contentID string) error {
_, err := r.pool.Exec(ctx,
tag, err := r.pool.Exec(ctx,
"UPDATE media_files SET content_id = $1, updated_at = NOW() WHERE id = $2",
contentID, fileID)
return err
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return ErrFileNotFound
}
return nil
}
// ReplaceContentID reassigns all files linked to one content item to another.
+247
View File
@@ -0,0 +1,247 @@
package secret
import (
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
// Executor is the minimal database surface the backfill needs; *pgxpool.Pool
// satisfies it.
type Executor interface {
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error)
}
// BackfillTarget describes one literal-credential column to sweep from plaintext
// to ciphertext in place. All field values are compile-time constants (table,
// column, and SQL key expressions), never user input, so interpolating them into
// the statements is safe.
type BackfillTarget struct {
// Table is the owning table, e.g. "subtitle_provider_config".
Table string
// Column is the secret column, e.g. "api_key".
Column string
// KeyExpr is the SQL expression selecting the row's stable update key, used
// in the UPDATE ... WHERE. Defaults to "id" when empty. It is rendered to
// text for comparison, e.g. "id::text" or "provider_name".
KeyExpr string
// AADExpr is the SQL expression producing the AAD pk part. Empty means use
// KeyExpr. For tables whose token AAD binds to a business key rather than the
// surrogate id (watch_provider_connections), this differs from KeyExpr.
AADExpr string
}
func (t BackfillTarget) keyExpr() string {
if t.KeyExpr == "" {
return "id"
}
return t.KeyExpr
}
func (t BackfillTarget) aadExpr() string {
if t.AADExpr == "" {
return t.keyExpr()
}
return t.AADExpr
}
// ColumnBackfillTargets is the set of literal-credential columns the startup
// backfill encrypts in place. The two arr api_key_ref columns are NOT here: they
// may hold a legacy server_settings reference rather than a literal key and need
// the separate resolve-then-encrypt path. The AAD expressions must match the
// repos' read-path AAD (secret.RowAAD with the same id string) exactly, or reads
// would fail to decrypt.
func ColumnBackfillTargets() []BackfillTarget {
const watchTupleAAD = "provider || ':' || user_id::text || ':' || profile_id"
return []BackfillTarget{
// Subtitles — AAD keyed by provider_name.
{Table: "subtitle_provider_config", Column: "api_key", KeyExpr: "provider_name"},
{Table: "subtitle_provider_config", Column: "password", KeyExpr: "provider_name"},
// Watch-sync — AAD keyed by the (provider, user_id, profile_id) tuple to
// match watchsync.TokenAAD (the id is DB-assigned on upsert).
{Table: "watch_provider_connections", Column: "access_token", KeyExpr: "id::text", AADExpr: watchTupleAAD},
{Table: "watch_provider_connections", Column: "refresh_token", KeyExpr: "id::text", AADExpr: watchTupleAAD},
// Webhook-sync — only access_token (webhook_secret is equality-looked-up).
{Table: "webhook_sync_connections", Column: "access_token", KeyExpr: "id::text"},
// History import — admin token + session tokens, keyed by row id.
{Table: "history_import_sources", Column: "admin_token", KeyExpr: "id::text"},
{Table: "history_import_connect_sessions", Column: "connect_access_token", KeyExpr: "id::text"},
{Table: "history_import_plex_sessions", Column: "auth_token", KeyExpr: "id::text"},
// Jellyfin-compat sessions — bridged Silo access/refresh tokens, keyed by
// the session token (the table's primary key; itself left in plaintext).
{Table: "jellycompat_sessions", Column: "streamapp_access_token", KeyExpr: "token"},
{Table: "jellycompat_sessions", Column: "streamapp_refresh_token", KeyExpr: "token"},
}
}
// BackfillColumns encrypts every plaintext value across the given targets in
// place. It is idempotent (rows already carrying an enc:v1: envelope are skipped
// by the SELECT filter, and the UPDATE's "AND col = original" guard makes
// concurrent multi-node boots converge without double-encrypting) and
// best-effort: per-row and per-target failures are collected and returned but
// never abort the sweep. Returns the count of newly encrypted rows.
func BackfillColumns(ctx context.Context, db Executor, cipher *Cipher, targets []BackfillTarget) (int, error) {
total := 0
var errs []error
for _, t := range targets {
n, err := backfillTarget(ctx, db, cipher, t)
total += n
if err != nil {
errs = append(errs, fmt.Errorf("%s.%s: %w", t.Table, t.Column, err))
}
}
return total, errors.Join(errs...)
}
// candidate is one plaintext row to encrypt.
type candidate struct {
key string // update WHERE key (rendered text)
aadID string // AAD pk part
value string // current plaintext value
}
func backfillTarget(ctx context.Context, db Executor, cipher *Cipher, t BackfillTarget) (int, error) {
// Select candidates first (can't UPDATE while iterating the same rows).
selectSQL := fmt.Sprintf(
`SELECT %s, %s, %s FROM %s WHERE %s IS NOT NULL AND %s <> '' AND %s NOT LIKE 'enc:v1:%%'`,
t.keyExpr(), t.aadExpr(), t.Column, t.Table, t.Column, t.Column, t.Column,
)
rows, err := db.Query(ctx, selectSQL)
if err != nil {
return 0, fmt.Errorf("select candidates: %w", err)
}
var candidates []candidate
for rows.Next() {
var c candidate
if err := rows.Scan(&c.key, &c.aadID, &c.value); err != nil {
rows.Close()
return 0, fmt.Errorf("scan candidate: %w", err)
}
candidates = append(candidates, c)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate candidates: %w", err)
}
updateSQL := fmt.Sprintf(
`UPDATE %s SET %s = $1 WHERE %s = $2 AND %s = $3`,
t.Table, t.Column, t.keyExpr(), t.Column,
)
count := 0
var errs []error
for _, c := range candidates {
ct, changed, err := cipher.EncryptIfPlaintext(c.value, RowAAD(t.Table, t.Column, c.aadID))
if err != nil {
errs = append(errs, fmt.Errorf("encrypt key=%s: %w", c.key, err))
continue
}
if !changed {
continue
}
tag, err := db.Exec(ctx, updateSQL, ct, c.key, c.value)
if err != nil {
errs = append(errs, fmt.Errorf("update key=%s: %w", c.key, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
// ReferenceResolver resolves a legacy api_key_ref value that may be a
// server_settings key name into the real credential. It returns "" when the
// value is not a known reference (i.e. the row already holds a literal key). In
// production this is the EncryptedSettingsRepo's Get, which decrypts a sensitive
// target and passes through a plaintext custom key.
type ReferenceResolver func(ctx context.Context, ref string) (string, error)
// ArrKeyBackfillTargets are the two arr api_key_ref columns that need the
// resolve-then-encrypt path (their legacy rows may reference server_settings).
func ArrKeyBackfillTargets() []BackfillTarget {
return []BackfillTarget{
{Table: "request_integrations", Column: "api_key_ref", KeyExpr: "id::text"},
{Table: "autoscan_connections", Column: "api_key_ref", KeyExpr: "id::text"},
}
}
// BackfillReferencedColumns encrypts the arr api_key_ref columns, first
// collapsing any legacy server_settings reference to the real credential. For
// each not-yet-encrypted row it calls resolve(value): a non-empty result means
// the row held a reference (encrypt the resolved credential), otherwise the row
// held a literal key (encrypt the value). Run this AFTER the sensitive-settings
// backfill so referenced settings are already consistent. Idempotent and
// best-effort. Returns the count of newly encrypted rows.
func BackfillReferencedColumns(ctx context.Context, db Executor, cipher *Cipher, resolve ReferenceResolver, targets []BackfillTarget) (int, error) {
total := 0
var errs []error
for _, t := range targets {
n, err := backfillReferencedTarget(ctx, db, cipher, resolve, t)
total += n
if err != nil {
errs = append(errs, fmt.Errorf("%s.%s: %w", t.Table, t.Column, err))
}
}
return total, errors.Join(errs...)
}
func backfillReferencedTarget(ctx context.Context, db Executor, cipher *Cipher, resolve ReferenceResolver, t BackfillTarget) (int, error) {
selectSQL := fmt.Sprintf(
`SELECT %s, %s FROM %s WHERE %s IS NOT NULL AND %s <> '' AND %s NOT LIKE 'enc:v1:%%'`,
t.keyExpr(), t.Column, t.Table, t.Column, t.Column, t.Column,
)
rows, err := db.Query(ctx, selectSQL)
if err != nil {
return 0, fmt.Errorf("select candidates: %w", err)
}
type rec struct{ key, value string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.key, &r.value); err != nil {
rows.Close()
return 0, fmt.Errorf("scan candidate: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate candidates: %w", err)
}
updateSQL := fmt.Sprintf(
`UPDATE %s SET %s = $1 WHERE %s = $2 AND %s = $3`,
t.Table, t.Column, t.keyExpr(), t.Column,
)
count := 0
var errs []error
for _, r := range recs {
credential := r.value
if resolved, err := resolve(ctx, r.value); err != nil {
errs = append(errs, fmt.Errorf("resolve key=%s: %w", r.key, err))
continue
} else if resolved != "" {
credential = resolved // the row held a reference, not a literal key
}
ct, err := cipher.Encrypt(credential, RowAAD(t.Table, t.Column, r.key))
if err != nil {
errs = append(errs, fmt.Errorf("encrypt key=%s: %w", r.key, err))
continue
}
tag, err := db.Exec(ctx, updateSQL, ct, r.key, r.value)
if err != nil {
errs = append(errs, fmt.Errorf("update key=%s: %w", r.key, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
+144
View File
@@ -0,0 +1,144 @@
package secret
import (
"context"
"fmt"
"testing"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
// fakeRows yields preset all-string rows for the backfill SELECT.
type fakeRows struct {
rows [][]string
i int
}
func (f *fakeRows) Next() bool { f.i++; return f.i <= len(f.rows) }
func (f *fakeRows) Scan(dest ...any) error {
row := f.rows[f.i-1]
if len(dest) != len(row) {
return fmt.Errorf("fakeRows: scan into %d dests, row has %d", len(dest), len(row))
}
for j, d := range dest {
p, ok := d.(*string)
if !ok {
return fmt.Errorf("fakeRows: dest %d is %T, want *string", j, d)
}
*p = row[j]
}
return nil
}
func (f *fakeRows) Close() {}
func (f *fakeRows) Err() error { return nil }
func (f *fakeRows) CommandTag() pgconn.CommandTag { return pgconn.CommandTag{} }
func (f *fakeRows) FieldDescriptions() []pgconn.FieldDescription { return nil }
func (f *fakeRows) Values() ([]any, error) { return nil, nil }
func (f *fakeRows) RawValues() [][]byte { return nil }
func (f *fakeRows) Conn() *pgx.Conn { return nil }
type capturedExec struct {
args []any
}
// fakeExec returns a fixed candidate set from Query and captures every Exec.
type fakeExec struct {
rows [][]string
execs []capturedExec
}
func (f *fakeExec) Query(context.Context, string, ...any) (pgx.Rows, error) {
return &fakeRows{rows: f.rows}, nil
}
func (f *fakeExec) Exec(_ context.Context, _ string, args ...any) (pgconn.CommandTag, error) {
f.execs = append(f.execs, capturedExec{args: args})
return pgconn.NewCommandTag("UPDATE 1"), nil
}
func TestBackfillColumns_EncryptsBindsAndSkipsEncrypted(t *testing.T) {
c := newTestCipher(t)
target := BackfillTarget{Table: "subtitle_provider_config", Column: "api_key", KeyExpr: "provider_name"}
// Pre-encrypted row must be skipped by the in-loop EncryptIfPlaintext guard.
preEnc, err := c.Encrypt("already", RowAAD(target.Table, target.Column, "opensubtitles"))
if err != nil {
t.Fatalf("seed encrypt: %v", err)
}
db := &fakeExec{rows: [][]string{
// {key, aadID, value}; for this target aadID == key (provider_name).
{"subdl", "subdl", "plain-subdl-key"},
{"opensubtitles", "opensubtitles", preEnc},
}}
n, err := BackfillColumns(context.Background(), db, c, []BackfillTarget{target})
if err != nil {
t.Fatalf("BackfillColumns: %v", err)
}
if n != 1 {
t.Fatalf("encrypted = %d, want 1 (pre-encrypted row skipped)", n)
}
if len(db.execs) != 1 {
t.Fatalf("exec count = %d, want 1", len(db.execs))
}
// args: [ciphertext, key, originalValue]
got := db.execs[0].args
ct, key, orig := got[0].(string), got[1].(string), got[2].(string)
if key != "subdl" || orig != "plain-subdl-key" {
t.Fatalf("update bound key=%q orig=%q", key, orig)
}
if !IsEncrypted(ct) {
t.Fatalf("update value %q is not enc:v1:", ct)
}
// Ciphertext must decrypt under the row-bound AAD back to the original.
plain, err := c.Decrypt(ct, RowAAD(target.Table, target.Column, key))
if err != nil || plain != "plain-subdl-key" {
t.Fatalf("decrypt = (%q, %v), want original", plain, err)
}
}
func TestBackfillReferencedColumns_ResolveThenEncrypt(t *testing.T) {
c := newTestCipher(t)
target := BackfillTarget{Table: "request_integrations", Column: "api_key_ref", KeyExpr: "id::text"}
// resolve mimics the settings decorator: a known settings key resolves to the
// real credential; anything else is a literal (returns "").
resolve := func(_ context.Context, ref string) (string, error) {
if ref == "requests.radarr.api_key" {
return "REAL-RADARR-KEY", nil
}
return "", nil
}
db := &fakeExec{rows: [][]string{
{"int-1", "requests.radarr.api_key"}, // a reference → encrypt the resolved key
{"int-2", "literal-key-xyz"}, // a literal → encrypt the value itself
}}
n, err := BackfillReferencedColumns(context.Background(), db, c, resolve, []BackfillTarget{target})
if err != nil {
t.Fatalf("BackfillReferencedColumns: %v", err)
}
if n != 2 || len(db.execs) != 2 {
t.Fatalf("encrypted=%d execs=%d, want 2/2", n, len(db.execs))
}
want := map[string]string{
"int-1": "REAL-RADARR-KEY", // resolved, not the setting name
"int-2": "literal-key-xyz",
}
for _, e := range db.execs {
ct, key, orig := e.args[0].(string), e.args[1].(string), e.args[2].(string)
plain, err := c.Decrypt(ct, RowAAD(target.Table, target.Column, key))
if err != nil {
t.Fatalf("decrypt for %s: %v", key, err)
}
if plain != want[key] {
t.Fatalf("row %s encrypted %q, want %q", key, plain, want[key])
}
// The UPDATE guard ($3) must be the ORIGINAL stored value so concurrent
// boots converge (the reference name for int-1, the literal for int-2).
if key == "int-1" && orig != "requests.radarr.api_key" {
t.Fatalf("int-1 guard = %q, want the original reference", orig)
}
}
}
+204
View File
@@ -0,0 +1,204 @@
// Package secret provides at-rest symmetric encryption for server-owned
// credentials (arr API keys, S3 keys, sensitive server_settings, per-table
// access tokens). Callers store the resulting ciphertext directly in the column
// that previously held the plaintext credential and decrypt on read, so the
// secret never lives in the database as naked text.
//
// The one primitive everything reuses is AES-256-GCM with a random 12-byte
// nonce and a versioned envelope (enc:v1:<base64url(nonce‖sealed)>). The 32-byte
// data key is HKDF-SHA256 derived from a master key (the SECRET_KEY env value)
// with a domain-separation label, so the key that protects the data lives
// outside Postgres and encrypted secrets survive a full database dump.
//
// Each ciphertext is GCM-bound to its logical row via additional-authenticated
// data (AAD) — "table:column:<pk>" (or "server_settings:<key>") — so a
// DB-write attacker cannot transplant a credential blob into another row or
// column: decrypting with a different AAD fails authentication.
package secret
import (
"crypto/aes"
"crypto/cipher"
"crypto/hkdf"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"errors"
"fmt"
"strings"
)
const (
// envelopePrefix tags a value as v1 ciphertext. It is exactly 7 ASCII
// characters, is not valid base64url, and does not collide with any real
// credential prefix (sa_ tokens, UUIDs, JWTs, URLs, integers) in practice.
envelopePrefix = "enc:v1:"
// hkdfInfo domain-separates the derived data key from any other use of the
// master key. Bump it alongside the envelope version on a future key
// rotation (enc:v2: with "silo/data-encryption/v2").
hkdfInfo = "silo/data-encryption/v1"
// MinMasterKeyLen is the minimum acceptable master-key length in bytes. The
// bootstrap loader enforces the same floor on SECRET_KEY before New is ever
// called.
MinMasterKeyLen = 32
// dataKeyLen is the AES-256 key size.
dataKeyLen = 32
)
// ErrUnknownVersion is returned by Decrypt when the envelope carries a version
// this build does not understand. It is the forward-compatibility hook for a
// future enc:v2: key rotation: an older binary surfaces an explicit error
// instead of silently mishandling a newer ciphertext.
var ErrUnknownVersion = errors.New("secret: unknown ciphertext version")
// Cipher encrypts and decrypts short credential strings with AES-256-GCM. The
// data key is derived once in New and the AEAD is built once and reused, so the
// hot path is a single Seal/Open with no per-call key schedule. A Cipher is
// safe for concurrent use by multiple goroutines.
type Cipher struct {
gcm cipher.AEAD
}
// New derives a Cipher from the master key (the raw SECRET_KEY value). The key
// must be at least MinMasterKeyLen bytes; New returns an error otherwise so a
// short or empty key can never silently produce a weak cipher.
func New(masterKey []byte) (*Cipher, error) {
if len(masterKey) < MinMasterKeyLen {
return nil, fmt.Errorf("secret: master key must be at least %d bytes, got %d", MinMasterKeyLen, len(masterKey))
}
dataKey, err := hkdf.Key(sha256.New, masterKey, nil /* salt */, hkdfInfo, dataKeyLen)
if err != nil {
return nil, fmt.Errorf("secret: derive data key: %w", err)
}
block, err := aes.NewCipher(dataKey)
if err != nil {
return nil, fmt.Errorf("secret: new aes cipher: %w", err)
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, fmt.Errorf("secret: new gcm: %w", err)
}
return &Cipher{gcm: gcm}, nil
}
// Encrypt seals plaintext under the data key, binding the ciphertext to aad (the
// row-identity string the matching Decrypt must supply). An empty plaintext
// returns ("", nil): an empty secret is never wrapped in an envelope, so an
// absent credential stays an empty column value. The result is
// enc:v1:<base64url(nonce‖sealed)>.
func (c *Cipher) Encrypt(plaintext, aad string) (string, error) {
if plaintext == "" {
return "", nil
}
nonce := make([]byte, c.gcm.NonceSize())
if _, err := rand.Read(nonce); err != nil {
return "", fmt.Errorf("secret: read nonce: %w", err)
}
// Seal appends the ciphertext to nonce, so the returned slice is nonce‖sealed.
sealed := c.gcm.Seal(nonce, nonce, []byte(plaintext), []byte(aad))
return envelopePrefix + base64.RawURLEncoding.EncodeToString(sealed), nil
}
// Decrypt opens an enc:v1: envelope produced by Encrypt with the same aad. Any
// failure — unknown version, malformed base64, truncation, tampering, the wrong
// key, or an aad mismatch — returns an error. Decrypt never falls back to
// returning the input ciphertext or an empty string; passing through
// non-prefixed legacy plaintext is the caller's responsibility (the read-path
// contract), not Decrypt's.
func (c *Cipher) Decrypt(ciphertext, aad string) (string, error) {
// Parse "enc:<version>:<body>" and dispatch on the version so a future
// enc:v2: can be added without touching existing call sites.
parts := strings.SplitN(ciphertext, ":", 3)
if len(parts) != 3 || parts[0] != "enc" {
return "", fmt.Errorf("secret: not an enc envelope")
}
switch parts[1] {
case "v1":
return c.openV1(parts[2], aad)
default:
return "", fmt.Errorf("%w: %q", ErrUnknownVersion, parts[1])
}
}
// openV1 decodes and authenticates a v1 body (the part after "enc:v1:").
func (c *Cipher) openV1(encoded, aad string) (string, error) {
sealed, err := base64.RawURLEncoding.DecodeString(encoded)
if err != nil {
return "", fmt.Errorf("secret: decode ciphertext: %w", err)
}
nonceSize := c.gcm.NonceSize()
if len(sealed) < nonceSize {
return "", fmt.Errorf("secret: ciphertext too short")
}
nonce, body := sealed[:nonceSize], sealed[nonceSize:]
plaintext, err := c.gcm.Open(nil, nonce, body, []byte(aad))
if err != nil {
return "", fmt.Errorf("secret: authenticate ciphertext: %w", err)
}
return string(plaintext), nil
}
// IsEncrypted reports whether s carries the v1 envelope prefix. It is a cheap
// prefix check, not a validity check: a value that begins with enc:v1: but
// holds a corrupt body still reports true here and then fails Decrypt — it never
// silently degrades to being used as a plaintext credential.
func IsEncrypted(s string) bool {
return strings.HasPrefix(s, envelopePrefix)
}
// DecryptIfEncrypted is the canonical read-path primitive shared by every
// decrypt site (the settings decorator and the per-table repos). It implements
// the read-path contract:
//
// 1. an empty value returns "" unchanged;
// 2. a non-enveloped (legacy plaintext) value is returned unchanged — during
// the backfill window a not-yet-encrypted credential is no worse than today;
// 3. an enc:v1: value is decrypted, and any failure (wrong key, tamper,
// truncation) is returned, never swallowed and never returned as the
// ciphertext string.
//
// This reconciles best-effort backfill (a skipped/failed row stays readable as
// plaintext) with the hard guarantee that a real enc:v1: value never silently
// degrades into using ciphertext as a credential.
func (c *Cipher) DecryptIfEncrypted(value, aad string) (string, error) {
if value == "" || !IsEncrypted(value) {
return value, nil
}
return c.Decrypt(value, aad)
}
// SettingsAAD returns the additional-authenticated-data string binding a
// server_settings ciphertext to its key. Read and write sites must both use it
// so the AAD matches; centralizing it here keeps the convention typo-proof
// across the settings decorator, the config watcher, and other settings
// readers.
func SettingsAAD(key string) string {
return "server_settings:" + key
}
// RowAAD returns the additional-authenticated-data string binding a ciphertext
// to one logical row: table:column:<pk>. Binding to row identity stops a
// DB-write attacker from transplanting a credential blob into another row or
// column — decrypting with a different (table, column, pk) fails authentication.
func RowAAD(table, column, pk string) string {
return table + ":" + column + ":" + pk
}
// EncryptIfPlaintext encrypts s only when it is non-empty, non-encrypted
// plaintext, returning (ciphertext, true, nil). An already-encrypted or empty
// value is returned unchanged with changed=false. This is the idempotent
// primitive the startup backfill uses to sweep a column without ever
// double-encrypting an already-wrapped value.
func (c *Cipher) EncryptIfPlaintext(s, aad string) (string, bool, error) {
if s == "" || IsEncrypted(s) {
return s, false, nil
}
ct, err := c.Encrypt(s, aad)
if err != nil {
return s, false, err
}
return ct, true, nil
}
+255
View File
@@ -0,0 +1,255 @@
package secret
import (
"crypto/rand"
"encoding/base64"
"errors"
"strings"
"testing"
)
// testKey returns a deterministic-length master key suitable for New.
func testKey(t *testing.T) []byte {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("read key: %v", err)
}
return key
}
func newTestCipher(t *testing.T) *Cipher {
t.Helper()
c, err := New(testKey(t))
if err != nil {
t.Fatalf("New: %v", err)
}
return c
}
func TestEncryptDecryptRoundTrip(t *testing.T) {
c := newTestCipher(t)
const (
plaintext = "sk-radarr-0123456789abcdef"
aad = "request_integrations:api_key_ref:abc123"
)
ct, err := c.Encrypt(plaintext, aad)
if err != nil {
t.Fatalf("Encrypt: %v", err)
}
if !strings.HasPrefix(ct, "enc:v1:") {
t.Fatalf("ciphertext missing enc:v1: prefix: %q", ct)
}
if !IsEncrypted(ct) {
t.Fatalf("IsEncrypted(%q) = false, want true", ct)
}
got, err := c.Decrypt(ct, aad)
if err != nil {
t.Fatalf("Decrypt: %v", err)
}
if got != plaintext {
t.Fatalf("round-trip = %q, want %q", got, plaintext)
}
}
func TestEncryptEmptyReturnsEmpty(t *testing.T) {
c := newTestCipher(t)
ct, err := c.Encrypt("", "any:aad")
if err != nil {
t.Fatalf("Encrypt empty: %v", err)
}
if ct != "" {
t.Fatalf("Encrypt(\"\") = %q, want empty (no envelope for an empty secret)", ct)
}
}
func TestEncryptNonDeterministic(t *testing.T) {
c := newTestCipher(t)
const plaintext, aad = "same-secret", "t:c:1"
a, err := c.Encrypt(plaintext, aad)
if err != nil {
t.Fatalf("Encrypt a: %v", err)
}
b, err := c.Encrypt(plaintext, aad)
if err != nil {
t.Fatalf("Encrypt b: %v", err)
}
if a == b {
t.Fatalf("two encryptions of the same plaintext+aad were identical; random nonce not applied")
}
// Both must still decrypt back to the same plaintext.
for _, ct := range []string{a, b} {
got, err := c.Decrypt(ct, aad)
if err != nil || got != plaintext {
t.Fatalf("Decrypt(%q) = (%q, %v), want (%q, nil)", ct, got, err, plaintext)
}
}
}
func TestDecryptTamperRejected(t *testing.T) {
c := newTestCipher(t)
const aad = "t:c:1"
ct, err := c.Encrypt("secret", aad)
if err != nil {
t.Fatalf("Encrypt: %v", err)
}
// Decode the body, flip a byte, re-encode, reattach the envelope.
body := strings.TrimPrefix(ct, "enc:v1:")
raw, err := base64.RawURLEncoding.DecodeString(body)
if err != nil {
t.Fatalf("decode: %v", err)
}
raw[len(raw)-1] ^= 0xFF
tampered := "enc:v1:" + base64.RawURLEncoding.EncodeToString(raw)
if _, err := c.Decrypt(tampered, aad); err == nil {
t.Fatalf("Decrypt of tampered ciphertext succeeded, want error")
}
}
func TestDecryptAADMismatchRejected(t *testing.T) {
c := newTestCipher(t)
ct, err := c.Encrypt("secret", "t:c:1")
if err != nil {
t.Fatalf("Encrypt: %v", err)
}
if _, err := c.Decrypt(ct, "t:c:2"); err == nil {
t.Fatalf("Decrypt with mismatched AAD succeeded, want error (row binding broken)")
}
}
func TestDecryptWrongKeyRejected(t *testing.T) {
a := newTestCipher(t)
b := newTestCipher(t)
const aad = "t:c:1"
ct, err := a.Encrypt("secret", aad)
if err != nil {
t.Fatalf("Encrypt: %v", err)
}
if _, err := b.Decrypt(ct, aad); err == nil {
t.Fatalf("Decrypt with the wrong key succeeded, want error")
}
}
func TestNewRejectsShortKey(t *testing.T) {
if _, err := New(make([]byte, MinMasterKeyLen-1)); err == nil {
t.Fatalf("New with a %d-byte key succeeded, want error", MinMasterKeyLen-1)
}
if _, err := New(nil); err == nil {
t.Fatalf("New(nil) succeeded, want error")
}
if _, err := New(make([]byte, MinMasterKeyLen)); err != nil {
t.Fatalf("New with a %d-byte key failed: %v", MinMasterKeyLen, err)
}
}
func TestIsEncrypted(t *testing.T) {
cases := map[string]bool{
"enc:v1:x": true,
"sa_abc": false,
"": false,
"ENC:V1:x": false,
"enc:v1": false,
"plain text": false,
}
for in, want := range cases {
if got := IsEncrypted(in); got != want {
t.Errorf("IsEncrypted(%q) = %v, want %v", in, got, want)
}
}
}
func TestDecryptUnknownVersion(t *testing.T) {
c := newTestCipher(t)
_, err := c.Decrypt("enc:v99:"+base64.RawURLEncoding.EncodeToString([]byte("whatever")), "t:c:1")
if !errors.Is(err, ErrUnknownVersion) {
t.Fatalf("Decrypt of enc:v99: returned %v, want ErrUnknownVersion", err)
}
}
func TestDecryptNonEnvelopeErrors(t *testing.T) {
c := newTestCipher(t)
// A non-prefixed legacy value is the caller's pass-through responsibility, but
// Decrypt itself must reject it rather than echo it back.
if _, err := c.Decrypt("plaintext-not-an-envelope", "t:c:1"); err == nil {
t.Fatalf("Decrypt of a non-envelope value succeeded, want error")
}
}
func TestDecryptShortCiphertext(t *testing.T) {
c := newTestCipher(t)
// Valid base64url but fewer bytes than the GCM nonce.
short := "enc:v1:" + base64.RawURLEncoding.EncodeToString([]byte("ab"))
if _, err := c.Decrypt(short, "t:c:1"); err == nil {
t.Fatalf("Decrypt of a too-short ciphertext succeeded, want error")
}
}
func TestDecryptIfEncrypted(t *testing.T) {
c := newTestCipher(t)
const aad = "server_settings:tmdb.api_key"
// Empty passes through.
if got, err := c.DecryptIfEncrypted("", aad); err != nil || got != "" {
t.Fatalf("DecryptIfEncrypted(\"\") = (%q, %v), want (\"\", nil)", got, err)
}
// Legacy plaintext passes through unchanged.
if got, err := c.DecryptIfEncrypted("legacy-plaintext-key", aad); err != nil || got != "legacy-plaintext-key" {
t.Fatalf("DecryptIfEncrypted(plaintext) = (%q, %v), want pass-through", got, err)
}
// A real enc:v1: value is decrypted.
ct, err := c.Encrypt("real-secret", aad)
if err != nil {
t.Fatalf("Encrypt: %v", err)
}
if got, err := c.DecryptIfEncrypted(ct, aad); err != nil || got != "real-secret" {
t.Fatalf("DecryptIfEncrypted(ct) = (%q, %v), want (\"real-secret\", nil)", got, err)
}
// A corrupt enc:v1: value errors — never falls back to the ciphertext.
if _, err := c.DecryptIfEncrypted(ct, "server_settings:wrong.key"); err == nil {
t.Fatalf("DecryptIfEncrypted with wrong AAD succeeded, want error")
}
}
func TestAADHelpers(t *testing.T) {
if got := SettingsAAD("auth.jwt_secret"); got != "server_settings:auth.jwt_secret" {
t.Errorf("SettingsAAD = %q", got)
}
if got := RowAAD("request_integrations", "api_key_ref", "abc"); got != "request_integrations:api_key_ref:abc" {
t.Errorf("RowAAD = %q", got)
}
}
func TestEncryptIfPlaintextIdempotent(t *testing.T) {
c := newTestCipher(t)
const aad = "t:c:1"
// First call encrypts.
ct, changed, err := c.EncryptIfPlaintext("secret", aad)
if err != nil {
t.Fatalf("EncryptIfPlaintext: %v", err)
}
if !changed {
t.Fatalf("first EncryptIfPlaintext reported changed=false, want true")
}
if !IsEncrypted(ct) {
t.Fatalf("EncryptIfPlaintext did not produce an envelope: %q", ct)
}
// Second call is a no-op on the already-encrypted value.
ct2, changed2, err := c.EncryptIfPlaintext(ct, aad)
if err != nil {
t.Fatalf("EncryptIfPlaintext (2nd): %v", err)
}
if changed2 {
t.Fatalf("second EncryptIfPlaintext reported changed=true, want false (double-encrypt)")
}
if ct2 != ct {
t.Fatalf("second EncryptIfPlaintext mutated the value: %q -> %q", ct, ct2)
}
// Empty is a no-op too.
got, changed3, err := c.EncryptIfPlaintext("", aad)
if err != nil || changed3 || got != "" {
t.Fatalf("EncryptIfPlaintext(\"\") = (%q, %v, %v), want (\"\", false, nil)", got, changed3, err)
}
}
+49 -9
View File
@@ -8,16 +8,44 @@ import (
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
// PgRepository implements Repository using PostgreSQL.
type PgRepository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
// NewPgRepository creates a new PostgreSQL repository.
func NewPgRepository(pool *pgxpool.Pool) *PgRepository {
return &PgRepository{pool: pool}
func NewPgRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *PgRepository {
return &PgRepository{pool: pool, cipher: cipher}
}
// providerSecretAAD binds a subtitle_provider_config secret column to its row
// (keyed by provider_name).
func (r *PgRepository) providerSecretAAD(column, providerName string) string {
return secret.RowAAD("subtitle_provider_config", column, providerName)
}
// decryptProviderConfig applies the read-path contract to the api_key and
// password columns (username is not a secret) and (re)derives the Has* flags
// from the decrypted values.
func (r *PgRepository) decryptProviderConfig(cfg *ProviderConfig) error {
apiKey, err := r.cipher.DecryptIfEncrypted(cfg.APIKey, r.providerSecretAAD("api_key", cfg.ProviderName))
if err != nil {
return fmt.Errorf("decrypt subtitle api key for %s: %w", cfg.ProviderName, err)
}
cfg.APIKey = apiKey
password, err := r.cipher.DecryptIfEncrypted(cfg.Password, r.providerSecretAAD("password", cfg.ProviderName))
if err != nil {
return fmt.Errorf("decrypt subtitle password for %s: %w", cfg.ProviderName, err)
}
cfg.Password = password
cfg.HasAPIKey = cfg.APIKey != ""
cfg.HasCredentials = cfg.Username != "" && cfg.Password != ""
return nil
}
func (r *PgRepository) InsertDownloadedSubtitle(ctx context.Context, sub *DownloadedSubtitle) error {
@@ -146,8 +174,9 @@ func (r *PgRepository) ListProviderConfigs(ctx context.Context) ([]ProviderConfi
&cfg.Username, &cfg.Password, &cfg.UpdatedAt); err != nil {
return nil, fmt.Errorf("scan provider config: %w", err)
}
cfg.HasAPIKey = cfg.APIKey != ""
cfg.HasCredentials = cfg.Username != "" && cfg.Password != ""
if err := r.decryptProviderConfig(&cfg); err != nil {
return nil, err
}
configs = append(configs, cfg)
}
return configs, rows.Err()
@@ -166,14 +195,25 @@ func (r *PgRepository) GetProviderConfig(ctx context.Context, providerName strin
if err != nil {
return nil, fmt.Errorf("get provider config: %w", err)
}
cfg.HasAPIKey = cfg.APIKey != ""
cfg.HasCredentials = cfg.Username != "" && cfg.Password != ""
if err := r.decryptProviderConfig(&cfg); err != nil {
return nil, err
}
return &cfg, nil
}
func (r *PgRepository) UpsertProviderConfig(ctx context.Context, cfg *ProviderConfig) error {
// Encrypt the secret columns; an empty result preserves the keep-existing
// CASE (toggling enabled with blank creds leaves stored creds intact).
apiKey, err := r.cipher.Encrypt(cfg.APIKey, r.providerSecretAAD("api_key", cfg.ProviderName))
if err != nil {
return fmt.Errorf("encrypt subtitle api key: %w", err)
}
password, err := r.cipher.Encrypt(cfg.Password, r.providerSecretAAD("password", cfg.ProviderName))
if err != nil {
return fmt.Errorf("encrypt subtitle password: %w", err)
}
// Only overwrite credentials when non-empty, so toggling enabled doesn't wipe them.
_, err := r.pool.Exec(ctx,
_, err = r.pool.Exec(ctx,
`INSERT INTO subtitle_provider_config (provider_name, enabled, api_key, username, password, updated_at)
VALUES ($1, $2, $3, $4, $5, NOW())
ON CONFLICT (provider_name) DO UPDATE SET
@@ -182,7 +222,7 @@ func (r *PgRepository) UpsertProviderConfig(ctx context.Context, cfg *ProviderCo
username = CASE WHEN EXCLUDED.username = '' THEN subtitle_provider_config.username ELSE EXCLUDED.username END,
password = CASE WHEN EXCLUDED.password = '' THEN subtitle_provider_config.password ELSE EXCLUDED.password END,
updated_at = NOW()`,
cfg.ProviderName, cfg.Enabled, cfg.APIKey, cfg.Username, cfg.Password)
cfg.ProviderName, cfg.Enabled, apiKey, cfg.Username, password)
if err != nil {
return fmt.Errorf("upsert provider config: %w", err)
}
@@ -0,0 +1,81 @@
package tasks
import (
"context"
"encoding/json"
"fmt"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/taskmanager"
)
const orphanedMediaCleanupBatchSize = 1000
type OrphanedMediaItemCleaner interface {
Cleanup(ctx context.Context, batchSize int) (catalog.OrphanedProvisionalCleanupStats, error)
}
type CleanupOrphanedMediaItemsTask struct {
cleaner OrphanedMediaItemCleaner
}
type cleanupOrphanedMediaItemsResult struct {
Candidates int `json:"candidates"`
Deleted int `json:"deleted"`
}
func NewCleanupOrphanedMediaItemsTask(cleaner OrphanedMediaItemCleaner) *CleanupOrphanedMediaItemsTask {
return &CleanupOrphanedMediaItemsTask{cleaner: cleaner}
}
func (t *CleanupOrphanedMediaItemsTask) Key() string { return "cleanup_orphaned_media_items" }
func (t *CleanupOrphanedMediaItemsTask) Name() string {
return "Clean Orphaned Media Items"
}
func (t *CleanupOrphanedMediaItemsTask) Description() string {
return "Deletes detached provisional media items that have no files, library memberships, or durable user/sync references."
}
func (t *CleanupOrphanedMediaItemsTask) Category() taskmanager.TaskCategory {
return taskmanager.TaskCategoryLibrary
}
func (t *CleanupOrphanedMediaItemsTask) IsHidden() bool { return false }
func (t *CleanupOrphanedMediaItemsTask) DefaultTriggers() []taskmanager.TriggerConfig {
return nil
}
func (t *CleanupOrphanedMediaItemsTask) Execute(ctx context.Context, progress taskmanager.ProgressReporter) error {
if t == nil || t.cleaner == nil {
progress.Report(100, "Orphaned media cleanup is not configured")
return nil
}
progress.Report(0, "Checking for orphaned provisional media items")
stats, err := t.cleaner.Cleanup(ctx, orphanedMediaCleanupBatchSize)
setCleanupOrphanedMediaItemsResult(progress, cleanupOrphanedMediaItemsResult{
Candidates: stats.Candidates,
Deleted: stats.Deleted,
})
if err != nil {
progress.Report(100, fmt.Sprintf("Orphaned media cleanup failed after deleting %d of %d candidates", stats.Deleted, stats.Candidates))
return err
}
if stats.Candidates == 0 {
progress.Report(100, "No orphaned provisional media items found")
return nil
}
progress.Report(100, fmt.Sprintf("Deleted %d orphaned provisional media items (%d candidates)", stats.Deleted, stats.Candidates))
return nil
}
func setCleanupOrphanedMediaItemsResult(progress taskmanager.ProgressReporter, result cleanupOrphanedMediaItemsResult) {
if progress == nil {
return
}
data, err := json.Marshal(result)
if err != nil {
return
}
progress.SetResultData(data)
}
@@ -0,0 +1,112 @@
package tasks
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/taskmanager"
)
type fakeOrphanedMediaCleaner struct {
stats catalog.OrphanedProvisionalCleanupStats
err error
calls int
batchSize int
}
func (f *fakeOrphanedMediaCleaner) Cleanup(_ context.Context, batchSize int) (catalog.OrphanedProvisionalCleanupStats, error) {
f.calls++
f.batchSize = batchSize
return f.stats, f.err
}
type orphanedMediaCleanupProgress struct {
reports []string
result json.RawMessage
}
func (p *orphanedMediaCleanupProgress) Report(_ float64, message string) {
p.reports = append(p.reports, message)
}
func (p *orphanedMediaCleanupProgress) SetResultData(data json.RawMessage) {
p.result = append(p.result[:0], data...)
}
func TestCleanupOrphanedMediaItemsTask(t *testing.T) {
cleaner := &fakeOrphanedMediaCleaner{
stats: catalog.OrphanedProvisionalCleanupStats{Candidates: 5, Deleted: 5},
}
task := NewCleanupOrphanedMediaItemsTask(cleaner)
if task.Key() != "cleanup_orphaned_media_items" {
t.Fatalf("Key() = %q, want cleanup_orphaned_media_items", task.Key())
}
if task.Category() != taskmanager.TaskCategoryLibrary {
t.Fatalf("Category() = %q, want %q", task.Category(), taskmanager.TaskCategoryLibrary)
}
if task.IsHidden() {
t.Fatal("IsHidden() = true, want false")
}
if triggers := task.DefaultTriggers(); len(triggers) != 0 {
t.Fatalf("DefaultTriggers() = %#v, want none", triggers)
}
progress := &orphanedMediaCleanupProgress{}
if err := task.Execute(context.Background(), progress); err != nil {
t.Fatalf("Execute returned error: %v", err)
}
if cleaner.calls != 1 {
t.Fatalf("cleaner calls = %d, want 1", cleaner.calls)
}
if cleaner.batchSize != orphanedMediaCleanupBatchSize {
t.Fatalf("batchSize = %d, want %d", cleaner.batchSize, orphanedMediaCleanupBatchSize)
}
last := progress.reports[len(progress.reports)-1]
if !strings.Contains(last, "Deleted 5 orphaned provisional media items") {
t.Fatalf("last progress report = %q", last)
}
var result cleanupOrphanedMediaItemsResult
if err := json.Unmarshal(progress.result, &result); err != nil {
t.Fatalf("unmarshal result data: %v", err)
}
if result.Candidates != 5 || result.Deleted != 5 {
t.Fatalf("result = %#v, want 5 candidates and 5 deleted", result)
}
}
func TestCleanupOrphanedMediaItemsTaskNoCandidates(t *testing.T) {
task := NewCleanupOrphanedMediaItemsTask(&fakeOrphanedMediaCleaner{})
progress := &orphanedMediaCleanupProgress{}
if err := task.Execute(context.Background(), progress); err != nil {
t.Fatalf("Execute returned error: %v", err)
}
last := progress.reports[len(progress.reports)-1]
if last != "No orphaned provisional media items found" {
t.Fatalf("last progress report = %q", last)
}
}
func TestCleanupOrphanedMediaItemsTaskReturnsCleanupError(t *testing.T) {
wantErr := errors.New("cleanup failed")
cleaner := &fakeOrphanedMediaCleaner{
stats: catalog.OrphanedProvisionalCleanupStats{Candidates: 5, Deleted: 2},
err: wantErr,
}
task := NewCleanupOrphanedMediaItemsTask(cleaner)
progress := &orphanedMediaCleanupProgress{}
err := task.Execute(context.Background(), progress)
if !errors.Is(err, wantErr) {
t.Fatalf("Execute error = %v, want %v", err, wantErr)
}
last := progress.reports[len(progress.reports)-1]
if !strings.Contains(last, "failed after deleting 2 of 5 candidates") {
t.Fatalf("last progress report = %q", last)
}
}
+5 -1
View File
@@ -112,7 +112,11 @@ func (s *Server) handleHealth(w http.ResponseWriter, _ *http.Request) {
}
func (s *Server) handleHWCapabilities(w http.ResponseWriter, _ *http.Request) {
info := playback.DetectHWAccel()
ffmpegPath := ""
if cfg := s.watcher.Config(); cfg != nil {
ffmpegPath = cfg.Playback.FFmpegPath
}
info := playback.DetectHWAccelWithFFmpeg(ffmpegPath)
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(info)
}
+51 -14
View File
@@ -5,10 +5,13 @@ import (
"encoding/json"
"errors"
"fmt"
"strconv"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
type Repository interface {
@@ -45,11 +48,22 @@ type Repository interface {
}
type PostgresRepository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewPostgresRepository(pool *pgxpool.Pool) *PostgresRepository {
return &PostgresRepository{pool: pool}
func NewPostgresRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *PostgresRepository {
return &PostgresRepository{pool: pool, cipher: cipher}
}
// TokenAAD binds an access/refresh token ciphertext to its connection. It uses
// the stable UNIQUE business key (provider, user_id, profile_id) rather than the
// surrogate id, because UpsertConnection lets Postgres assign/keep the id (ON
// CONFLICT), so the id is not known before the write — the tuple is, and it
// identifies the row just as uniquely. Exported so the (raw-SQL) Trakt
// collection-token resolver binds tokens identically.
func TokenAAD(column, provider string, userID int, profileID string) string {
return secret.RowAAD("watch_provider_connections", column, provider+":"+strconv.Itoa(userID)+":"+profileID)
}
func (r *PostgresRepository) GetServerSetting(ctx context.Context, key string) (string, error) {
@@ -61,7 +75,14 @@ func (r *PostgresRepository) GetServerSetting(ctx context.Context, key string) (
if err != nil {
return "", fmt.Errorf("server_settings get %q: %w", key, err)
}
return value, nil
// This repo reads watchsync.<provider>.client_id/client_secret (sensitive
// settings) directly, bypassing the settings decorator, so apply the same
// read-path decryption here.
out, err := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if err != nil {
return "", fmt.Errorf("decrypt server_settings %q: %w", key, err)
}
return out, nil
}
func (r *PostgresRepository) UpsertAuthSession(
@@ -126,6 +147,14 @@ func (r *PostgresRepository) GetAuthSession(ctx context.Context, id string) (Dev
}
func (r *PostgresRepository) UpsertConnection(ctx context.Context, conn Connection) (Connection, error) {
accessToken, err := r.cipher.Encrypt(conn.AccessToken, TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return Connection{}, fmt.Errorf("encrypt watch access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(conn.RefreshToken, TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return Connection{}, fmt.Errorf("encrypt watch refresh token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO watch_provider_connections (
id, provider, user_id, profile_id, provider_account_id, provider_username,
@@ -176,8 +205,8 @@ func (r *PostgresRepository) UpsertConnection(ctx context.Context, conn Connecti
conn.ProfileID,
conn.ProviderAccountID,
conn.ProviderUsername,
conn.AccessToken,
conn.RefreshToken,
accessToken,
refreshToken,
conn.TokenExpiresAt,
conn.ImportWatchedEnabled,
conn.ImportProgressEnabled,
@@ -195,7 +224,7 @@ func (r *PostgresRepository) UpsertConnection(ctx context.Context, conn Connecti
conn.LastError,
encodeSyncCursors(conn.SyncCursors),
)
saved, err := scanConnection(row)
saved, err := r.scanConnection(row)
if err != nil {
return Connection{}, fmt.Errorf("upsert watch provider connection: %w", err)
}
@@ -219,7 +248,7 @@ func (r *PostgresRepository) GetConnection(
FROM watch_provider_connections
WHERE provider = $1 AND user_id = $2 AND profile_id = $3
`, provider, userID, profileID)
conn, err := scanConnection(row)
conn, err := r.scanConnection(row)
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, false, nil
}
@@ -241,7 +270,7 @@ func (r *PostgresRepository) GetConnectionByID(ctx context.Context, id string) (
FROM watch_provider_connections
WHERE id = $1::uuid
`, id)
conn, err := scanConnection(row)
conn, err := r.scanConnection(row)
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, false, nil
}
@@ -300,7 +329,7 @@ func (r *PostgresRepository) ListConnectionsDueForSync(
var conns []Connection
for rows.Next() {
conn, scanErr := scanConnection(rows)
conn, scanErr := r.scanConnection(rows)
if scanErr != nil {
return nil, fmt.Errorf("scan due watch provider connection: %w", scanErr)
}
@@ -509,7 +538,7 @@ func (r *PostgresRepository) ListLocalWatchEventConnections(
var conns []Connection
for rows.Next() {
conn, scanErr := scanConnection(rows)
conn, scanErr := r.scanConnection(rows)
if scanErr != nil {
return nil, fmt.Errorf("scan local watch event connection: %w", scanErr)
}
@@ -555,7 +584,7 @@ func (r *PostgresRepository) ListFavoriteEventConnections(
var conns []Connection
for rows.Next() {
conn, scanErr := scanConnection(rows)
conn, scanErr := r.scanConnection(rows)
if scanErr != nil {
return nil, fmt.Errorf("scan favorite event connection: %w", scanErr)
}
@@ -862,7 +891,7 @@ func (r *PostgresRepository) ListScrobbleConnections(ctx context.Context, userID
defer rows.Close()
var conns []Connection
for rows.Next() {
conn, err := scanConnection(rows)
conn, err := r.scanConnection(rows)
if err != nil {
return nil, fmt.Errorf("scan scrobble connection: %w", err)
}
@@ -1033,7 +1062,7 @@ func scanSyncRun(row pgx.Row) (SyncRun, error) {
return run, nil
}
func scanConnection(row pgx.Row) (Connection, error) {
func (r *PostgresRepository) scanConnection(row pgx.Row) (Connection, error) {
var conn Connection
var rawSyncCursors []byte
err := row.Scan(
@@ -1067,6 +1096,14 @@ func scanConnection(row pgx.Row) (Connection, error) {
if err != nil {
return Connection{}, err
}
// Decrypt the tokens (read-path contract), bound to the connection's stable
// business key — matching TokenAAD on the write path.
if conn.AccessToken, err = r.cipher.DecryptIfEncrypted(conn.AccessToken, TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return Connection{}, fmt.Errorf("decrypt watch access token: %w", err)
}
if conn.RefreshToken, err = r.cipher.DecryptIfEncrypted(conn.RefreshToken, TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return Connection{}, fmt.Errorf("decrypt watch refresh token: %w", err)
}
conn.SyncCursors = decodeSyncCursors(rawSyncCursors)
return conn, nil
}
+35 -9
View File
@@ -7,16 +7,28 @@ import (
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
var ErrConnectionNotFound = errors.New("webhook sync connection not found")
type Repository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// webhookTokenAAD binds a webhook_sync_connections access_token ciphertext to
// its row. Defined at package scope so the secret package reference is isolated
// from GetConnectionBySecret, whose parameter shadows the import name.
// (webhook_secret is NOT encrypted — it is matched by exact value in
// GetConnectionBySecret and is carved out for a separate hashing follow-up.)
func webhookTokenAAD(id string) string {
return secret.RowAAD("webhook_sync_connections", "access_token", id)
}
func (r *Repository) ProfileExistsForUser(ctx context.Context, userID int, profileID string) (bool, error) {
@@ -31,6 +43,10 @@ func (r *Repository) ProfileExistsForUser(ctx context.Context, userID int, profi
}
func (r *Repository) CreateConnection(ctx context.Context, conn Connection) (*Connection, error) {
accessToken, err := r.cipher.Encrypt(conn.AccessToken, webhookTokenAAD(conn.ID))
if err != nil {
return nil, fmt.Errorf("encrypt webhook access token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO webhook_sync_connections (
id, user_id, provider, server_id, server_name, base_url, access_token, default_profile_id, webhook_secret
@@ -39,9 +55,9 @@ func (r *Repository) CreateConnection(ctx context.Context, conn Connection) (*Co
webhook_secret, account_discovery_available,
last_webhook_received_at, last_webhook_error_at, COALESCE(last_webhook_error_message, ''),
created_at, updated_at`,
conn.ID, conn.UserID, conn.Provider, conn.ServerID, conn.ServerName, conn.BaseURL, conn.AccessToken, conn.DefaultProfileID, conn.WebhookSecret,
conn.ID, conn.UserID, conn.Provider, conn.ServerID, conn.ServerName, conn.BaseURL, accessToken, conn.DefaultProfileID, conn.WebhookSecret,
)
return scanConnection(row)
return r.scanConnection(row)
}
func (r *Repository) ListConnections(ctx context.Context, userID int) ([]Connection, error) {
@@ -72,6 +88,9 @@ func (r *Repository) ListConnections(ctx context.Context, userID int) ([]Connect
); err != nil {
return nil, fmt.Errorf("scanning webhook sync connection: %w", err)
}
if c.AccessToken, err = r.cipher.DecryptIfEncrypted(c.AccessToken, webhookTokenAAD(c.ID)); err != nil {
return nil, fmt.Errorf("decrypt webhook access token: %w", err)
}
out = append(out, c)
}
if err := rows.Err(); err != nil {
@@ -88,7 +107,7 @@ func (r *Repository) GetConnection(ctx context.Context, userID int, id string) (
created_at, updated_at
FROM webhook_sync_connections
WHERE id = $1 AND user_id = $2`, id, userID)
return scanConnection(row)
return r.scanConnection(row)
}
func (r *Repository) GetConnectionBySecret(ctx context.Context, secret string) (*Connection, error) {
@@ -99,10 +118,10 @@ func (r *Repository) GetConnectionBySecret(ctx context.Context, secret string) (
created_at, updated_at
FROM webhook_sync_connections
WHERE webhook_secret = $1`, secret)
return scanConnection(row)
return r.scanConnection(row)
}
func scanConnection(row pgx.Row) (*Connection, error) {
func (r *Repository) scanConnection(row pgx.Row) (*Connection, error) {
var c Connection
if err := row.Scan(
&c.ID, &c.UserID, &c.Provider, &c.ServerID, &c.ServerName, &c.BaseURL, &c.AccessToken, &c.DefaultProfileID,
@@ -115,6 +134,13 @@ func scanConnection(row pgx.Row) (*Connection, error) {
}
return nil, fmt.Errorf("scanning webhook sync connection: %w", err)
}
// Decrypt the access token (read-path). webhook_secret is intentionally left
// as-is (equality-looked-up; carved out for the hashing follow-up).
token, err := r.cipher.DecryptIfEncrypted(c.AccessToken, webhookTokenAAD(c.ID))
if err != nil {
return nil, fmt.Errorf("decrypt webhook access token: %w", err)
}
c.AccessToken = token
return &c, nil
}
@@ -131,7 +157,7 @@ func (r *Repository) UpdateConnection(ctx context.Context, userID int, id string
created_at, updated_at`,
id, userID, input.ServerName, input.DefaultProfileID,
)
return scanConnection(row)
return r.scanConnection(row)
}
func (r *Repository) DeleteConnection(ctx context.Context, userID int, id string) error {
@@ -0,0 +1,41 @@
package migrations
import (
"strings"
"testing"
)
func TestCleanupOrphanedProvisionalItemsPreservesDurableReferences(t *testing.T) {
migrationBytes, err := FS.ReadFile("sql/20260607220000_cleanup_orphaned_provisional_items.sql")
if err != nil {
t.Fatalf("read migration: %v", err)
}
migration := normalizeSQL(string(migrationBytes))
for _, want := range []string{
"public.abs_bookmarks ab WHERE ab.library_item_id = mi.content_id",
"public.abs_playback_sessions aps WHERE aps.content_id = mi.content_id",
"public.abs_rss_feeds arf WHERE arf.library_item_id = mi.content_id",
"public.episodes e WHERE e.series_id = mi.content_id",
"public.podcast_feeds pf WHERE pf.media_item_id = mi.content_id",
"public.seasons s WHERE s.series_id = mi.content_id",
"public.user_audio_preferences uap WHERE uap.series_id = mi.content_id",
"public.user_home_item_dismissals uhid WHERE uhid.media_item_id = mi.content_id",
"public.user_home_item_dismissals uhid_series WHERE uhid_series.series_id = mi.content_id",
"public.user_personal_collection_items upci WHERE upci.media_item_id = mi.content_id",
"public.user_series_playback_preferences uspp WHERE uspp.series_id = mi.content_id",
"public.user_subtitle_preferences usp WHERE usp.series_id = mi.content_id",
} {
if !strings.Contains(migration, normalizeSQL(want)) {
t.Fatalf("cleanup migration missing durable reference guard %q", want)
}
}
for _, droppedTable := range []string{
"abs_collection_items",
"abs_playlist_items",
"abs_playlists",
} {
if strings.Contains(migration, droppedTable) {
t.Fatalf("cleanup migration references dropped table %q", droppedTable)
}
}
}
@@ -0,0 +1,142 @@
-- +goose Up
-- +goose StatementBegin
-- Clean up provisional catalog rows left behind by library-delete/matcher races.
-- Stale external IDs and other item-owned metadata cascade from media_items.
-- Durable user/sync references do not all have media_items foreign keys, so
-- keep any provisional row that is still referenced outside derived metadata.
DELETE FROM public.media_items mi
WHERE mi.status IN ('pending', 'unmatched', 'ambiguous')
AND NOT EXISTS (
SELECT 1 FROM public.media_item_libraries mil
WHERE mil.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.media_files mf
WHERE mf.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.episodes e
WHERE e.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.seasons s
WHERE s.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.library_collection_items lci
WHERE lci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_bookmarks ab
WHERE ab.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_playback_sessions aps
WHERE aps.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_rss_feeds arf
WHERE arf.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.downloads d
WHERE d.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.playback_history_admin pha
WHERE pha.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_bindings psib
WHERE psib.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_state psis
WHERE psis.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.podcast_feeds pf
WHERE pf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_favorites uf
WHERE uf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_downloads ud
WHERE ud.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_history_hidden_items uhhi
WHERE uhhi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid
WHERE uhid.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid_series
WHERE uhid_series.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_audio_preferences uap
WHERE uap.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_personal_collection_items upci
WHERE upci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_ratings ur
WHERE ur.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_series_playback_preferences uspp
WHERE uspp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_subtitle_preferences usp
WHERE usp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_history uwh
WHERE uwh.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_progress uwp
WHERE uwp.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watchlist uwl
WHERE uwl.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_favorite_items wpfi
WHERE wpfi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_history_exports wphe
WHERE wphe.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_scrobble_sessions wpss
WHERE wpss.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_rooms wtr
WHERE wtr.selected_content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_suggestions wts
WHERE wts.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.webhook_sync_item_state wsis
WHERE wsis.media_item_id = mi.content_id
);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
-- Data cleanup only; deleted orphaned provisional rows cannot be reconstructed.
-- +goose StatementEnd
+2
View File
@@ -2049,7 +2049,9 @@ export interface AdminStats {
total_files: number;
total_users: number;
total_movies: number;
total_movie_files?: number;
total_shows: number;
total_show_files?: number;
active_streams: number;
total_storage_bytes: number;
watch_provider_activity: WatchProviderActivity;
+1 -1
View File
@@ -60,7 +60,7 @@ export const OVERLAY_PRESETS: Record<PresetId, OverlayPreset> = {
label: "Pill",
description: "Larger pill with more padding. Works well with icons.",
badgeClass:
"rounded-full border border-white/15 px-2.5 py-1 text-[10px] font-semibold tracking-wide uppercase leading-none backdrop-blur-sm",
"rounded-full border border-white/15 px-2.5 py-1 text-[10px] font-semibold tracking-wide uppercase leading-none",
badgeStyle: (accent) => ({
background: accent
? `color-mix(in srgb, ${accent} 20%, rgba(20,20,30,0.7))`
+133 -42
View File
@@ -1,14 +1,17 @@
import type { ReactNode } from "react";
import { useCallback, useEffect, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { Link, useNavigate } from "react-router";
import { AdminSessionActions } from "@/components/AdminSessionActions";
import { useEventChannel } from "@/components/realtimeEventsContext";
import { fetchAdminStats, useAdminStats, useAdminSessions } from "@/hooks/queries/admin/stats";
import { useAdminUsers } from "@/hooks/queries/admin/users";
import {
useAdminLibraries,
useCancelLibraryScans,
useScanAllLibraries,
useScanLibrary,
} from "@/hooks/queries/admin/libraries";
import { useActiveScans } from "@/hooks/queries/admin/scans";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
@@ -31,6 +34,7 @@ import {
Pause,
Library,
ScanLine,
Square,
} from "lucide-react";
import { Skeleton } from "@/components/ui/skeleton";
import type {
@@ -38,16 +42,35 @@ import type {
AdminStats,
Library as LibraryType,
AdminUser,
ScanRun,
WatchProviderActivity,
} from "@/api/types";
import { useQueryClient } from "@tanstack/react-query";
import { adminKeys } from "@/hooks/queries/keys";
import { usePageActivity } from "@/hooks/usePageActivity";
import { cn } from "@/lib/utils";
import { compareActiveScans, formatActiveScanMode, formatActiveScanProgress } from "@/lib/scanRuns";
const REFRESH_SPINNER_MIN_VISIBLE_MS = 1_000;
const DASHBOARD_AUTO_REFRESH_MS = 60_000;
const RELATIVE_UPDATED_LABEL_TICK_MS = 30_000;
function formatFileCount(count: number | null | undefined) {
if (count == null) {
return "—";
}
return count === 1 ? "1 file" : `${count.toLocaleString()} files`;
}
function formatDashboardLibraryScanProgress(scan: ScanRun, activeScanCount: number) {
const status = scan.status === "running" ? "Scanning" : "Queued";
const progress = formatActiveScanProgress(scan);
const detail =
progress || (scan.status === "running" ? formatActiveScanMode(scan) : "Waiting for capacity");
const extraScans = activeScanCount > 1 ? ` + ${activeScanCount - 1} more` : "";
return `${status}: ${detail}${extraScans}`;
}
export default function AdminDashboard() {
const queryClient = useQueryClient();
const statsQuery = useAdminStats();
@@ -222,6 +245,7 @@ export default function AdminDashboard() {
<Button
variant="default"
size="sm"
className="cursor-pointer"
onClick={() => {
if (libraries.length > 0) {
scanAll.mutate();
@@ -311,13 +335,13 @@ function StatsRow({
{
label: "Total Movies",
value: stats.total_movies.toLocaleString(),
sub: `of ${stats.total_items.toLocaleString()} items`,
sub: formatFileCount(stats.total_movie_files),
icon: <Film className="h-4 w-4" />,
},
{
label: "Total Shows",
value: stats.total_shows.toLocaleString(),
sub: `${stats.total_files.toLocaleString()} files total`,
sub: formatFileCount(stats.total_show_files),
icon: <Tv className="h-4 w-4" />,
},
{
@@ -329,7 +353,7 @@ function StatsRow({
{
label: "Storage",
value: storageDisplay,
sub: `${stats.total_files.toLocaleString()} files`,
sub: formatFileCount(stats.total_files),
icon: <HardDrive className="h-4 w-4" />,
},
];
@@ -629,7 +653,26 @@ function LibrariesCard({
isLoading: boolean;
error: unknown;
}) {
useEventChannel("scans");
const scanLibrary = useScanLibrary();
const cancelScans = useCancelLibraryScans();
const { data: activeScans = [] } = useActiveScans();
const activeScansByLibraryId = useMemo(() => {
const scansByLibraryID = new Map<number, ScanRun[]>();
for (const scan of activeScans) {
if (scan.status !== "accepted" && scan.status !== "running") {
continue;
}
const scans = scansByLibraryID.get(scan.library_id) ?? [];
scans.push(scan);
scansByLibraryID.set(scan.library_id, scans);
}
for (const scans of scansByLibraryID.values()) {
scans.sort(compareActiveScans);
}
return scansByLibraryID;
}, [activeScans]);
return (
<Card>
@@ -652,47 +695,95 @@ function LibrariesCard({
No libraries configured.
</div>
) : (
libraries.map((lib) => (
<div
key={lib.id}
className="bg-surface border-border hover:bg-surface-hover flex cursor-pointer items-center gap-3 rounded-md border p-3 transition-colors duration-150"
>
{lib.poster_url ? (
<img
src={lib.poster_url}
alt={lib.name}
className="border-border h-8 w-14 flex-shrink-0 rounded border object-cover"
/>
) : (
<div className="bg-primary/5 border-primary/10 flex h-10 w-10 flex-shrink-0 items-center justify-center rounded-lg border">
<Library className="text-primary h-4 w-4" />
libraries.map((lib) => {
const activeLibraryScans = activeScansByLibraryId.get(lib.id) ?? [];
const primaryActiveScan = activeLibraryScans[0];
const hasActiveScan = activeLibraryScans.length > 0;
const isScanStarting = scanLibrary.isPending && scanLibrary.variables === lib.id;
const isCancellingScan = cancelScans.isPending && cancelScans.variables === lib.id;
const scanProgressLabel = primaryActiveScan
? formatDashboardLibraryScanProgress(primaryActiveScan, activeLibraryScans.length)
: isScanStarting
? "Starting scan..."
: "";
return (
<div
key={lib.id}
className="bg-surface border-border hover:bg-surface-hover flex items-center gap-3 rounded-md border p-3 transition-colors duration-150"
>
{lib.poster_url ? (
<img
src={lib.poster_url}
alt={lib.name}
className="border-border h-8 w-14 flex-shrink-0 rounded border object-cover"
/>
) : (
<div className="bg-primary/5 border-primary/10 flex h-10 w-10 flex-shrink-0 items-center justify-center rounded-lg border">
<Library className="text-primary h-4 w-4" />
</div>
)}
<div className="min-w-0 flex-1">
<div className="text-sm font-bold">{lib.name}</div>
<div className="text-muted-foreground flex flex-wrap items-center gap-x-1.5 gap-y-0.5 text-[11px]">
<span>
{lib.type} · {lib.paths.length} {lib.paths.length === 1 ? "path" : "paths"}
</span>
{scanProgressLabel ? (
<>
<span className="text-border/70">·</span>
<span className="max-w-[22rem] truncate text-amber-300 tabular-nums">
{scanProgressLabel}
</span>
</>
) : null}
</div>
</div>
)}
<div className="min-w-0 flex-1">
<div className="text-sm font-bold">{lib.name}</div>
<div className="text-muted-foreground text-[11px]">
{lib.type} · {lib.paths.length} {lib.paths.length === 1 ? "path" : "paths"}
<div className="flex flex-shrink-0 items-center gap-2">
<Button
variant={hasActiveScan ? "destructive" : "ghost"}
size="icon"
className="h-7 w-7 cursor-pointer"
onClick={(e) => {
e.stopPropagation();
if (hasActiveScan) {
cancelScans.mutate(lib.id);
return;
}
scanLibrary.mutate(lib.id);
}}
disabled={hasActiveScan ? isCancellingScan : isScanStarting}
title={hasActiveScan ? "Stop Library Scans" : "Scan Library"}
aria-label={hasActiveScan ? `Stop scans for ${lib.name}` : `Scan ${lib.name}`}
>
{hasActiveScan ? (
<Square className="h-3 w-3 fill-current" />
) : (
<ScanLine className={cn("h-3 w-3", isScanStarting && "animate-pulse")} />
)}
</Button>
<div
className={cn(
"h-2 w-2 rounded-full",
hasActiveScan || isScanStarting
? "bg-amber-400 shadow-[0_0_6px_rgba(251,191,36,0.65)]"
: lib.enabled
? "bg-green-500"
: "bg-muted-foreground/30",
hasActiveScan && "animate-pulse",
)}
title={
hasActiveScan
? "Scan in progress"
: lib.enabled
? "Library enabled"
: "Library disabled"
}
/>
</div>
</div>
<div className="flex flex-shrink-0 items-center gap-2">
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={(e) => {
e.stopPropagation();
scanLibrary.mutate(lib.id);
}}
title="Scan"
>
<ScanLine className="h-3 w-3" />
</Button>
<div
className={`h-2 w-2 rounded-full ${lib.enabled ? "bg-green-500" : "bg-muted-foreground/30"}`}
/>
</div>
</div>
))
);
})
)}
</CardContent>
</Card>
@@ -58,6 +58,7 @@ export default function PlaybackSettings() {
{ value: "auto", label: "Auto" },
{ value: "qsv", label: "Intel Quick Sync (QSV)" },
{ value: "vaapi", label: "VA-API" },
{ value: "nvenc", label: "NVIDIA NVENC" },
{ value: "none", label: "Software" },
]}
value={form.getValue("playback.hw_accel")}
@@ -202,6 +203,8 @@ function formatResolved(resolved: string): string {
return "Intel Quick Sync (QSV)";
case "vaapi":
return "VA-API";
case "nvenc":
return "NVIDIA NVENC";
case "none":
return "Software";
default: