Compare commits

..
Author SHA1 Message Date
Quick fe934b59d3 feat(audiobooks): add first-party audio playback API 2026-06-08 17:32:26 -04:00
9e29e7b330 feat(security): encrypt server-owned credentials at rest (#45) (#95)
* feat(security): encrypt server-owned credentials at rest

Introduce AES-256-GCM at-rest encryption (HKDF-derived from a required
SECRET_KEY) for server-owned credentials, with row-bound AAD, a versioned
enc:v1: envelope, and an idempotent startup backfill.

- internal/secret: cipher + RowAAD/SettingsAAD + the startup backfill engine.
- SECRET_KEY required at bootstrap; cipher threaded as an explicit dependency.
- server_settings: EncryptedSettingsRepo decorator over the audited
  SensitiveSettingKeys (also drives admin redaction); the config watcher and
  watch-sync settings reads decrypt too.
- Arr keys inline-encrypted; the ambiguous SecretResolver indirection removed
  from requests/autoscan.
- Per-table columns encrypted: subtitles, watch-sync, webhook-sync (not
  webhook_secret), history-import, and the jellycompat session's bridged Silo
  access/refresh tokens.
- Startup backfill (resolve-then-encrypt for arr refs) is best-effort and
  primary-node gated.

Equality-looked-up secrets and plugin_runtime_configs.config_value are out of
scope (need hashing / cross-repo design) — see
docs/architecture/secret-encryption.md.

Refs #45

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(compose): require SECRET_KEY in docker-compose

The server now fatals without SECRET_KEY, so the integrated service (and the
commented distributed proxy/transcode examples) pass it through with a
fail-fast guard matching the existing MEDIA_ROOT pattern. Distributed worker
nodes must use the SAME key as the primary to decrypt shared data.
Generate with: openssl rand -base64 48.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): encrypt history import session credentials

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:25:48 -04:00
d3626fca00 feat(jellycompat): add /Users endpoint and unified sa_ admin-key auth (#91)
* fix(api): allow HEAD on /api/v1/direct-download

Firefox (and some download managers) issue a HEAD request before
starting a download. The route only registered GET, so HEAD returned
405 Method Not Allowed and the browser aborted the download.

Mirrors the pattern already used by /stream/{session_id}, which
registers both GET and HEAD on the same handler. ServeDirect is built
on http.ServeContent / ServeFile, which natively handle HEAD by
writing headers without a body, so no handler changes are needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: design jellyfin autoscan scan compatibility

* refactor(scan): extract scan trigger resolver

* refactor(api): share scan target resolution

* feat(jellycompat): accept admin api keys for autoscan

* feat(jellycompat): add autoscan media update route

* docs: document jellyfin autoscan setup

* fix(jellycompat): harden autoscan auth and batch scan enqueue

- Reject nil API keys and bound last-used update with a 5s timeout
- Stop leaking internal queue errors in autoscan responses
- Batch scan enqueues via new CreateBatch and reuse folder list across path resolves

* chore: add planning docs and requests updates

- Add plans for date-named episodes and Jellyfin autoscan compat
- Update requests handlers, service, and UI hooks
- Remove Makefile.local.example

* refactor(scantrigger): drop redundant Target.LibraryID field

- Read library ID from Target.Folder.ID everywhere
- Guard scan queue enqueue against nil Folder
- Simplify admin API key auth error plumbing

* fix(catalog): gate search overview-only matches behind title FTS

- Always apply stats CTE + CROSS JOIN so single-word queries no longer flood results with description-only hits
- Require overview_rank >= 0.15 for overview-only fallback rows
- Switch title gate from contiguous LIKE to title_rank > 0 so reordered-token title matches aren't demoted

* ci(docker): build image on push to main via self-hosted runner

- Trigger Docker image builds on pushes to main instead of nightly cron
- Run on self-hosted Linux runner
- Drop the `nightly` tag

* docs(specs): add design for TMDB-backed request section in search

Adds the design for surfacing requestable TMDB results inside the main
catalog search (Cmd+K dialog and full results page) as a clearly
delimited "Request to Add" section that never blocks or displaces
library results.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(specs): address Codex adversarial review for search request section

Splits discovery eligibility from submission eligibility so blocked
and quota-exhausted viewers still see the requestable section with
disabled per-row CTAs, matching the documented behavior. Documents
the required extensions to useRequestSearch — signal forwarding,
viewer-identity-keyed cache, and invalidation on auth/profile/
settings/limit changes — so the planned 5-minute staleTime is
safe and cancellation works as described.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(plans): add implementation plan for search request section

Twelve TDD tasks covering: api() signal contract test, useCanRequest
hook, viewer-keyed requestKeys.search, useRequestSearch extension
(signal + viewer key + 5min staleTime + enabled override), invalidation
cascade tests, RequestPosterCard optional onRequest, RequestToAddSection
component (dialog + grid variants), GlobalSearch and Catalog wiring
with empty-state suppression for the library-0/TMDB-pending edge case,
final lint/test pass, and manual smoke. Notes a single deviation from
the spec: submitDisabledReason is null in the initial implementation,
with per-row request data driving disabled UI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(plans): address Codex adversarial review for search request section

Fixes the high-severity finding that RequestToAddSection's internal
useRequestSearch call was not gated on discoveryEnabled, allowing
/api/v1/requests/search and TMDB lookups to fire for users without
request access. The plan now (1) passes { enabled: discoveryEnabled }
to the section's hook, (2) gates the parent mount in GlobalSearch and
Catalog on canRequest.discoveryEnabled as defense in depth, and (3)
adds tests asserting both the enabled forwarding and the no-mount
behavior when discovery is disabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(api): pin AbortSignal forwarding contract on api()

* feat(hooks): add useCanRequest gating hook for discovery eligibility

* refactor(keys): add viewerKey to requestKeys.search

* feat(requests): key useRequestSearch by viewer, forward signal, raise staleTime

* test(requests): document viewer-keyed cache isolation and invalidation cascade

* feat(request-card): make onRequest optional on discover variant

* feat(search): add RequestToAddSection dialog variant

* feat(search): add RequestToAddSection grid variant for Catalog page

* feat(search): render RequestToAddSection in the Cmd+K dialog with empty-state suppression

* feat(catalog): render RequestToAddSection grid with empty-state suppression

* chore(web): format request search section

* test(web): avoid unsupported Array.at in search request tests

* commit message

{"subject":"fix(search): prevent empty-state flash before TMDB fallback renders","body":"- Add isResolving to useCanRequest and gate empty states on it across GlobalSearch and Catalog\n- Debounce TMDB query in Catalog and hide ItemGrid when the request section may rescue an empty library\n- Track per-card submit state in RequestToAddSection grid so concurrent requests don't trample each other\n- Suppress anonymous TMDB request-search fetches to avoid cross-viewer cache leakage"}

* fix(jellycompat): tolerate autoscan sidecar updates

* fix(webhooksync): skip events for unmapped external users

- Require explicit profile mapping instead of falling back to the default profile
- Update settings UI copy to reflect that unmapped users are ignored

* refactor(admin): show per-section loading and error states

- Replace page-level loading gate with skeletons per section on dashboard and stats
- Surface query errors inline instead of blocking the whole page
- Disable "Scan All Libraries" when no libraries are configured

* fix(search): address request search review feedback

* feat(subtitles): restore upload management

* feat(auth): add assignable user permissions

* feat(auth): expose user permissions

* feat(api): authorize item metadata curation

* feat(api): route metadata curation by permission

* feat(web): add permission helpers

* feat(web): assign metadata curation permission

* fix(web): keep device profile hooks unconditional

* feat(web): show metadata tools to curators

* fix(auth): address metadata curation review issues

* fix(auth): tighten curator job response review fixes

* docs: add metadata curation permission plan

* test(auth): expand session revocation coverage

* docs: add PageBack component design spec

* fix(auth): gate media file paths on metadata curation permission

- Allow curators (not just admins) to view media file paths and locations
- Apply library access filter to file-level access checks

* fix(metadata): break duplicate provider candidate ties

- Score candidate metadata completeness and auto-match the richer duplicate when title/year/type tie
- Enrich near-duplicate candidates via the provider chain before initial match selection
- Seed both movie and series match queues for mixed-type libraries and wait for TV queue settle
- Add taskmanager worker test coverage and a plan doc for the tie-breaker work

* feat(ui): add shared PageBack component for consistent back navigation

Replace the eight inconsistent back affordances across user-facing pages
with a single absolute-positioned chevron pill, so the control lives in
the same screen position regardless of title length or hero content.

DetailBreadcrumb keeps its textual hierarchy path but no longer owns the
back chevron; PageBack does. DetailHero gains a topNav slot consumed by
Movie/Series/Season/Episode/Request detail pages. Non-hero pages drop
their bespoke back buttons and add PageBack inside a relative wrapper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(ui): add floating variant to PageBack for sticky nav

- Add `floating` prop to pin PageBack to viewport on lg+ screens
- Switch styling from glass-subtle to glass with shadow for better contrast
- Use floating variant on SettingsLayout

* fix(ui): make PageBack destinations deterministic

* feat(episode-carousel): highlight currently viewed episode

- Add "Now Viewing" badge with pulsing indicator on current episode
- Replace border with primary-color ring for current episode card
- Set aria-current="page" on links to the current episode

* feat(jellycompat): sign image tags and accept them without session

- HMAC-sign image tags using the configured JWT secret
- Serve item/season/episode images via signed tag without requiring a session or cache hit

* fix(jellycompat): harden signed image tags

* fix(jellycompat): stabilize signed image tags across restarts

{"subject":"fix(jellycompat): stabilize signed image tags across restarts","body":"- Sign library poster and episode parent series image tags from canonical paths/thumbhashes instead of presigned URLs so tags survive restarts\n- Accept signed canonical tags in the image handler without a session and fall back to legacy URL-derived cache tags\n- Always fetch series detail for episodes to build stable parent image tags"}

* fix(metadata): accept exact cross-provider match ties

* Optimize episode added_at sorting

* fix(libraryingest): treat drainer shutdown cancel as clean stop

TV/series full scans (libraries with new or updated items) were recorded as
"cancelled" with an empty error message and never completed matching.

When the file-walk finishes, the ingest executor waits out a settle window and
then calls stopDrainers() to shut down the concurrent match goroutines. That
cancels the drainer context while a ProcessBatchByFolderAndPathPrefix call may
still be in flight. The drainer treated the resulting context.Canceled as a
fatal error: it pushed the error to drainerErrCh and called cancel() on the
whole scan context, so scanqueue.process() mapped it to cancelRun().

Large/slow libraries (many series, slow provider lookups) keep a batch in
flight continuously, so stopDrainers() almost always landed mid-call and the
scan was cancelled; small/fast libraries were usually idle at that instant and
completed normally.

Treat a cancelled drainer context as a deliberate shutdown: return cleanly
without escalating. Genuine external cancellation still reaches the run via the
main goroutine's scanCtx checks, so real cancels are not swallowed.

Adds a regression test (settle window made injectable) that fails against the
old handler with 'concurrent match scope ...: context canceled' and passes
with the fix.

* perf(catalog): add episode browse index fast path

* fix(catalog): address episode catalog review feedback

* Fix settle-window drainer cancellation in library ingest

* fix(catalog): support relative date filters

* fix(collections): cap smart collection results

* fix(library): preserve episode browse url

* fix(catalog): use season posters for episode cards

* feat(sections): show episode context in cards

* feat(calendar): show local episode airtimes

* Enable DRI passthrough in docker compose

Co-authored-by: Codex <noreply@openai.com>

* feat(ui): paginate the Ambiguous Roots table

Match the sibling tables on the Admin Libraries page (Troubleshooting/unmatched):
use the existing usePagination hook + PaginationBar (10/page, auto-hidden when
<=10 rows), render pag.rows, and reset to page 0 when the library selector or
search filter changes.

* feat(admin): enlarge match-candidate posters + hover-to-enlarge

Unmatched-item match dialog rendered candidate posters at 44x64px, too small
to identify a film. Bump to 64x96 (2:3) and add a portaled Tooltip hover preview
(192x288) using the existing poster image, so operators can tell candidates apart.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(admin): search unmatched items across the whole table, not just the page

The unmatched-items search filtered only the current page's rows client-side.
Push the query server-side: HandleListUnmatchedItems takes an optional 'q' param
and filters title/library/type/status with parameterized ILIKE across all rows,
paginating the filtered set. Frontend hook takes a debounced search, resets to
page 1 on change, keeps the section mounted while searching. Also fixes stale
test mocks that returned the pre-pagination array shape instead of {items,total}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(admin): prettier-format match dialog; reset unmatched page in onChange

Run prettier over the Tooltip-wrapped poster JSX, and reset the unmatched-items
page in the search input's onChange rather than a useEffect (avoids the
react-hooks/set-state-in-effect warning / cascading renders).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(admin): wire QueryClientProvider + missing hook mocks so the suite runs

The AdminLibraries test file failed all 6 tests with 'No QueryClient set' on
this branch and at the parent commit -- pre-existing infrastructure gap. With
that fixed, several hooks that the page imports (useCancelLibraryScans,
useLibraryRoots, useUpsertLibraryRootOverride, useDeleteLibraryRootOverride,
useActiveScans) and the UNMATCHED_PAGE_SIZE constant also needed mocking. One
stale assertion on the renamed 'Root path' header is updated; the deeper
troubleshooting test, which mocked useSkippedLibraryRoots but the section was
refactored to useLibraryRoots(_, 'ambiguous'), is skipped with a TODO -- a real
rewrite is needed and is out of scope for this MR.

5 of 6 tests now run and pass; the 6th is properly flagged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(admin): search all unmatched item library memberships

* fix(naming): strip unsubstituted Sonarr tokens ({TvdbId}/{imdb-}) from titles

These tokens survived the provider-tag regex ([\w]+ doesn't match braces),
polluting parsed titles (e.g. 'A Girl & Her Guard Dog [tvdb-{TvdbId}]') so
they could not score-match. Broaden the regex to drop {...} and empty tokens.

* fix(naming): numeric-only titles are not bare provider IDs

'86' / '22 7' were parsed as trailing tvdb ids, tripping the trusted-ID gate
so the correct title match was rejected. Require a letter in the name before
treating a trailing number as a bare id.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(naming): document bare-id trade-off; cover CJK title + movies numeric

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): auto-accept a year-corroborated single distinct show

A search that resolves to one distinct show (one candidate, or the same
title+year returned once per source as unmerged TVDB/TMDB rows) whose year
matches the parsed year is now auto-accepted via the existing top-ranked
candidate, even when the fuzzy title score is in the 55-69 band. The 55/70/15
thresholds are unchanged; this only adds a year-gated acceptance for
effectively-unique results (recovers lone-correct-result items like 1201 (1993)).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(matcher): exercise the single-distinct-show guard properly + conflicting-ID case; doc notes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(matcher): tolerate concurrent-merge ErrItemNotFound in series episode-link ensure

A scan drainer and the background MatchWorker can process the same folder
concurrently. When a provider-ID merge moves a series' episodes to the survivor
and deletes the source, an in-flight ensureSeriesEpisodeLinks(sourceID) hits
catalog.ErrItemNotFound and was failing the whole scan. The episodes are already
reattached, so this is benign: log and continue (matching the lenient call sites)
instead of failing. Genuine errors still abort.

* diag(matcher): debug-log per-candidate match scores

Adds a DEBUG-gated log in selectInitialMatchCandidate printing each scored
candidate (title/year/type/sources/provider_ids/score) against the hint, so
operators can see why an item did or didn't auto-match. Zero-cost when debug
logging is off; no change to matching behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): resolve cross-source ties by library provider priority

Accept a year-corroborated single distinct show when the TOP tie-group (within
15 pts of best) is one show across providers, ignoring low-score noise below it,
and pick the winner by the library's metadata-provider chain order (providerPriority,
highest-first; falls back to top-scored). Recovers items like '100 Days Wild'
that are returned identically by TVDB and TMDB. Thresholds (55/70/15) unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(matcher): accept cross-source-corroborated ties without a hint year

When the top tie-group is one distinct show returned by 2+ distinct providers
(candidatesAreSingleDistinctShow already verifies matching title+year), accept it
even if the hint has no parsed year (year-less folders like '100 Deeds for Eddie
McDowd'). Multi-source agreement substitutes for the year guard; lone single-source
no-year results stay subject to the single-candidate >=70 gate. Thresholds unchanged.

* diag(matcher): debug-log provider search query + per-provider result counts

Adds DEBUG logs in the ModeInitialMatch search path: each provider's result
count for the query, and the assembled raw/candidate totals. Lets us see when a
provider search returns zero ('no metadata found') vs a scoring/tie issue.
Zero behavior change.

* fix(naming): parse bare bracketed IMDb IDs ([tt10011226]/{tt...})

Folders tagged with a bare IMDb id in brackets (Plex/Kodi style, e.g.
'17 Blocks (2021) [tt10011226]') had the id silently dropped — folderIDPattern
needs an 'imdb-' prefix and trailingImdbIDPattern needs an un-bracketed trailing
tt-id. Recognize bracketed bare tt-ids so these items get the trusted-ID match
path instead of falling to title+year scoring.

* feat(metadata): match sole exact-title candidate despite year off by <=2

Folder years routinely differ from provider release years by a year or two
(festival vs wide release, regional dates), zeroing the year bonus and leaving
a lone exact-title candidate at 63-68 — just under the single-candidate >=70
gate (e.g. Dead Reckoning 1947 vs 1946, 17 Blocks 2021 vs 2019, Stasi FC). Add
title corroboration to the existing lone-result rule: a sole distinct show whose
normalized title exactly matches and whose year is within +/-2 is accepted. The
55 floor still rejects low-similarity titles (e.g. Hotel Transylvania Puppy! vs
Puppy!). No 55/70/15 threshold change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(lint): gofmt single-space alignment in root_inference.go var block

When inferProviderTagRe was broadened to handle unsubstituted Sonarr token
placeholders ({TvdbId}/{imdb-}), the regex grew long enough that gofmt prefers
single-space rather than column-aligned spacing across the var block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metadata): tighten matcher and bare IMDb parsing

* docs: add resilient library deletion design spec

Batched, deadlock-retrying rewrite of delete_library to replace the
single multi-minute transaction that deadlocks on large libraries.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs: add implementation plan for resilient library deletion

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(catalog): add deadlock-retry helper for batched deletes

* test(catalog): clarify cancel-path expectation in retry test

* feat(catalog): add deleteInBatches loop helper

* refactor(catalog): make image-dir helpers querier-agnostic

Add rowQuerier interface satisfied by both *pgxpool.Pool and pgx.Tx.
Split collectImageDirs into collectRawImageDirs (raw collection) and a
thin wrapper that filters via filterUnreferencedImageDirs. Both helpers
now accept rowQuerier so a later task can call them from pool-level
batch deletes without an open transaction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(catalog): delete libraries in deadlock-retrying batches

Replaces the single multi-minute delete transaction with phased, batched
autocommit deletes (orphan items, media files, memberships, folder row),
each retried on deadlock. Holds only short locks, survives concurrent
writers, and is resumable on failure.

* refactor(catalog): wrap orphan-batch iteration error

* fix(catalog): clamp still/poster/logo backdrops to largest cached variant

Episode stills used as backdrops only exist at w500/w300 in the cache, so
requesting a w1280/w1920 backdrop width 404s. Add catalog.BackdropVariantPath
+ imageTypeFromCachedPath and route featured (w1920) and Continue Watching /
Next Up (w1280) backdrops through it; still/poster/logo paths clamp to their
type's largest cached variant while real backdrops keep the requested width.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* perf(startup): defer non-critical init off the HTTP listener path

Collect catalog-size-dependent seeding (metadata match queues, legacy
series-group cleanup) and the watch-provider scrobble sweep into a
backgroundInit slice that runs sequentially in a background goroutine after
the server is ready, instead of blocking startup before the listener accepts
connections. Steps log failures and stop early on shutdown.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(migrations): make air_timezone column add idempotent

Use ADD COLUMN IF NOT EXISTS so re-running 162 on a database that already has
the column is a no-op.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* build: stamp git revision via Makefile ldflags

`make build` did not inject buildinfo's `revisionOverride`/`dirtyOverride`
ldflags (the Dockerfile already does), so binaries built via make report
their version as "unavailable" in the admin Build panel whenever Go's VCS
metadata isn't embedded. Mirror the Dockerfile by computing the git
revision + dirty state and passing them through `-ldflags -X`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): accessibility & UX fixes from visual QA pass

Accessibility (WCAG AA):
- Lighten the Standard-theme `--muted-foreground` (#6e6e78 -> #9696a0,
  ~3.4:1 -> >=5.3:1) and darken the light-theme equivalent so secondary
  text meets 1.4.3 contrast app-wide; the opt-in High Contrast mode is no
  longer the only conformant path.
- Give icon-only controls accessible names (4.1.2): the password show/hide
  toggle (also drop tabIndex={-1} so it's keyboard reachable), and the
  Edit/Delete/health/copy/refresh actions across the Users, Libraries,
  Nodes, API Keys, Catalog Maintenance and Job History admin tables.
- Fix the Switch off-state (invisible track -> visible border + fill) and
  the PlaybackSettings SettingRow label association (the <label htmlFor>
  pointed at a wrapping <div>; the id now lands on the Switch/SelectTrigger).
- Login: wrap the card in <main> and add an <h1>; Profiles: add an
  accessible PIN-protected label and a corner lock badge.
- Player + catalog: role="status" on the initial loading overlay; scope the
  catalog count ("0 in library" for search) and announce it via aria-live;
  trim the verbose poster-link name to the title.

UX / consistency:
- Emphasize overdue scheduled tasks (warning colour + icon + word, not
  colour alone).
- Per-source catalog subtitles instead of one shared string.
- Add a Reconnect affordance when the admin log stream drops (it does not
  auto-retry).
- Page titles for Watch Party + all admin sub-pages (incl. plugins); admin
  heading capitalisation normalised to Title Case.
- Show "dev build" instead of "unavailable" when no build revision is
  stamped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): re-check orphan status when deleting library items

Orphan detection moved outside the media_items delete in the batched
library-delete rewrite, opening a TOCTOU race: a concurrent scan/import
could attach one of the collected content IDs to another library between
collectOrphanBatch and the delete, after which the unconditional
`DELETE FROM media_items WHERE content_id = ANY($1)` would still remove the
shared row and cascade away the newly-added membership — dropping the item
from the other library. Re-check the orphan invariant inside the delete
(NOT EXISTS a membership in another folder) and count rows actually deleted.

Addresses PR #21 review (P1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(metadata): persist a cleared air_timezone instead of skipping it

Clearing a previously-set air timezone sent JSON null, which decodes to a
nil *string that UpdateMetadata treats as "skip this column", so the old
value remained. The dialog now sends "" (accepted by ValidateAirTimezone),
and UpdateMetadata maps air_timezone through NULLIF so an empty value
persists as SQL NULL (matching the nullable column) rather than "".

Addresses PR #21 review (P2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(startup): sweep open scrobbles before accepting playback

The open-scrobble sweep was queued in the deferred background-init list,
which runs concurrently with the HTTP listener; a resume immediately after
restart could start new scrobbles before the previous process's open
sessions were stopped, leaving overlapping/stale scrobbles on remote
providers. Run the sweep synchronously before the listener starts, bounded
by a 30s timeout so an unreachable provider can't hang startup (the heavier
non-critical init stays deferred).

Addresses PR #21 review (P2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): scan air_timezone in paginated item queries

scanItemsWithTotal was not updated for the new air_timezone column, yet the
shared column lists it reads (itemColumns, qualifiedListItemColumns) include
it. Search and BrowseFavorites build their SELECTs from those lists with
COUNT(*) OVER (), so each row carried one more column than the scan had
destinations and every call failed at scan time with a pgx mismatch. Add the
missing &item.AirTimezone target between AirTime and ShowStatus.

Found during PR #21 review (critical: Search/Favorites runtime regression).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* revert(startup): keep open-scrobble sweep deferred for fast startup

Reverts 62563f5c. Running the sweep synchronously before the listener could
add up to 30s to restart-before-playback when a watch provider is
unreachable, which regresses the deliberate startup-deferral from a1a6c6cf.
Prefer the fast-startup behavior and accept the small window where a resume
immediately after restart may create a duplicate scrobble; the sweep returns
to the deferred background-init list. (Panic-safety for that list is added in
a follow-up commit.)

Per PR #21 review decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(startup): recover from panics in deferred background init

The deferred background-init steps run in a detached goroutine after the
HTTP listener is already accepting connections. An unrecovered panic in any
step (queue seeding, legacy cleanup, scrobble sweep) would crash the entire
live server. Wrap each step in a recover that logs the panic with a stack
and continues to the next step.

Found during PR #21 review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): make usernames and emails case-insensitive

Login identifiers were compared case-sensitively, so "John" and "john"
were distinct accounts and a user could not log in unless they matched the
exact casing used at registration.

Convert users.username and users.email to the citext type (migration 165).
citext compares case-insensitively while preserving the originally stored
casing for display, so the existing unique constraints become
case-insensitive and `WHERE username = $1` / `email = $1` lookups match
regardless of case with no change to the query code itself.

Also add auth.NormalizeUsername/NormalizeEmail (trim-only; case preserved),
applied at the repository chokepoints (Create, Update, GetByUsername,
GetByEmail) and before validation in the create paths, so surrounding
whitespace no longer defeats matching or creates lookalike accounts.

Verified non-destructively against the dev DB: mixed-case lookups resolve
to the same row, case-variant inserts are rejected by the unique
constraint, and the down migration cleanly reverts to text.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(sections): add trending_discover home section

A library-agnostic home section that surfaces external global trending
(TMDB or Trakt, admin-selectable) mixing movies + series, matched to
titles in the viewer's enabled libraries. TMDB uses /trending/all/{window}
(natively mixed); Trakt merges trending movies + shows. Fetched live with
a 1h in-process cache, so no background job or stored collection — and no
per-library duplication.

Appears in the admin section gallery via its recipe presets (TMDB Trending
Today/This Week, Trakt Trending); featured -> hero via the existing flag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add trending_discover persistent snapshot design

Replace the in-process 1h trending cache with a background-refreshed,
persisted snapshot for reliability under upstream failure and sync-run
observability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add trending_discover persistent snapshot implementation plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(sections): tidy trending_discover fetch and cache helpers

Extract newTrendingEntry, reuse orderMediaItems, and collapse concurrent
cache-miss loads with singleflight. Baseline for the persistent snapshot work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sections): add trending_discover_snapshots table

* feat(sections): add trending snapshot model and repository

* feat(sections): list enabled trending_discover section configs

* feat(sections): add trending refresher with persisted snapshots

* feat(tasks): add refresh_trending_discover task

* refactor(sections): read trending_discover from persisted snapshot

* feat: wire trending refresh task and snapshot reader

* chore(sections): satisfy lint (wrap trakt errors, lift source/window constants)

* fix(migrations): renumber trending_discover_snapshots 166 -> 167

The shared dev DB already recorded version 166 (166_trending_blend_collection_type
from another branch), so the integer-version migration runner silently skipped our
166 and the table was never created — the trending section errored out empty.
167 is the next free version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sections): harden trending refresher per PR review

- Interleave Trakt movies/shows by rank so the mixed row shows both types
  instead of burying all series past the display limit.
- Treat any Trakt sub-fetch failure as fatal (errors.Join) so a partial
  result never overwrites the last-good snapshot with a media type missing.
- Skip non-title entries (TMDB trending/all returns media_type "person") in
  both ID batching and ordering so they can't match an unrelated library title.
- Guard the refresh task against a nil refresher.
- Tests: person skip, Trakt interleave, Trakt partial-failure preserves
  last-good, snapshot read error propagation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(catalog): project air_timezone in episode catalog subquery

episodeCatalogSelectBody is the derived "mi" relation that episode catalog
hydration and preview read qualifiedListItemColumns("mi") from. The local
episode airtimes feature (808f265f) added air_timezone to the shared column
lists but not to this hand-written subquery, so the outer projection
referenced mi.air_timezone, which the subquery never exposed.

Postgres returns SQLSTATE 42703 (undefined_column), which is not one of the
codes episodeCatalogEntriesUnavailable treats as "fast path unavailable" (it
only catches 42P01/42883), so episode catalog requests failed with HTTP 500
instead of degrading. movie and series scopes query media_items directly, so
the column is present there and only episode scope broke.

Add si.air_timezone to the subquery, and add a regression test asserting that
episodeCatalogSelectBody exposes every column qualifiedListItemColumns reads
off mi, so future additions to the shared column lists cannot silently drift
from the episode read model again.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(calendar): order events by viewer-local wall-clock time

The local-airtime change re-sorted calendar events in Go using air_at,
the absolute UTC instant, which is nil whenever air_timezone is unset.
Since air_timezone is only inferred for a few networks/countries, most
events fell through to the alphabetical title tiebreak while still
displaying their raw air_time, so each day appeared scrambled.

Sort each local day by the wall-clock time the viewer actually sees,
mirroring the client: zoned events convert air_at into the viewer
timezone, unzoned events use the raw air_time, and date-only entries
(no air_time) sort last. The timezone reasoning lives in the new
catalog.CalendarEventLocalTime helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(calendar): add presets design spec and implementation plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): generalize personal filter to an id-set restriction

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): add per-profile followed/favorites/watchlist/watched resolvers

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): resolve presets to id-sets and overlay watched status

Also drops the now-unused Filter/UserID/ProfileID fields from the
blendUpcomingIntoDiscoverRows CalendarFilter literal in recommendations.go,
which only wants an unrestricted windowed query.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): wire popular and trending sources into calendar handler

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): add watched field to CalendarEvent type

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): preset selector with responsive pills, persistence, empty-state nudge

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): dim and check-mark already-watched event cards

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(calendar): hide server-wide Popular preset in web UI for now

Popular reflects server-wide watch counts, which are sparse on a
low-traffic server. Hidden from the selector, URL allowlist, and
empty-state nudge; backend filter and the CalendarFilter type are
left intact so re-enabling is a one-line change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(calendar): simplify preset handler and reuse storage util

- Extract hardcoded trending snapshot source/window to named constants.
- Collapse the three identical personal-preset nil-checks into one case.
- Persist the selected preset through the shared storage util (try/catch
  wrapped) instead of raw localStorage with manual SSR guards.
- Derive KNOWN_FILTERS from PRESET_OPTIONS so the lists can't drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(jellycompat): add /Users endpoint and unified sa_ admin-key auth

A user could not connect Tunarr to the Jellyfin-compat API. Tunarr's media-source
health check probes both /Users/Me and /Users; for API-key connections (its
recommended method) there is no "me", so it relies on GET /Users, which we did
not implement. Adding it alone was insufficient: a Silo sa_ admin key only
authorized two autoscan routes, so an API-key source would pass the ping but 401
on every browse/stream call.

This adds GET /Users and unifies auth so an sa_ admin key authorizes the same
browse/stream routes a session token does — matching Jellyfin, where an API key
authorizes every endpoint:

- GET /Users returns the caller's own user as a single-element list
  (current-profile-only; Silo is multi-account, so listing all users would leak
  across households). Behavior verified against real Jellyfin 10.11.8.
- An sa_ admin key synthesizes a compat session bound to the account's primary
  profile, injected into request context so existing handlers work unchanged.
  Applied to the browse group (RequireSessionOrAPIKeySession) and the stream
  group (PlaybackSessionAuth); /Library/VirtualFolders keeps its admin-bool path.
- The key + owning user are re-validated on every request (revocation is
  immediate); only the primary-profile lookup is cached. HLS follow-ups that
  carry only PlaySessionId resolve the negotiated session's sa_ CompatToken.

Validated end-to-end on dev: connect -> list user -> libraries -> browse ->
PlaybackInfo -> HLS stream, including PlaySessionId-only follow-ups.

Security notes: an admin key acts as the primary (parent) profile, so it bypasses
child-profile parental/PIN restrictions and can mutate the primary profile's
watch state — acceptable for an admin-trust key.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Silo Server Migration <noreply@silo-server.invalid>
Co-authored-by: zZebrahz <zzebrahz@gmail.com>
Co-authored-by: CoffeeKnyte <67730400+CoffeeKnyte@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Silo Server Developer <warmasterx555@gmail.com>
2026-06-08 14:25:10 -04:00
d3v1l1989andGitHub 65ea9b651c fix(web): drop backdrop-blur from Pill card overlay preset (#89)
The Pill overlay preset was the only one of the five to carry
`backdrop-blur-sm` (backdrop-filter: blur(4px)) on its badge class.
Each enabled badge renders its own element, so a typical library grid
produces hundreds of backdrop-filter layers the browser must re-sample
and blur on every repaint — including every scroll frame, since the
poster art behind each badge moves while scrolling. This caused severe
scroll/interaction jank, most visibly on Chrome on Windows.

The blur is also effectively invisible: the Pill background is already
rgba(20,20,30,0.7) (70% opaque), so removing it has no meaningful visual
impact while eliminating the per-frame GPU cost. The other four presets
are unaffected.
2026-06-08 09:57:47 -04:00
QuickandGitHub f5b3cba3cb fix(catalog): use semi-joins for discovery library scope (#88)
* fix(catalog): use semi-joins for discovery library scope

* fix(catalog): require membership for deny-only discovery
2026-06-08 09:33:55 -04:00
zZebrahzandGitHub 96bae2f346 fix(library): clean orphaned provisional items (#76)
* fix(library): clean orphaned provisional items

* test(metadata): use item not found sentinel in fake repo

* fix(library): preserve abs references during orphan cleanup

* fix(library): avoid dropped abs cleanup tables

* fix(library): harden provisional orphan cleanup

* fix(library): preserve home dismissal series orphans

* fix(library): Delete matched items created after the first orphan sweep
2026-06-07 21:32:40 -04:00
978e1b4954 feat(playback): NVENC support for transcoding (#79)
* feat(playback): NVENC support for transcoding

* fix(playback): probe nvenc before auto-selecting

* fix(playback): use safe NVENC smoke probe dimensions

---------

Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-06-07 21:27:56 -04:00
zZebrahzandGitHub c4e67effc3 fix(admin): correct dashboard library counts and scans (#78)
* fix(admin): correct dashboard library counts and scans

* fix(admin): handle mixed library dashboard counts

* fix(admin): refine dashboard count affordances
2026-06-07 19:59:10 -04:00
6b1554a86c Fix scoped catalog refresh invalidation (#51)
* Fix scoped catalog refresh invalidation

* Remove duplicate catalog refetch

* fix(web): keep library lists fresh on scoped catalog events

---------

Co-authored-by: RXWatcher <14085001+RXWatcher@users.noreply.github.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-06-07 18:15:34 -04:00
Quick 480ca44306 fix(recommendations): improve taste seed cold start ranking
Closes #66
2026-06-07 17:42:52 -04:00
d3v1l1989andGitHub ecd85f9ca1 fix(jellycompat): enforce account-level library restrictions (#72)
* fix(jellycompat): enforce account-level library restrictions

The compat AccessFilterFn re-implemented a subset of the access policy
and never loaded users.library_ids, so admin-assigned account library
restrictions did not apply to the Jellyfin-compat API: restricted users
could list, browse, search, and play items from every enabled library.

Replace the hand-rolled closure with the shared access.Resolver via a
new jellycompat.NewScopeAccessFilter adapter so account restrictions,
profile restrictions, user-disabled libraries, and rating/quality
ceilings apply exactly as on the native API. PIN verification is
skipped because compat login already verifies profile PINs.

Also: fail closed (empty allowlist) when scope resolution errors
instead of returning an unrestricted filter; treat an empty allowlist
as restricted-to-nothing in ListUserLibraries instead of unrestricted;
and hide user-disabled libraries from the compat views list.

* test(jellycompat): cover library list filtering via folder source stub

Extract a folderListSource interface for the two FolderRepository
methods directContentService uses (matching the existing browseSource/
itemAccessSource pattern) so ListUserLibraries is unit-testable, and
add coverage for the restricted-allowlist path and the user-disabled
library filtering in the views list.
2026-06-07 17:19:40 -04:00
QuickandGitHub f9bb94a299 [codex] Fix autoscan plugin bindings and poll status (#75)
* fix(autoscan): bind sources by plugin id

* fix(autoscan): skip overlapping source polls
2026-06-07 17:18:55 -04:00
eb6024573e feat(audiobooks): make audiobook libraries first-class catalog items (#73)
* docs(audiobooks): design spec for plugin absorption

Plan to absorb silo-plugin-audiobooks into silo-server as a first-party
feature. Audiobooks land in silo's existing SPA; ABS clients connect
directly. Hard constraints: reuse existing tables (media_items,
media_files, user_watch_progress, user_playback_sessions, people,
item_people, library_collections); only two new tables (abs_sessions,
podcast_feeds) and at most one column add (media_libraries.kind);
silo's main :8080 listener handles ABS Socket.io natively. Out of
scope: audiobook requests flow, smart collections, share links,
external recommender, custom metadata providers, separate audiobook
SPA.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(audiobooks): implementation plan sub-plan 1 (discovery + schema)

First of six sub-plans for the absorption. Six tasks: a discovery
audit that resolves the spec's Risk questions, four idempotent SQL
migrations (abs_sessions, podcast_feeds, media_libraries.kind,
audiobooks.enabled feature flag), and an empty-but-compiling
internal/audiobooks package scaffolded into cmd/silo. Lands as a
strict no-op for users (feature flag defaults to false).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(audiobooks): discovery findings for absorption sub-plan 1

Locks schema/code decisions for migrations 139-142 and downstream
sub-plans. Resolves open Risk questions from the absorption design spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): migration 139 add abs_sessions table

Parallel of jellycompat_sessions for Audiobookshelf-compatible clients.
Lets ABS mobile/desktop apps maintain a device-bound session that
silo's audiobooks/abs handlers will validate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(audiobooks): match codebase conventions in migration 139

Lowercases type keywords in the abs_sessions CREATE TABLE body to
match neighboring migrations, fixes the client_version column
alignment, and replaces the misleading "parallel to
jellycompat_sessions" header comment with a more accurate
description of the table's role.

Cosmetic only — the running schema is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): migration 140 add podcast_feeds table

Side table on media_items for RSS-subscribed podcasts. Holds feed URL,
ETag/Last-Modified for conditional fetches, last-refresh timestamp, and
the per-feed refresh interval consumed by the upcoming
podcastfeed.Refresher scheduled task.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(audiobooks): uppercase PRIMARY KEY in migration 140

Aligns with the codebase convention (type keywords lowercase,
constraint keywords uppercase) established in migration 139's
post-style-fix form. Cosmetic only — running schema is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(audiobooks): migration 141 no-op for media_folders.type

Sub-plan 1 originally reserved migration 141 to add a 'kind' column to
media_libraries discriminating audiobook/podcast libraries. Discovery
audit (sub-plan 1 Task 1) found that the actual table is media_folders
and it already has a type text NOT NULL column with no CHECK constraint
or enum, so 'audiobooks' and 'podcasts' can be added as future values
without DDL.

Landing this migration as a documented no-op preserves the version
numbering audit trail and pins the decision in git history. The
matching down migration is also a no-op.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): migration 142 add audiobooks.enabled flag

Server-settings row that gates the absorbed audiobooks feature.
Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent
sub-plans branch on this flag and operators flip it to 'true' at
cutover.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): scaffold internal/audiobooks package

Empty-but-compiling Service that reads the audiobooks.enabled feature
flag from server_settings. Wired into cmd/silo so the package is
referenced from the binary; no routes mounted, no scheduled tasks
registered, no DB writes. Subsequent sub-plans hang scanner branches,
ABS handlers, Socket.io, podcast refresher, and SPA pages off this
Service.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(audiobooks): cosmetic cleanups in scaffolded package

Two pre-emptive cleanups flagged by code review before sub-plan 2
copies the patterns:

  1. Sort the internal/audiobooks import after internal/adminjob in
     cmd/silo/main.go (alphabetical).
  2. Drop the redundant "audiobooks: " prefix from the Enabled() error
     wrap; matches how every other top-level service package
     (watchstate, scanqueue, metadata, etc.) formats errors.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(audiobooks): implementation plan sub-plan 2 (scanner)

Second of six sub-plans. 10 tasks: PersonKind constants for Author and
Narrator, audio-extension recognizer, library-type helpers, a
walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter
extraction via ffprobe, single-file and multi-file audiobook parsers,
scanner write path producing media_items.type='audiobook', and a
filesystem podcast parser (RSS deferred to sub-plan 5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): add Author and Narrator PersonKind constants

Discovery audit confirmed item_people.kind is unconstrained smallint
with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook
people-links written by the upcoming scanner branches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): add audio-extension recognizer for scanner

Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by
upcoming audiobook and podcast scanner branches to filter directory
walks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): library-type recognizers for scanner dispatch

isAudiobookLibraryType and isPodcastLibraryType match singular and
plural forms case-insensitively, mirroring isMovieLibraryType. Used by
upcoming scanner walk branches (Task 4) that filter audio files into
audiobook and podcast libraries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(scanner): replace movieLibrary bool with typed walkMode

Lets walkLogicalTree dispatch on multiple library shapes (video, movie,
audiobook, podcast) without proliferating boolean flags. Behavior for
existing video and movie libraries is unchanged; audiobook and podcast
modes will be consumed by the upcoming audiobook.go and podcast.go
parsers in later tasks of this sub-plan.

walkModeFor() derives the mode from a media_folders.type string;
unknown types default to walkModeVideo to preserve prior behavior for
any caller still passing a raw type.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): expose ffprobe format tags on ProbeData

The audiobook scanner needs format-level tags (title, artist, album,
date) for media_items metadata; ffprobe already parses them in
ffprobeFormat.Tags but ProbeData previously discarded them. Add
FormatTags map[string]string to ProbeData, populate it in
convertProbeData via a new normalizeFormatTags helper that lowercases
keys and trims values.

Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and
format tags, and a test that verifies ProbeFile() returns both
correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): parser for single-file audiobook folders

parseAudiobookFolder reads tags + chapters via the existing ProbeFile
(now that Task 5 exposes FormatTags on ProbeData) and produces a
parsedAudiobook struct. Title falls back from "title" tag to "album";
author from "artist" -> "album_artist" -> "composer"; series from
"album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools).
Year parsed from "date" or "year" tags, tolerating ISO dates and
parenthesized forms.

Single-file case only; multi-file folders (one audio file per chapter)
return a placeholder error and arrive in Task 7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): multi-file audiobook folder support

Folders containing N audio files (one per chapter/part) get one
parsedAudiobookFile per file; each file's chapter list is synthesized
as a single chapter with title = filename stem. Title/author/series/
year come from the first file's tags.

Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in
favor of the existing firstNonEmpty already in probe.go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): scanner write path produces audiobook media_items

ScanAudiobookFolder walks an audiobooks-typed media folder and treats
each immediate subdirectory as one audiobook. For each parsed audiobook
it upserts:
  - one media_items row with type='audiobook'
  - one media_files row per audio file (with chapters JSONB)
  - author/narrator links in item_people (kind=7, kind=8)

Adds itemRepo and personRepo to the Scanner struct, wired from
fileRepo.Pool() in NewScanner — no constructor signature change needed.

ScanFolder dispatches to this path when folder.Type='audiobooks',
bypassing the per-file movie/TV pipeline because audiobooks are
folder-scoped entities.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): filesystem podcast scanner

ScanPodcastFolder walks a podcasts-typed media folder, treating each
subdirectory as a podcast show and each audio file inside as an
episode. Writes media_items.type='podcast' + episodes rows + media_files
rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5;
this task covers filesystem-only ingestion.

ScanFolder dispatches to this path when folder.Type='podcasts'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(audiobooks): implementation plan sub-plan 5 (podcasts)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): expose audiobooks/podcasts library types in admin UI

Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown
in the admin libraries page so operators can flag a folder as an
audiobook or podcast library. Extends contentLevelsForType() so the
admin UI's downstream filtering treats those types correctly
(audiobook -> ['audiobook'], podcasts -> ['podcast',
'podcast_episode']).

Backend scanner branches for these types were already wired in
sub-plan 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge

origin/main adds 139_media_requests at the same number our local
audiobook branch had used for abs_sessions. Renumber ours to 147 to
free up 139 for the upstream migration. The schema_versions row is
updated in lockstep on the running database so the migrator sees the
abs_sessions migration as already applied at its new version.

Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook
feature flag, abs playback sessions, podcast episode guid, audiobook
series, audiobook title cleanup) stay where they are — they don't
collide with anything on origin/main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge

origin/main added 140_user_permissions at the same version this branch
had used for podcast_feeds. Renumber ours to 157 (next free above the
collections-unify migration at 156) so 140 is free for the upstream
migration. schema_versions on the running database is updated in lockstep
so the migrator sees podcast_feeds as already applied at its new version.

Same pattern as d59c1cb (renumber 139_abs_sessions to 147 for the prior
main merge). Pending migrations after this rename: 132 (downloaded
subtitles admin index, main), 140 (user_permissions, main), and 156
(unify_user_collections, this branch).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(migrations): renumber 141_media_folders_kind_noop to 159 for origin/main merge

Same shape as eb8f67d (the 140→157 renumber from the previous main
merge). origin/main added 141_episode_title_sort_index at the same
version this branch had used for media_folders_kind_noop. Renumber
ours to 159 (next free above the audiobook_series truncate at 158) so
141 is open for the upstream migration. schema_versions on the
running database is updated in lockstep so the migrator sees
media_folders_kind_noop as already applied at its new version.

Pending migrations on silo-prod after this rename: 141
(episode_title_sort_index, main) and any other newer ones from main
that the branch hasn't picked up yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(migrations): renumber 142_audiobooks_feature_flag to 160 for origin/main merge

Companion to 3c6f062's 141 renumber — origin/main also added
142_episode_catalog_entries (alongside 141_episode_title_sort_index)
at a version this branch had used for the audiobooks feature flag.
Renumber ours to 160 so 142 is open for the upstream migration;
schema_versions on silo-prod is updated in lockstep so the migrator
sees audiobooks_feature_flag as already applied at its new version.

This was the only remaining collision (verified by checking for
duplicate version prefixes across migrations/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(audiobooks): address foundation review comments

* fix(audiobooks): tighten scanner identity handling

* fix(audiobooks): propagate scanner cancellation

* chore(audiobooks): adopt goose migration layout

* docs(audiobooks): implementation plan sub-plan 3 (API + frontend MVP)

Third of six sub-plans. 9 tasks: three REST endpoints (list/detail/
progress), TanStack Query hooks + types, three React pages
(Library/Detail/Player), and navigation integration. Scoped to MVP —
author/series indices, smart collections, share links, and other
nice-to-haves from the spec are deferred. Streaming reuses silo's
existing /api/v1/stream/{session_id}; no new transcode code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): list endpoint at GET /api/v1/audiobooks

Paginated list of media_items with type='audiobook' scoped to the
caller's accessible libraries via the existing access filter.
Mirrors silo's existing list-style handlers for movies and series.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): detail endpoint at GET /api/v1/audiobooks/{id}

Returns the media_items row, its media_files (with chapters JSONB),
author/narrator extracted from item_people (kinds 7/8), and the
caller's per-profile listening progress from user_watch_progress.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): progress endpoint at POST /api/v1/audiobooks/{id}/progress

UPSERTs user_watch_progress for the caller's (user_id, profile_id,
content_id). Body carries position_seconds; clients are expected to
post every 5-10s during playback plus on pause/seek (matching silo's
existing video progress cadence).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): frontend types and TanStack Query hooks

TypeScript types match the JSON shapes from the new
/api/v1/audiobooks endpoints (list, detail, progress). Three hooks:
useAudiobookLibrary (list), useAudiobook (detail), and
useReportAudiobookProgress (mutation that invalidates the detail
query on success so progress updates reflect immediately).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): library grid page at /audiobooks

Renders a paginated grid of audiobook cards using the
useAudiobookLibrary hook. Each card links to /audiobooks/book/{id}.
Cards show poster, title, and year; falls back to a "No cover"
placeholder when the audiobook has no poster_url. Empty state hints
to operators that they need to set a library's type to 'audiobooks'.

Routes themselves are wired in Task 8 (navigation integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): detail page with chapter list

Renders cover, title, author, narrator, year, and overview alongside a
chapter list. Clicking a chapter opens an inline sticky
AudiobookPlayer at that chapter's start. A "Resume" button restarts
playback at the saved progress position if present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): HTML5 audio player with chapter navigation

Single-file audiobook playback for MVP. Multi-file queuing arrives in
a follow-up. Streams via the existing /api/v1/direct-download GET
endpoint. Position is reported to /api/v1/audiobooks/{id}/progress
every 10s while playing plus on pause/seek/end. Skip-30s, playback
rate select, chapter list panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(audiobooks): wire navigation and routes

Adds an Audiobooks entry to the sidebar and registers the two new
routes (/audiobooks for the library grid, /audiobooks/book/:id for
detail). The player renders inline inside the detail page; no
dedicated player route is required for MVP.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(audiobooks): address native API review comments

* feat(audiobooks): add ABS compatibility and polish

* fix(audiobooks): stabilize ABS playback progress reporting

* fix(audiobooks): clean up ABS branch review fixes

* chore(audiobooks): adopt goose layout for ABS migrations

* fix(audiobooks): align player seek bar props

* feat(audiobooks): make libraries first-class catalog items

* feat(admin): add server restart endpoint

* fix(audiobooks): address review comment findings

---------

Co-authored-by: RXWatcher <14085001+RXWatcher@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 15:57:05 -04:00
zZebrahzandGitHub c469550fa0 fix(libraries): collapse admin library diagnostics (#61)
* Fix troubleshooting visibility on admin libraries

* Remove metadata matcher badge from admin libraries

* test(libraries): cover hidden matcher queue badge
2026-06-07 12:51:45 -04:00
d3v1l1989andGitHub 3353456d46 fix(jellycompat): aggregate series watch state from episode progress (#70)
A series never has a user_watch_progress row of its own, so series items
always rendered Played:false with no UnplayedItemCount — every show looked
unwatched in compat clients' library views, and the IsPlayed browse filter
treated all series as unplayed. Roll episode progress up to series rows in
the browse/search/detail paths, mirroring applySeasonUserData and the
native API's rollup semantics (WatchedCount/UnplayedCount/InProgressCount).
Progress lookups are chunked at 500 ids (per-user stores may be
SQLite-backed) and the rollup is capped at 250 series per page so
max-limit browses of series-only libraries stay bounded.
2026-06-07 11:42:44 -04:00
d3v1l1989andGitHub 826f0d0b9a fix(jellycompat): serve real MediaSources on Resume/NextUp pages (#69)
Infuse and SenPlayer build Continue Watching rows from the listing's
MediaSources and discard responses that carry only the detail-field stub,
leaving the row empty. Keep the progress scan list-level (the per-entry
GetItemDetail fanout removed in the error-report-2026-05-08 §6 fix stays
removed), but re-map the returned page — bounded by the request limit,
capped at 100 — through the detail path when the client requests
detail-level Fields, matching HandleLatest's existing pattern. The stub
remains as the fallback past the cap and is made honest: playability
flags set, collections serialized empty instead of null.
2026-06-07 11:10:15 -04:00
0b3723c7f2 fix(jellycompat): enforce allow_4k_transcode (#65)
* fix(jellycompat): enforce allow_4k_transcode in playback negotiation and transcode start

The compat layer never read the setting, so clients that couldn't direct
play a 4K file got a full video encode regardless. Suppress
SupportsTranscoding for 4K sources during negotiation and guard the local
and remote transcode starts; audio-only transcodes (video copy) stay
allowed, matching the native handler.

Fixes #64

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(jellycompat): cover settings read error in 4K transcode default-deny

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:47:59 -04:00
181 changed files with 10700 additions and 3210 deletions
+19
View File
@@ -1,6 +1,12 @@
# Silo Docker deployment
# For the default docker compose setup, MEDIA_ROOT is the main value you need to set.
# Optional: layer compose files through env. Leave unset for CPU-only defaults.
# Linux/macOS example for enabling NVIDIA override:
# COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
# Windows example:
# COMPOSE_FILE=docker-compose.yml;docker-compose.nvidia.yml
# Required for docker compose: host path to your media library.
# Example: MEDIA_ROOT=/srv/media
MEDIA_ROOT=/srv/media
@@ -40,6 +46,19 @@ POSTGRES_SHM_SIZE=8gb
# PROXY_PORT=8083
# TRANSCODE_PORT=8082
# Optional NVIDIA override controls (used by docker-compose.nvidia.yml).
# NVIDIA_GPU_COUNT=1
# REQUIRED: master key for at-rest credential encryption. Silo refuses to start
# without it. Integration API keys, S3 keys, and other server-owned secrets are
# encrypted under a key derived from this value, so it must be at least 32
# characters and kept secret. Generate one with:
# openssl rand -base64 48
# Back it up SEPARATELY from your database dumps (treat it like a CA private
# key): losing it makes every encrypted secret unrecoverable. See
# docs/architecture/secret-encryption.md.
# SECRET_KEY=replace-with-output-of-openssl-rand-base64-48
# Run from source / advanced overrides
# Only DATABASE_URL is required when running Silo outside the default docker compose stack.
# DATABASE_URL=postgres://silo:password@localhost:5432/silo
+23
View File
@@ -34,6 +34,29 @@ The easiest way to run Silo is with Docker Compose. The default stack assumes yo
If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing `DATABASE_URL` and `REDIS_URL` instead.
### Optional NVIDIA/NVENC
GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.
Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.
Use the optional override file when you want NVENC:
```sh
docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
```
If you want this controlled from `.env`, set `COMPOSE_FILE`:
```dotenv
COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
NVIDIA_GPU_COUNT=1
```
Windows uses `;` instead of `:` between compose files.
Then `docker compose up -d` will include the NVIDIA override automatically.
4. **Configure through the admin UI**
Add libraries, users, metadata providers, and playback settings from the web interface.
+102 -63
View File
@@ -4,7 +4,6 @@ import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"flag"
"fmt"
"io/fs"
@@ -18,6 +17,7 @@ import (
"sort"
"strconv"
"strings"
"sync/atomic"
"syscall"
"time"
@@ -77,6 +77,7 @@ import (
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/server"
"github.com/Silo-Server/silo-server/internal/subtitles"
@@ -141,7 +142,7 @@ func mustGetSetting(store interface {
func configureOperationalLogging(
ctx context.Context,
pool *pgxpool.Pool,
settingsRepo *catalog.ServerSettingsRepo,
settingsRepo catalog.SettingsStore,
redisCfg config.RedisConfig,
logStreamHub *logstream.Hub,
baseHandler slog.Handler,
@@ -252,6 +253,38 @@ func maybeApplyPostgresTuning(ctx context.Context, pool *pgxpool.Pool, appMaxCon
}
}
// runCredentialBackfills sweeps any plaintext server-owned credential to
// ciphertext on the primary (migration-running) node. All passes are
// best-effort: a failed row leaves the prior plaintext (no new exposure) and
// still reads via the read-path pass-through, so a backfill error must never
// block boot. The sensitive-settings pass runs first so the arr
// resolve-then-encrypt pass sees consistent referenced settings.
func runCredentialBackfills(ctx context.Context, pool *pgxpool.Pool, cipher *secret.Cipher, settings *catalog.EncryptedSettingsRepo) {
settingsN, err := settings.BackfillSensitiveSettings(ctx)
if err != nil {
slog.Error("secret backfill: sensitive settings", "error", err)
}
columnsN, err := secret.BackfillColumns(ctx, pool, cipher, secret.ColumnBackfillTargets())
if err != nil {
slog.Error("secret backfill: credential columns", "error", err)
}
historyServersN, err := historyimport.NewRepository(pool, cipher).BackfillSessionServerSecrets(ctx)
if err != nil {
slog.Error("secret backfill: history import session server credentials", "error", err)
}
// The arr resolver is the encrypting settings decorator: it decrypts a
// sensitive target (e.g. requests.radarr.api_key) or passes through a
// plaintext custom key, exactly replicating the deleted resolveAPIKey.
arrN, err := secret.BackfillReferencedColumns(ctx, pool, cipher, settings.Get, secret.ArrKeyBackfillTargets())
if err != nil {
slog.Error("secret backfill: arr api keys", "error", err)
}
if total := settingsN + columnsN + historyServersN + arrN; total > 0 {
slog.Info("secret backfill: encrypted plaintext credentials at rest",
"settings", settingsN, "columns", columnsN, "history_session_servers", historyServersN, "arr_keys", arrN, "total", total)
}
}
func main() {
envFile := flag.String("env", ".env", "path to .env bootstrap file")
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
@@ -266,6 +299,15 @@ func main() {
log.Fatalf("bootstrap: %v", err)
}
// Construct the at-rest credential cipher from SECRET_KEY immediately after
// bootstrap, before any settings repo is built. It is threaded explicitly as
// a dependency into every repo that stores a server-owned secret — never a
// package-level global.
dataCipher, err := secret.New(bc.SecretKey)
if err != nil {
log.Fatalf("secret cipher: %v", err)
}
// Step 2: Connect to PostgreSQL (bootstrap pool with default max connections)
bootstrapDBCfg := config.DatabaseConfig{URL: bc.DatabaseURL, MaxConnections: 20}
pool, err := database.NewPool(ctx, bootstrapDBCfg)
@@ -314,7 +356,11 @@ func main() {
// Run migrations only for integrated/api modes. Proxy and transcode nodes
// should never alter the schema — they may scale independently and would
// race or apply migrations before the primary node is deliberately upgraded.
if bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == "" {
// The same gate decides whether this node runs the credential-encryption
// backfills: only the primary (migration-running) node sweeps plaintext to
// ciphertext; secondary nodes read whatever the primary encrypted.
isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == ""
if isPrimaryNode {
migCtx, migCancel := context.WithTimeout(ctx, 5*time.Minute)
if migErr := database.RunMigrations(migCtx, pool, migrations.FS, "sql"); migErr != nil {
migCancel()
@@ -324,8 +370,14 @@ func main() {
slog.Info("database migrations applied")
}
// Step 3: Load settings from DB
settingsRepo := catalog.NewServerSettingsRepo(pool)
// Step 3: Load settings from DB. settingsRepo is the encrypting decorator so
// every consumer (config.LoadFromDB, admin, ABS, watchers) transparently sees
// plaintext while sensitive keys rest as ciphertext. The settings backfill
// (run after migrations, before this GetAll) is wired further below.
settingsRepo := catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
if isPrimaryNode {
runCredentialBackfills(ctx, pool, dataCipher, settingsRepo)
}
settings, err := settingsRepo.GetAll(ctx)
if err != nil {
log.Fatalf("loading settings: %v", err)
@@ -396,7 +448,9 @@ func main() {
log.Fatalf("recreating pool with configured max_connections: %v", err)
}
}
settingsRepo = catalog.NewServerSettingsRepo(pool)
// Re-wrap with the encrypting decorator so the recreated pool's settings repo
// still encrypts/decrypts — no raw settings repo may escape into later wiring.
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
nodeID := resolveNodeIdentity()
// Step 9: Validate
@@ -425,6 +479,8 @@ func main() {
appCtx, appCancel := context.WithCancel(ctx)
defer appCancel()
restartReqCh := make(chan struct{}, 1)
var restartRequested atomic.Bool
eventBus := cache.NewEventBus(cfg.Redis.URL)
logStreamHub := logstream.NewHub(nodeID, eventBus)
@@ -458,7 +514,7 @@ func main() {
DatabaseURL: cfg.Database.URL,
JFListen: cfg.JellyfinCompat.Listen,
}
watcher := nodeconfig.NewWatcher(pool, eventBus, bootstrap)
watcher := nodeconfig.NewWatcher(pool, dataCipher, eventBus, bootstrap)
if err := watcher.Start(appCtx); err != nil {
slog.Error("config watcher start failed", "error", err)
os.Exit(1)
@@ -517,6 +573,7 @@ func main() {
BootstrapSensitiveValues: bootstrapSensitiveValues,
AppContext: appCtx,
DB: pool,
SecretCipher: dataCipher,
EventBus: eventBus,
LogStreamHub: logStreamHub,
RealtimeHub: realtimeHub,
@@ -525,14 +582,20 @@ func main() {
OpsLogRepo: opsRepo,
FFmpegLogSink: playback.NewSlogFFmpegLogSink(slog.Default(), nodeID),
PublicURL: os.Getenv("SILO_PUBLIC_URL"),
RequestServerRestart: func(context.Context) error {
if !restartRequested.CompareAndSwap(false, true) {
return handlers.ErrServerRestartAlreadyRequested
}
restartReqCh <- struct{}{}
return nil
},
}
audiobooksService := audiobooks.New(&audiobooksSettingsAdapter{repo: settingsRepo})
audiobooksEnabled, err := audiobooksService.Enabled(appCtx)
absCompatEnabled, err := audiobooksService.ABSCompatEnabled(appCtx)
if err != nil {
slog.Warn("audiobooks feature disabled; failed to read setting", "err", err)
audiobooksEnabled = false
slog.Warn("Audiobookshelf compatibility disabled; failed to read setting", "err", err)
absCompatEnabled = false
}
deps.AudiobooksEnabled = audiobooksEnabled
adminJobCancelRegistry := adminjob.NewCancelRegistry()
deps.AdminJobCancelRegistry = adminJobCancelRegistry
if needsWorkers && deps.DB != nil {
@@ -587,7 +650,7 @@ func main() {
log.Fatalf("register watch provider: %v", err)
}
watchProviderService = watchsync.NewService(
watchsync.NewPostgresRepository(deps.DB),
watchsync.NewPostgresRepository(deps.DB, deps.SecretCipher),
watchProviderRegistry,
)
deps.WatchProviderService = watchProviderService
@@ -1172,7 +1235,7 @@ func main() {
deps.UserStoreProvider = userStoreProvider
}
if watchProviderService != nil {
historyRepo := historyimport.NewRepository(deps.DB)
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, catalog.NewProviderIDRepository(deps.DB))
watchProviderService.
WithMatcher(historyimport.NewMatcher(historyRepo)).
@@ -1435,6 +1498,7 @@ func main() {
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil {
taskMgr.Register(tasks.NewScanLibrariesTask(deps.FolderRepo, deps.LibraryScanQueue, deps.EventBus))
}
taskMgr.Register(tasks.NewCleanupOrphanedMediaItemsTask(catalog.NewOrphanedProvisionalCleaner(deps.DB)))
if deps.IntroAnalyzer != nil {
taskMgr.Register(tasks.NewDetectIntroMarkersTask(deps.IntroAnalyzer, settingsRepo))
}
@@ -1470,14 +1534,13 @@ func main() {
taskMgr.Register(tasks.NewSyncWatchProvidersTask(watchProviderService))
}
requestReconcileSvc := mediarequests.NewService(
mediarequests.NewRepository(deps.DB),
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
nil,
mediarequests.NewCatalogPresence(
catalog.NewItemRepository(deps.DB),
catalog.NewProviderIDRepository(deps.DB),
),
)
requestReconcileSvc.SetSecretResolver(settingsRepo)
requestReconcileSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if userStoreProvider != nil {
reconcileResolver := access.NewResolver(
@@ -1489,13 +1552,15 @@ func main() {
}
taskMgr.Register(tasks.NewReconcileRequestsTask(requestReconcileSvc, 100))
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && pluginService != nil && pluginInstallationStore != nil {
autoscanRepo := autoscan.NewRepository(deps.DB)
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if err := autoscanRepo.MarkInterruptedEvents(appCtx); err != nil {
slog.Warn("autoscan: failed to mark interrupted polls", "err", err)
}
autoscanSvc := api.BuildAutoscanService(
autoscanRepo,
pluginService,
pluginInstallationStore,
mediarequests.NewRepository(deps.DB),
settingsRepo,
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -1519,7 +1584,7 @@ func main() {
taskMgr.Register(tasks.NewRepairProviderIDIntegrityTask(metadata.NewProviderIDIntegrityRepairer(deps.DB), historyReconciler))
taskMgr.Register(tasks.NewReconcileWatchHistoryTask(historyReconciler))
taskMgr.Register(tasks.NewSyncPodcastFeedsTask(podcastfeed.New(), podcastfeed.NewDBStore(deps.DB)))
if audiobooksEnabled && audiobookEnricher != nil {
if audiobookEnricher != nil {
taskMgr.Register(tasks.NewSyncAudiobookMetadataTask(audiobookEnricher))
}
if pluginInstallationStore != nil && pluginRuntimeConfigStore != nil && pluginService != nil {
@@ -1542,7 +1607,7 @@ func main() {
// available. Routes are mounted at the root level by NewRouter (not under
// /api/v1/) so ABS clients resolve /login, /api/*, /abs/api/*, and
// /abs/socket.io/* without path prefix hacks.
if audiobooksEnabled && deps.DB != nil {
if absCompatEnabled && deps.DB != nil {
absUserRepo := auth.NewUserRepository(deps.DB)
absSessionRepo := auth.NewSessionRepository(deps.DB)
absJWTService := auth.NewJWTService(
@@ -1815,6 +1880,7 @@ func main() {
compatDeps := jellycompat.Dependencies{
Config: cfg,
DB: deps.DB,
SecretCipher: dataCipher,
ClientIPResolver: ipResolver,
ProxyPool: deps.ProxyPool,
TranscodePool: deps.TranscodePool,
@@ -1868,7 +1934,7 @@ func main() {
compatDeps.FileResolver = deps.FileRepo
}
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB)
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
// Construct auth service for jellycompat login.
userRepo := auth.NewUserRepository(deps.DB)
@@ -1884,47 +1950,17 @@ func main() {
provider := auth.NewLocalProvider(userRepo, sessionRepo)
compatDeps.AuthService = auth.NewService(provider, jwtService, sessionRepo, userRepo, nil, nil, nil)
// Access filter resolver for profile-scoped library access.
compatDeps.AccessFilterFn = func(ctx context.Context, userID int, profileID string) catalog.AccessFilter {
if userStoreProvider == nil {
return catalog.AccessFilter{}
}
store, err := userStoreProvider.ForUser(ctx, userID)
if err != nil {
return catalog.AccessFilter{}
}
profile, err := store.GetProfile(ctx, profileID)
if err != nil || profile == nil {
return catalog.AccessFilter{}
}
filter := catalog.AccessFilter{
MaxContentRating: profile.MaxContentRating,
}
if profile.LibraryRestrictionsEnabled && len(profile.AllowedLibraryIDs) > 0 {
filter.AllowedLibraryIDs = profile.AllowedLibraryIDs
}
// Apply user-disabled library IDs.
if raw, err := store.GetSetting(ctx, "disabled_library_ids"); err == nil && raw != "" {
var disabled []int
if json.Unmarshal([]byte(raw), &disabled) == nil && len(disabled) > 0 {
if filter.AllowedLibraryIDs != nil {
filtered := make([]int, 0, len(filter.AllowedLibraryIDs))
disSet := make(map[int]struct{}, len(disabled))
for _, id := range disabled {
disSet[id] = struct{}{}
}
for _, id := range filter.AllowedLibraryIDs {
if _, ok := disSet[id]; !ok {
filtered = append(filtered, id)
}
}
filter.AllowedLibraryIDs = filtered
} else {
filter.DisabledLibraryIDs = disabled
}
}
}
return filter
// Access filter resolver for viewer-scoped library access.
// Backed by the shared access.Resolver so account-level library
// restrictions (users.library_ids), profile restrictions,
// user-disabled libraries, and rating/quality ceilings apply to
// the compat API exactly as they do to the native API.
if userStoreProvider != nil {
compatDeps.AccessFilterFn = jellycompat.NewScopeAccessFilter(access.NewResolver(
userRepo,
userStoreProvider,
nil, // profile tokens unused: compat login already verifies PINs
))
}
}
@@ -2016,6 +2052,9 @@ func main() {
case sig := <-sigCh:
appCancel()
slog.Info("received signal, shutting down", "signal", sig)
case <-restartReqCh:
appCancel()
slog.Info("server restart requested, shutting down")
case serverErr := <-errCh:
appCancel()
slog.Error("server error, shutting down", "error", serverErr)
@@ -2395,7 +2434,7 @@ func mapFolderTypeToMediaType(t string) string {
// exposes Get) to the audiobooks.SettingsReader interface (which
// requires GetString). The two signatures are identical modulo name.
type audiobooksSettingsAdapter struct {
repo *catalog.ServerSettingsRepo
repo catalog.SettingsStore
}
func (a *audiobooksSettingsAdapter) GetString(ctx context.Context, key string) (string, error) {
+20
View File
@@ -0,0 +1,20 @@
services:
silo:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: ${NVIDIA_GPU_COUNT:-1}
capabilities: [gpu]
# Optional distributed-mode example. Most single-host installs should
# just use the integrated service above.
# silo-transcode:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: ${NVIDIA_GPU_COUNT:-1}
# capabilities: [gpu]
+10 -1
View File
@@ -36,6 +36,11 @@ services:
restart: unless-stopped
environment:
MODE: integrated
# Master key for at-rest credential encryption. Required — the server
# refuses to start without it. Generate with: openssl rand -base64 48
# Back it up SEPARATELY from your database dumps; losing it makes encrypted
# secrets unrecoverable. See docs/architecture/secret-encryption.md.
SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
REDIS_URL: redis://redis:6379
SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
@@ -46,7 +51,7 @@ services:
- "${ABS_PORT:-13378}:13378"
volumes:
- ${MEDIA_ROOT:?Set MEDIA_ROOT in .env to the host media path}:${MEDIA_CONTAINER_ROOT:-/mnt/media}:ro
- ${MEDIA_BOOKS_ROOT:-${MEDIA_ROOT}}:${MEDIA_BOOKS_CONTAINER_ROOT:-${MEDIA_CONTAINER_ROOT:-/mnt/media}}:ro
- ${MEDIA_BOOKS_ROOT:-${MEDIA_ROOT}}:${MEDIA_BOOKS_CONTAINER_ROOT:-${MEDIA_CONTAINER_ROOT:-/mnt/media}/books}:ro
- ${SILO_DATA_ROOT:-/opt/silo}/plugins:/var/lib/silo/plugins
- ${SILO_DATA_ROOT:-/opt/silo}/transcode:/tmp/silo-transcode
- ${SILO_DATA_ROOT:-/opt/silo}/catalog-seeds:/catalog-seeds:ro
@@ -71,6 +76,8 @@ services:
# profiles: [proxy]
# environment:
# MODE: proxy
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# PORT: "8080"
@@ -95,6 +102,8 @@ services:
# profiles: [transcode]
# environment:
# MODE: transcode
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
+142
View File
@@ -0,0 +1,142 @@
# At-rest credential encryption
Silo encrypts third-party integration credentials and other server-owned secrets
at rest. Sonarr/Radarr API keys, S3 keys, watch-sync OAuth tokens, the
Audiobookshelf-compat signing key, and the sensitive `server_settings` entries
are stored as ciphertext in PostgreSQL and decrypted only in memory. The key that
protects them lives **outside** the database, so a full DB dump or compromise
does not expose the secrets.
## How it works
- **Cipher:** AES-256-GCM with a random 12-byte nonce per value. Implemented in
`internal/secret`.
- **Key derivation:** the data key is HKDF-SHA256–derived from the `SECRET_KEY`
environment variable with a versioned domain label. `SECRET_KEY` is the
encryption root; it is never stored in the database.
- **Envelope:** every ciphertext is stored as `enc:v1:<base64url(nonce‖sealed)>`.
The `enc:v1:` prefix is the version marker (see *Key rotation* below).
- **Row binding (AAD):** each ciphertext is GCM-bound to its logical row via
additional-authenticated data — `table:column:<pk>` for per-row columns,
`server_settings:<key>` for settings. A DB-write attacker therefore cannot move
a credential blob from one row/column to another: decrypting under a different
binding fails authentication.
- **Read path:** an empty value stays empty; a non-`enc:v1:` value is treated as
legacy plaintext and passed through unchanged (so a not-yet-migrated row keeps
working); an `enc:v1:` value is decrypted and **any** failure (wrong key,
tampering, truncation) is surfaced as an error — it is never silently used as a
credential.
## SECRET_KEY
`SECRET_KEY` is **required**. The server refuses to start without it (it fatals
in `config.LoadBootstrap`), and it must be at least 32 characters.
Generate one with:
```bash
openssl rand -base64 48
```
Set it in the deployment environment (or in `.env` for source/dev runs, which
`godotenv` loads at startup). See `.env.example`.
### Back it up — separately from the database
Treat `SECRET_KEY` like a CA private key:
- **Store it outside your database backups.** A DB dump contains only ciphertext;
the value of at-rest encryption is entirely lost if the key travels with the
dump.
- Keep it in a secrets manager / sealed secret / password manager, not in the
repo or in the same bucket as your Postgres backups.
- Rotating the *deployment* (new host, restored DB) requires the **same**
`SECRET_KEY`. Restoring a database backup onto a node with a different
`SECRET_KEY` makes every encrypted secret unreadable.
### Key loss
If `SECRET_KEY` is lost, the encrypted secrets are **unrecoverable** — that is the
point of keeping the key out of the database. Recovery means re-entering the
affected credentials:
- Re-enter Sonarr/Radarr and Autoscan API keys, S3 keys, watch-sync connections,
history-import tokens, and subtitle provider credentials.
- `auth.jwt_secret` becomes unreadable, so all existing sessions are invalid and
users must log in again (a new secret is generated on next boot if the row is
cleared — see below).
## Rollout (automatic backfill)
On the first boot after deploying this change, the primary (migration-running)
node runs an idempotent, best-effort startup backfill that encrypts any
remaining plaintext in place:
1. sensitive `server_settings` keys,
2. the per-table credential columns (subtitles, watch-sync, webhook-sync,
history-import, including temporary server-list credentials stored in
session JSON),
3. the two arr `api_key_ref` columns — these are **resolved-then-encrypted**: a
legacy row that held a `server_settings` reference (e.g.
`requests.radarr.api_key`) is collapsed to the real credential before being
encrypted.
The backfill is safe to run repeatedly: already-encrypted values are skipped, and
a per-row guard makes concurrent multi-node boots converge without
double-encrypting. A failed row is logged and left as plaintext (no new exposure)
rather than blocking boot. Proxy/transcode nodes skip the backfill; they read
whatever the primary encrypted. No manual steps are required.
## Rollback / downgrade hazard
Downgrading to a binary that predates this change is **not** safe while secrets
are encrypted, because the old binary has no read path: it would read
`enc:v1:auth.jwt_secret` as a literal JWT secret (invalidating all sessions) and
read `enc:v1:`-prefixed integration keys as garbage credentials.
To downgrade safely you must first return the affected values to plaintext, for
example:
- Decrypt the sensitive `server_settings` back to plaintext (run a one-off that
reads each via the encrypting repo and writes the plaintext via the raw repo),
**or**
- At minimum, clear `auth.jwt_secret` so the older binary regenerates a fresh
plaintext one (this still logs everyone out), and re-enter any integration
credentials as plaintext.
There is no automatic "decrypt everything" downgrade path in this release; plan
downgrades accordingly.
## Key rotation (future)
The `enc:v1:` envelope and the HKDF domain label are versioned. A future rotation
would introduce `enc:v2:` with a new derived key (and `Decrypt` already dispatches
on the version, returning an explicit error for an unknown version). Rotation is
out of scope for this release; the versioning exists so it can be added without a
data migration of the envelope format.
## Scope and known gaps
Covered: arr (Requests + Autoscan) API keys, S3 keys, all sensitive
`server_settings`, watch-sync tokens, webhook-sync `access_token`,
history-import admin/session tokens and temporary server-list credentials,
subtitle provider `api_key`/`password`,
the Jellyfin-compat session's bridged Silo access/refresh tokens
(`jellycompat_sessions.streamapp_access_token` / `streamapp_refresh_token`), and
the ABS signing key.
Deliberately **not** encrypted (tracked as follow-ups):
- **Equality-looked-up secrets** — these are matched by exact value
(`WHERE … = $1`), and AES-GCM is randomized, so encrypting them would break the
lookup. They need a deterministic **blind-index hash** column instead:
`api_keys.api_key`, `webhook_sync_connections.webhook_secret`,
`jellycompat_sessions.token`, and `watch_together_rooms.join_token`.
- **Plugin runtime config** — `plugin_runtime_configs.config_value` is opaque
plugin-defined JSONB whose secret fields are manifest-declared and whose runtime
lives in a separate repo; encrypting it correctly needs a coordinated,
manifest-aware design.
Excluded (not a gap): `plex_sync_connections.*` is a dead table (zero Go
references); `oauth_completion.token_ciphertext` is already AES-GCM;
`users.password_hash` and the `*_hash` columns are already hashed.
+145
View File
@@ -0,0 +1,145 @@
# First-Party Audio API
This API is the Silo-native contract for audiobook playback. Clients should use
it for Silo first-party audio experiences instead of the Audiobookshelf
compatibility API. Music can reuse the same session, track, and progress
concepts when the music UX lands.
## Authentication
All endpoints require normal Silo user authentication. Playback start, sync,
close, and bookmark writes also require the active profile headers used by the
first-party clients.
Track stream URLs are returned from the start response and are scoped to the
created audio session. The server validates that the session still belongs to
the authenticated user, and to the active profile when a profile is attached.
## Start Playback
`POST /api/v1/audio/playback/start`
Request:
```json
{
"content_id": "128814341867700228",
"start_position": 120,
"restart": false
}
```
Response:
```json
{
"session_id": "uuid",
"content_id": "128814341867700228",
"type": "audiobook",
"title": "Book Title",
"subtitle": "Author or series subtitle",
"poster_url": "https://server/api/v1/images/...",
"backdrop_url": "https://server/api/v1/images/...",
"total_duration_seconds": 8746,
"resume_position_seconds": 120,
"tracks": [
{
"index": 0,
"file_id": 123,
"file_name": "Part 1.mp3",
"duration_seconds": 3600,
"start_offset_seconds": 0,
"stream_url": "/api/v1/audio/playback/{session_id}/tracks/0",
"stream_type": "progressive",
"play_method": "direct",
"codec": "mp3",
"container": "mp3",
"bitrate": 128000,
"chapters": []
}
],
"chapters": [
{
"index": 0,
"title": "Chapter 1",
"start_seconds": 0,
"end_seconds": 1800,
"track_index": 0
}
],
"bookmarks": []
}
```
`total_duration_seconds`, `resume_position_seconds`, global chapter times, and
sync positions are cumulative book seconds, not per-file seconds. Clients map
global time to the active track by finding the last track whose
`start_offset_seconds` is less than or equal to the target time.
Tracks are ordered by presentation part index when available, then by path and
file id. The server direct-plays AVFoundation-safe formats and returns
`play_method: "aac"` for fallback streams that are remuxed/transcoded to an
Apple-safe AAC container.
## Stream Track
`GET /api/v1/audio/playback/{session_id}/tracks/{track_index}`
The response is the media stream for the selected track. Clients should request
the track for the current global time and seek within that track using:
`local_seconds = global_seconds - track.start_offset_seconds`
## Sync Progress
`PATCH /api/v1/audio/playback/{session_id}/sync`
Request:
```json
{
"position": 1234,
"duration": 8746,
"is_paused": false
}
```
The server persists `position` as the audiobook progress position. The client
should send cumulative book seconds on a throttle while playing, on manual
seeks, and once more before closing the session.
## Close Playback
`POST /api/v1/audio/playback/{session_id}/close`
Request:
```json
{
"position": 1234,
"duration": 8746,
"is_paused": true
}
```
Close performs a final progress sync and expires the in-memory audio session.
## Bookmarks
`GET /api/v1/audio/items/{content_id}/bookmarks`
`POST /api/v1/audio/items/{content_id}/bookmarks`
`DELETE /api/v1/audio/items/{content_id}/bookmarks/{time_seconds}`
Create request:
```json
{
"time_seconds": 1234,
"title": "Chapter note"
}
```
Bookmark positions are cumulative book seconds so they remain stable across
multi-file audiobooks and client platforms.
+11 -25
View File
@@ -20,12 +20,13 @@ type autoscanQueuer = autoscan.Queuer
// RequestIntegrationLookup adapts the Requests repository to the autoscan
// connection resolver's RequestIntegrationLookup: it resolves a soft-linked
// Requests integration to its base URL and (write-only) api-key ref.
// Requests integration to its base URL and api key (the repo decrypts the key
// on read, so this returns plaintext).
type RequestIntegrationLookup struct {
Repo *mediarequests.Repository
}
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error) {
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error) {
integration, err := l.Repo.GetIntegration(ctx, integrationID)
if err != nil {
return "", "", err
@@ -64,8 +65,8 @@ type PluginScanSourceAdapter struct {
Svc *plugins.Service
}
func (a PluginScanSourceAdapter) ScanSourceClient(ctx context.Context, installationID int, capabilityID string) (autoscan.PollChangesClient, error) {
return a.Svc.ScanSourceClient(ctx, installationID, capabilityID)
func (a PluginScanSourceAdapter) ScanSourceClient(ctx context.Context, pluginID, capabilityID string) (autoscan.PollChangesClient, error) {
return a.Svc.ScanSourceClientByPluginID(ctx, pluginID, capabilityID)
}
// scanSourceCapabilityType is the plugin capability type autoscan discovery
@@ -75,15 +76,6 @@ const scanSourceCapabilityType = "scan_source.v1"
// PluginScanSourceLister adapts the plugin installation store to
// autoscan.ScanSourceLister: it enumerates every installed scan_source.v1
// capability across ALL installed plugins, regardless of enabled state.
//
// Using List (not ListEnabled) is deliberate for orphan detection: a temporarily
// DISABLED-but-installed plugin must NOT be treated as orphaned. If we dropped
// disabled plugins here, PollOnce would prune their sources and skip them with no
// last_error — they'd vanish silently. Keeping them present means PollOnce still
// attempts them; the plugin client fails to load (not running) and the source
// gets a visible RecordError instead. Only a fully UNINSTALLED plugin (gone from
// List entirely) is a true orphan. The Add-source picker shares this all-installed
// set, which is fine — operators can bind sources against an installed capability.
type PluginScanSourceLister struct {
Store *plugins.InstallationStore
}
@@ -104,10 +96,9 @@ func (l PluginScanSourceLister) ListScanSources(ctx context.Context) ([]autoscan
continue
}
out = append(out, autoscan.DiscoveredSource{
InstallationID: c.InstallationID,
CapabilityID: c.ID,
PluginID: inst.PluginID,
DisplayName: scanSourceDisplayName(inst.PluginID, c),
PluginID: inst.PluginID,
CapabilityID: c.ID,
DisplayName: scanSourceDisplayName(inst.PluginID, c),
})
}
}
@@ -135,27 +126,22 @@ func scanSourceDisplayName(pluginID string, c *plugins.Capability) string {
}
}
// AutoscanSecretResolver resolves an encrypted api-key reference to plaintext.
// It is satisfied by the server settings repo (Get(ctx, key) (string, error)).
type AutoscanSecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
}
// BuildAutoscanService wires the v2 autoscan engine from its concrete
// dependencies. Both the HTTP router (manual trigger) and the background poll
// task share this constructor so the adapter wiring lives in exactly one place.
// Credentials are now decrypted inline by the autoscan/requests repos, so there
// is no separate secret resolver to thread.
func BuildAutoscanService(
repo *autoscan.Repository,
pluginService *plugins.Service,
installationStore *plugins.InstallationStore,
requestsRepo *mediarequests.Repository,
secrets AutoscanSecretResolver,
folderRepo *catalog.FolderRepository,
queue autoscanQueuer,
redisClient *redis.Client,
) *autoscan.Service {
provider := autoscan.NewPluginProvider(PluginScanSourceAdapter{pluginService})
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo}, secrets)
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo})
svc := autoscan.NewService(
repo,
provider,
+4 -23
View File
@@ -1137,29 +1137,10 @@ func (h *AdminHandler) HandleUpdateItemMetadata(w http.ResponseWriter, r *http.R
// --- Server Settings endpoints ---
var sensitiveSettingKeys = map[string]bool{
"auth.jwt_secret": true,
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"redis.url": true,
"recommendations.openai_api_key": true,
"recommendations.embedding_auth_token": true,
"tmdb.api_key": true,
"introdb.api_key": true,
"mdblist.api_key": true,
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
"subtitle_ai.api_key": true,
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
}
// sensitiveSettingKeys is the audited allowlist of secret-bearing settings
// keys, shared with the at-rest encryption decorator so redaction and
// encryption can never drift apart. See catalog.SensitiveSettingKeys.
var sensitiveSettingKeys = catalog.SensitiveSettingKeys
// HandleGetSettings handles GET /admin/settings.
func (h *AdminHandler) HandleGetSettings(w http.ResponseWriter, r *http.Request) {
+44 -8
View File
@@ -22,7 +22,9 @@ type AdminStats struct {
TotalFiles int `json:"total_files"`
TotalUsers int `json:"total_users"`
TotalMovies int `json:"total_movies"`
TotalMovieFiles int `json:"total_movie_files"`
TotalShows int `json:"total_shows"`
TotalShowFiles int `json:"total_show_files"`
ActiveStreams int `json:"active_streams"`
TotalStorageBytes int64 `json:"total_storage_bytes"`
WatchProviderActivity WatchProviderActivity `json:"watch_provider_activity"`
@@ -127,13 +129,15 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
}
var (
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalShows int64
activeStreams int64
totalStorage int64
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalMovieFiles int64
totalShows int64
totalShowFiles int64
activeStreams int64
totalStorage int64
)
row := pool.QueryRow(ctx, `
@@ -151,8 +155,34 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
file_stats AS (
SELECT
COUNT(*)::bigint AS total_files,
COUNT(*) FILTER (WHERE file_kind = 'movie')::bigint AS total_movie_files,
COUNT(*) FILTER (WHERE file_kind = 'series')::bigint AS total_show_files,
COALESCE(SUM(file_size), 0)::bigint AS total_storage_bytes
FROM media_files
FROM (
SELECT
media_files.file_size,
CASE
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'movie'
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'series'
THEN 'series'
WHEN episodes.content_id IS NOT NULL
THEN 'series'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
WHEN lower(trim(media_folders.type)) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(media_folders.type)) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
ELSE ''
END AS file_kind
FROM media_files
JOIN media_folders ON media_folders.id = media_files.media_folder_id
LEFT JOIN media_items ON media_items.content_id = media_files.content_id
LEFT JOIN episodes ON episodes.content_id = media_files.episode_id
) classified_files
),
session_stats AS (
SELECT COUNT(*)::bigint AS active_streams
@@ -163,7 +193,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
item_stats.total_items,
file_stats.total_files,
item_stats.total_movies,
file_stats.total_movie_files,
item_stats.total_shows,
file_stats.total_show_files,
session_stats.active_streams,
file_stats.total_storage_bytes
FROM user_stats
@@ -176,7 +208,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
&totalItems,
&totalFiles,
&totalMovies,
&totalMovieFiles,
&totalShows,
&totalShowFiles,
&activeStreams,
&totalStorage,
); err != nil {
@@ -194,7 +228,9 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
TotalItems: int(totalItems),
TotalFiles: int(totalFiles),
TotalMovies: int(totalMovies),
TotalMovieFiles: int(totalMovieFiles),
TotalShows: int(totalShows),
TotalShowFiles: int(totalShowFiles),
ActiveStreams: int(activeStreams),
TotalStorageBytes: totalStorage,
WatchProviderActivity: activity,
+757
View File
@@ -0,0 +1,757 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"fmt"
"math"
"net/http"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/oklog/ulid/v2"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/userstore"
)
const audioPlaybackSessionTTL = 12 * time.Hour
type AudioDetailService interface {
GetItemDetail(ctx context.Context, contentID string, filter catalog.AccessFilter) (*catalog.ItemDetail, error)
}
type AudioFileResolver interface {
GetByContentID(ctx context.Context, contentID string) ([]*models.MediaFile, error)
GetByID(ctx context.Context, id int) (*models.MediaFile, error)
}
type AudioHandler struct {
detailSvc AudioDetailService
files AudioFileResolver
storeProvider userstore.UserStoreProvider
bookmarkStore *PGAudioBookmarkStore
mu sync.Mutex
sessions map[string]audioPlaybackSession
now func() time.Time
}
func NewAudioHandler(
detailSvc AudioDetailService,
files AudioFileResolver,
storeProvider userstore.UserStoreProvider,
bookmarkStore *PGAudioBookmarkStore,
) *AudioHandler {
return &AudioHandler{
detailSvc: detailSvc,
files: files,
storeProvider: storeProvider,
bookmarkStore: bookmarkStore,
sessions: make(map[string]audioPlaybackSession),
now: func() time.Time { return time.Now().UTC() },
}
}
type audioPlaybackSession struct {
ID string
UserID int
ProfileID string
ContentID string
TotalSeconds float64
Files []audioSessionFile
CreatedAt time.Time
LastSeenAt time.Time
ExpiresAt time.Time
}
type audioSessionFile struct {
FileID int
FilePath string
Start float64
Duration float64
Mode string
TrackIdx int
}
type audioStartPlaybackRequest struct {
ContentID string `json:"content_id"`
StartPosition *float64 `json:"start_position,omitempty"`
Restart bool `json:"restart,omitempty"`
}
type audioProgressRequest struct {
Position float64 `json:"position"`
Duration float64 `json:"duration,omitempty"`
IsPaused bool `json:"is_paused"`
}
type audioPlaybackResponse struct {
SessionID string `json:"session_id"`
ContentID string `json:"content_id"`
Type string `json:"type"`
Title string `json:"title"`
Subtitle string `json:"subtitle,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
BackdropURL string `json:"backdrop_url,omitempty"`
TotalDurationSeconds float64 `json:"total_duration_seconds"`
ResumePositionSeconds float64 `json:"resume_position_seconds"`
Tracks []audioTrackResponse `json:"tracks"`
Chapters []audioChapter `json:"chapters"`
Bookmarks []audioBookmark `json:"bookmarks"`
}
type audioTrackResponse struct {
Index int `json:"index"`
FileID int `json:"file_id"`
FileName string `json:"file_name,omitempty"`
DurationSeconds float64 `json:"duration_seconds"`
StartOffsetSeconds float64 `json:"start_offset_seconds"`
StreamURL string `json:"stream_url"`
StreamType string `json:"stream_type"`
PlayMethod string `json:"play_method"`
Codec string `json:"codec,omitempty"`
Container string `json:"container,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
Chapters []audioChapter `json:"chapters"`
}
type audioChapter struct {
Index int `json:"index"`
Title string `json:"title,omitempty"`
StartSeconds float64 `json:"start_seconds"`
EndSeconds float64 `json:"end_seconds,omitempty"`
TrackIndex int `json:"track_index,omitempty"`
}
type audioBookmark struct {
ID string `json:"id"`
ContentID string `json:"content_id"`
TimeSeconds float64 `json:"time_seconds"`
Title string `json:"title"`
CreatedAt string `json:"created_at,omitempty"`
UpdatedAt string `json:"updated_at,omitempty"`
}
type audioBookmarkRequest struct {
TimeSeconds *float64 `json:"time_seconds"`
Title string `json:"title"`
}
func (h *AudioHandler) HandleStartPlayback(w http.ResponseWriter, r *http.Request) {
if h == nil || h.detailSvc == nil || h.files == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Audio playback is not configured")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if userID == 0 || profileID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Profile is required")
return
}
var req audioStartPlaybackRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
req.ContentID = strings.TrimSpace(req.ContentID)
if req.ContentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return
}
filter := requestAccessFilter(r)
detail, err := h.detailSvc.GetItemDetail(r.Context(), req.ContentID, filter)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load item detail")
return
}
if detail == nil || detail.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "Audiobook not found")
return
}
files, err := h.files.GetByContentID(r.Context(), req.ContentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load audio files")
return
}
files = catalog.FilterMediaFilesByAccess(files, filter)
sortAudioFiles(files)
if len(files) == 0 {
writeError(w, http.StatusNotFound, "not_found", "No playable audio files found")
return
}
sessionID := ulid.Make().String()
tracks, sessionFiles, chapters, totalDuration := buildAudioTrackResponses(sessionID, files)
if detail.Audiobook != nil && detail.Audiobook.TotalDurationSeconds > 0 {
totalDuration = float64(detail.Audiobook.TotalDurationSeconds)
}
resume := resolveAudioResumePosition(r.Context(), h.storeProvider, userID, profileID, req.ContentID)
if req.Restart {
resume = 0
} else if req.StartPosition != nil && isFiniteNonNegative(*req.StartPosition) {
resume = clampFloat(*req.StartPosition, 0, totalDuration)
}
bookmarks, _ := h.listBookmarks(r.Context(), userID, profileID, req.ContentID)
session := audioPlaybackSession{
ID: sessionID,
UserID: userID,
ProfileID: profileID,
ContentID: req.ContentID,
TotalSeconds: totalDuration,
Files: sessionFiles,
CreatedAt: h.now(),
LastSeenAt: h.now(),
ExpiresAt: h.now().Add(audioPlaybackSessionTTL),
}
h.putSession(session)
writeJSON(w, http.StatusOK, audioPlaybackResponse{
SessionID: sessionID,
ContentID: detail.ContentID,
Type: detail.Type,
Title: detail.Title,
Subtitle: audioSubtitle(detail),
PosterURL: detail.PosterURL,
BackdropURL: detail.BackdropURL,
TotalDurationSeconds: totalDuration,
ResumePositionSeconds: resume,
Tracks: tracks,
Chapters: chapters,
Bookmarks: bookmarks,
})
}
func (h *AudioHandler) HandleSyncPlayback(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
var req audioProgressRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
duration := req.Duration
if duration <= 0 {
duration = session.TotalSeconds
}
position := clampFloat(req.Position, 0, duration)
if err := h.persistProgress(r.Context(), session.UserID, session.ProfileID, session.ContentID, position, duration); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync audio progress")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"session_id": session.ID,
"position": position,
"duration": duration,
"is_paused": req.IsPaused,
})
}
func (h *AudioHandler) HandleClosePlayback(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
var req audioProgressRequest
_ = json.NewDecoder(r.Body).Decode(&req)
if isFiniteNonNegative(req.Position) {
duration := req.Duration
if duration <= 0 {
duration = session.TotalSeconds
}
position := clampFloat(req.Position, 0, duration)
_ = h.persistProgress(r.Context(), session.UserID, session.ProfileID, session.ContentID, position, duration)
}
h.deleteSession(session.ID)
w.WriteHeader(http.StatusNoContent)
}
func (h *AudioHandler) HandleStreamTrack(w http.ResponseWriter, r *http.Request) {
session, ok := h.sessionForRequest(w, r)
if !ok {
return
}
rawIndex := chi.URLParam(r, "track_index")
index, err := strconv.Atoi(rawIndex)
if err != nil || index < 0 || index >= len(session.Files) {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid track index")
return
}
file := session.Files[index]
mode := r.URL.Query().Get("mode")
if mode == "" {
mode = file.Mode
}
if mode == "aac" {
seek := 0.0
if raw := r.URL.Query().Get("seek"); raw != "" {
if parsed, err := strconv.ParseFloat(raw, 64); err == nil && parsed >= 0 {
seek = parsed
}
}
if err := playback.ServeRemux(w, r, file.FilePath, "mp4", seek, true, 0); err != nil {
return
}
return
}
_ = playback.ServeDirectPlay(w, r, file.FilePath)
}
func (h *AudioHandler) HandleListBookmarks(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
bookmarks, err := h.listBookmarks(r.Context(), userID, profileID, contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list bookmarks")
return
}
writeJSON(w, http.StatusOK, map[string]any{"bookmarks": bookmarks})
}
func (h *AudioHandler) HandleCreateBookmark(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
if h.bookmarkStore == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Bookmarks are not configured")
return
}
var req audioBookmarkRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
if req.TimeSeconds == nil || !isFiniteNonNegative(*req.TimeSeconds) {
writeError(w, http.StatusBadRequest, "bad_request", "time_seconds is required")
return
}
filter := requestAccessFilter(r)
detail, err := h.detailSvc.GetItemDetail(r.Context(), contentID, filter)
if err != nil || detail == nil || detail.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "Audiobook not found")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
bookmark, err := h.bookmarkStore.Upsert(r.Context(), userID, profileID, contentID, *req.TimeSeconds, req.Title)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save bookmark")
return
}
writeJSON(w, http.StatusOK, bookmark)
}
func (h *AudioHandler) HandleDeleteBookmark(w http.ResponseWriter, r *http.Request) {
contentID, ok := h.contentIDParam(w, r)
if !ok {
return
}
if h.bookmarkStore == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Bookmarks are not configured")
return
}
rawTime := chi.URLParam(r, "time_seconds")
timeSeconds, err := strconv.ParseFloat(rawTime, 64)
if err != nil || !isFiniteNonNegative(timeSeconds) {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid bookmark time")
return
}
if err := h.bookmarkStore.Delete(r.Context(), apimw.GetUserID(r.Context()), apimw.GetProfileID(r.Context()), contentID, timeSeconds); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to delete bookmark")
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *AudioHandler) contentIDParam(w http.ResponseWriter, r *http.Request) (string, bool) {
contentID := strings.TrimSpace(chi.URLParam(r, "content_id"))
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return "", false
}
return contentID, true
}
func (h *AudioHandler) sessionForRequest(w http.ResponseWriter, r *http.Request) (audioPlaybackSession, bool) {
sessionID := chi.URLParam(r, "session_id")
if sessionID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Session ID is required")
return audioPlaybackSession{}, false
}
session, ok := h.getSession(sessionID)
if !ok {
writeError(w, http.StatusNotFound, "not_found", "Audio playback session not found")
return audioPlaybackSession{}, false
}
if session.UserID != apimw.GetUserID(r.Context()) {
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
return audioPlaybackSession{}, false
}
profileID := apimw.GetProfileID(r.Context())
if profileID != "" && session.ProfileID != profileID {
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another profile")
return audioPlaybackSession{}, false
}
return session, true
}
func (h *AudioHandler) putSession(session audioPlaybackSession) {
h.mu.Lock()
defer h.mu.Unlock()
h.pruneExpiredLocked(h.now())
h.sessions[session.ID] = session
}
func (h *AudioHandler) getSession(sessionID string) (audioPlaybackSession, bool) {
h.mu.Lock()
defer h.mu.Unlock()
now := h.now()
h.pruneExpiredLocked(now)
session, ok := h.sessions[sessionID]
if !ok {
return audioPlaybackSession{}, false
}
if now.After(session.ExpiresAt) {
delete(h.sessions, sessionID)
return audioPlaybackSession{}, false
}
session.LastSeenAt = now
h.sessions[sessionID] = session
return session, true
}
func (h *AudioHandler) deleteSession(sessionID string) {
h.mu.Lock()
defer h.mu.Unlock()
delete(h.sessions, sessionID)
}
func (h *AudioHandler) pruneExpiredLocked(now time.Time) {
for id, session := range h.sessions {
if now.After(session.ExpiresAt) {
delete(h.sessions, id)
}
}
}
func (h *AudioHandler) persistProgress(ctx context.Context, userID int, profileID, contentID string, position, duration float64) error {
if h.storeProvider == nil {
return nil
}
store, err := h.storeProvider.ForUser(ctx, userID)
if err != nil {
return err
}
return store.SetProgress(ctx, profileID, contentID, position, duration, userstore.ProgressThresholds{})
}
func (h *AudioHandler) listBookmarks(ctx context.Context, userID int, profileID, contentID string) ([]audioBookmark, error) {
if h.bookmarkStore == nil {
return []audioBookmark{}, nil
}
return h.bookmarkStore.List(ctx, userID, profileID, contentID)
}
func buildAudioTrackResponses(sessionID string, files []*models.MediaFile) ([]audioTrackResponse, []audioSessionFile, []audioChapter, float64) {
tracks := make([]audioTrackResponse, 0, len(files))
sessionFiles := make([]audioSessionFile, 0, len(files))
allChapters := []audioChapter{}
offset := 0.0
for index, file := range files {
duration := float64(file.Duration)
if duration < 0 {
duration = 0
}
mode := audioDeliveryMode(file)
streamURL := fmt.Sprintf("/api/v1/audio/playback/%s/tracks/%d", sessionID, index)
if mode == "aac" {
streamURL += "?mode=aac"
}
chapters := audioChaptersForTrack(index, offset, file.Chapters)
allChapters = append(allChapters, chapters...)
tracks = append(tracks, audioTrackResponse{
Index: index,
FileID: file.ID,
FileName: filepath.Base(file.FilePath),
DurationSeconds: duration,
StartOffsetSeconds: offset,
StreamURL: streamURL,
StreamType: "progressive",
PlayMethod: mode,
Codec: file.CodecAudio,
Container: file.Container,
Bitrate: file.Bitrate,
Chapters: chapters,
})
sessionFiles = append(sessionFiles, audioSessionFile{
FileID: file.ID,
FilePath: file.FilePath,
Start: offset,
Duration: duration,
Mode: mode,
TrackIdx: index,
})
offset += duration
}
return tracks, sessionFiles, allChapters, offset
}
func audioChaptersForTrack(trackIndex int, offset float64, chapters []models.MediaChapter) []audioChapter {
out := make([]audioChapter, 0, len(chapters))
for _, chapter := range chapters {
start := offset + maxFloat(0, chapter.StartSeconds)
end := 0.0
if chapter.EndSeconds > 0 {
end = offset + chapter.EndSeconds
}
out = append(out, audioChapter{
Index: len(out),
Title: chapter.Title,
StartSeconds: start,
EndSeconds: end,
TrackIndex: trackIndex,
})
}
return out
}
func sortAudioFiles(files []*models.MediaFile) {
sort.SliceStable(files, func(i, j int) bool {
left, right := files[i], files[j]
if left.PresentationPartIndex != right.PresentationPartIndex {
if left.PresentationPartIndex == 0 {
return false
}
if right.PresentationPartIndex == 0 {
return true
}
return left.PresentationPartIndex < right.PresentationPartIndex
}
if left.FilePath != right.FilePath {
return left.FilePath < right.FilePath
}
return left.ID < right.ID
})
}
func audioDeliveryMode(file *models.MediaFile) string {
if file == nil {
return "direct"
}
codec := strings.ToLower(strings.TrimSpace(file.CodecAudio))
container := strings.ToLower(strings.TrimPrefix(strings.TrimSpace(file.Container), "."))
ext := strings.ToLower(strings.TrimPrefix(filepath.Ext(file.FilePath), "."))
if container == "" {
container = ext
}
switch codec {
case "aac", "mp3", "alac", "flac":
return "direct"
case "":
switch container {
case "mp3", "m4a", "m4b", "mp4", "aac", "flac", "wav", "aiff", "aif":
return "direct"
}
}
return "aac"
}
func audioSubtitle(detail *catalog.ItemDetail) string {
if detail == nil || detail.Audiobook == nil {
return ""
}
parts := []string{}
if len(detail.Audiobook.Authors) > 0 {
names := make([]string, 0, len(detail.Audiobook.Authors))
for _, author := range detail.Audiobook.Authors {
if strings.TrimSpace(author.Name) != "" {
names = append(names, author.Name)
}
}
if len(names) > 0 {
parts = append(parts, strings.Join(names, ", "))
}
}
if len(detail.Audiobook.Narrators) > 0 {
names := make([]string, 0, len(detail.Audiobook.Narrators))
for _, narrator := range detail.Audiobook.Narrators {
if strings.TrimSpace(narrator.Name) != "" {
names = append(names, narrator.Name)
}
}
if len(names) > 0 {
parts = append(parts, "Narrated by "+strings.Join(names, ", "))
}
}
return strings.Join(parts, " • ")
}
func resolveAudioResumePosition(ctx context.Context, provider userstore.UserStoreProvider, userID int, profileID, contentID string) float64 {
if provider == nil || userID == 0 || profileID == "" {
return 0
}
store, err := provider.ForUser(ctx, userID)
if err != nil {
return 0
}
progress, err := store.GetProgress(ctx, profileID, contentID)
if err != nil || progress == nil || progress.Completed {
return 0
}
if progress.PositionSeconds < 30 {
return 0
}
if progress.DurationSeconds > 0 && progress.PositionSeconds >= progress.DurationSeconds-5 {
return 0
}
return progress.PositionSeconds
}
func isFiniteNonNegative(value float64) bool {
return !math.IsNaN(value) && !math.IsInf(value, 0) && value >= 0
}
func clampFloat(value, lower, upper float64) float64 {
if !isFiniteNonNegative(value) {
return lower
}
if value < lower {
return lower
}
if upper > lower && value > upper {
return upper
}
return value
}
func maxFloat(a, b float64) float64 {
if a > b {
return a
}
return b
}
type PGAudioBookmarkStore struct {
Pool *pgxpool.Pool
}
func NewPGAudioBookmarkStore(pool *pgxpool.Pool) *PGAudioBookmarkStore {
if pool == nil {
return nil
}
return &PGAudioBookmarkStore{Pool: pool}
}
func (s *PGAudioBookmarkStore) List(ctx context.Context, userID int, profileID, contentID string) ([]audioBookmark, error) {
if s == nil || s.Pool == nil {
return []audioBookmark{}, nil
}
rows, err := s.Pool.Query(ctx, `
SELECT id, library_item_id, time_seconds, title, created_at, updated_at
FROM abs_bookmarks
WHERE user_id = $1
AND COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid)
= COALESCE($2::uuid, '00000000-0000-0000-0000-000000000000'::uuid)
AND library_item_id = $3
ORDER BY time_seconds ASC`,
userID, profileArg(profileID), contentID,
)
if errors.Is(err, pgx.ErrNoRows) {
return []audioBookmark{}, nil
}
if err != nil {
return nil, err
}
defer rows.Close()
bookmarks := []audioBookmark{}
for rows.Next() {
var createdAt, updatedAt time.Time
var bookmark audioBookmark
if err := rows.Scan(&bookmark.ID, &bookmark.ContentID, &bookmark.TimeSeconds, &bookmark.Title, &createdAt, &updatedAt); err != nil {
return nil, err
}
bookmark.CreatedAt = createdAt.UTC().Format(time.RFC3339)
bookmark.UpdatedAt = updatedAt.UTC().Format(time.RFC3339)
bookmarks = append(bookmarks, bookmark)
}
if err := rows.Err(); err != nil {
return nil, err
}
return bookmarks, nil
}
func (s *PGAudioBookmarkStore) Upsert(ctx context.Context, userID int, profileID, contentID string, timeSeconds float64, title string) (audioBookmark, error) {
if s == nil || s.Pool == nil {
return audioBookmark{}, fmt.Errorf("bookmark store unavailable")
}
id := ulid.Make().String()
var createdAt, updatedAt time.Time
var bookmark audioBookmark
err := s.Pool.QueryRow(ctx, `
INSERT INTO abs_bookmarks
(id, user_id, profile_id, library_item_id, time_seconds, title)
VALUES ($1, $2, $3::uuid, $4, $5, $6)
ON CONFLICT (
user_id,
COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid),
library_item_id,
time_seconds
) DO UPDATE
SET title = EXCLUDED.title,
updated_at = now()
RETURNING id, library_item_id, time_seconds, title, created_at, updated_at`,
id, userID, profileArg(profileID), contentID, timeSeconds, strings.TrimSpace(title),
).Scan(&bookmark.ID, &bookmark.ContentID, &bookmark.TimeSeconds, &bookmark.Title, &createdAt, &updatedAt)
if err != nil {
return audioBookmark{}, err
}
bookmark.CreatedAt = createdAt.UTC().Format(time.RFC3339)
bookmark.UpdatedAt = updatedAt.UTC().Format(time.RFC3339)
return bookmark, nil
}
func (s *PGAudioBookmarkStore) Delete(ctx context.Context, userID int, profileID, contentID string, timeSeconds float64) error {
if s == nil || s.Pool == nil {
return fmt.Errorf("bookmark store unavailable")
}
_, err := s.Pool.Exec(ctx, `
DELETE FROM abs_bookmarks
WHERE user_id = $1
AND COALESCE(profile_id, '00000000-0000-0000-0000-000000000000'::uuid)
= COALESCE($2::uuid, '00000000-0000-0000-0000-000000000000'::uuid)
AND library_item_id = $3
AND time_seconds = $4`,
userID, profileArg(profileID), contentID, timeSeconds,
)
return err
}
func profileArg(profileID string) any {
if strings.TrimSpace(profileID) == "" {
return nil
}
return profileID
}
+201
View File
@@ -0,0 +1,201 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
type stubAudioDetailService struct {
detail *catalog.ItemDetail
}
func (s stubAudioDetailService) GetItemDetail(context.Context, string, catalog.AccessFilter) (*catalog.ItemDetail, error) {
return s.detail, nil
}
type stubAudioFiles struct {
files []*models.MediaFile
}
func (s stubAudioFiles) GetByContentID(context.Context, string) ([]*models.MediaFile, error) {
return append([]*models.MediaFile(nil), s.files...), nil
}
func (s stubAudioFiles) GetByID(_ context.Context, id int) (*models.MediaFile, error) {
for _, file := range s.files {
if file.ID == id {
return file, nil
}
}
return nil, nil
}
func TestAudioStartPlaybackOrdersTracksAndBuildsGlobalChapters(t *testing.T) {
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{
testAudioFile(3, "part-03.mp3", 30, 30),
testAudioFile(1, "part-01.mp3", 10, 10),
testAudioFile(2, "part-02.mp3", 20, 20),
}},
testUserStoreProvider{store: newPlaybackTestStore(t)},
nil,
)
req := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
req = req.WithContext(newAuthorizedPlaybackContext())
rec := httptest.NewRecorder()
handler.HandleStartPlayback(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", rec.Code, rec.Body.String())
}
var resp audioPlaybackResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if len(resp.Tracks) != 3 {
t.Fatalf("tracks = %d, want 3", len(resp.Tracks))
}
if resp.Tracks[0].FileID != 1 || resp.Tracks[1].FileID != 2 || resp.Tracks[2].FileID != 3 {
t.Fatalf("track order = [%d %d %d], want [1 2 3]", resp.Tracks[0].FileID, resp.Tracks[1].FileID, resp.Tracks[2].FileID)
}
if resp.Tracks[1].StartOffsetSeconds != 10 || resp.Tracks[2].StartOffsetSeconds != 30 {
t.Fatalf("offsets = %.0f %.0f, want 10 30", resp.Tracks[1].StartOffsetSeconds, resp.Tracks[2].StartOffsetSeconds)
}
if len(resp.Chapters) != 3 || resp.Chapters[2].StartSeconds != 30 {
t.Fatalf("global chapters = %+v, want third chapter at 30s", resp.Chapters)
}
}
func TestAudioSyncPersistsCumulativeProgressAndEnforcesOwner(t *testing.T) {
store := newPlaybackTestStore(t)
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{
testAudioFile(1, "part-01.mp3", 10, 10),
testAudioFile(2, "part-02.mp3", 20, 20),
}},
testUserStoreProvider{store: store},
nil,
)
startReq := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
startReq = startReq.WithContext(newAuthorizedPlaybackContext())
startRec := httptest.NewRecorder()
handler.HandleStartPlayback(startRec, startReq)
if startRec.Code != http.StatusOK {
t.Fatalf("start status = %d body=%s", startRec.Code, startRec.Body.String())
}
var start audioPlaybackResponse
if err := json.Unmarshal(startRec.Body.Bytes(), &start); err != nil {
t.Fatalf("decode start: %v", err)
}
otherCtx := apimw.SetClaims(context.Background(), &auth.Claims{UserID: 2, Role: "user", TokenType: auth.TokenTypeAccess})
otherCtx = apimw.SetProfileID(otherCtx, "profile-1")
denied := httptest.NewRequest(http.MethodPatch, "/api/v1/audio/playback/"+start.SessionID+"/sync", strings.NewReader(`{"position":12,"duration":30}`))
denied = withAudioRouteParam(denied.WithContext(otherCtx), "session_id", start.SessionID)
deniedRec := httptest.NewRecorder()
handler.HandleSyncPlayback(deniedRec, denied)
if deniedRec.Code != http.StatusForbidden {
t.Fatalf("foreign sync status = %d, want 403", deniedRec.Code)
}
syncReq := httptest.NewRequest(http.MethodPatch, "/api/v1/audio/playback/"+start.SessionID+"/sync", strings.NewReader(`{"position":18,"duration":30}`))
syncReq = withAudioRouteParam(syncReq.WithContext(newAuthorizedPlaybackContext()), "session_id", start.SessionID)
syncRec := httptest.NewRecorder()
handler.HandleSyncPlayback(syncRec, syncReq)
if syncRec.Code != http.StatusOK {
t.Fatalf("sync status = %d body=%s", syncRec.Code, syncRec.Body.String())
}
progress, err := store.GetProgress(context.Background(), "profile-1", "book-1")
if err != nil {
t.Fatalf("get progress: %v", err)
}
if progress == nil || progress.PositionSeconds != 18 || progress.DurationSeconds != 30 {
t.Fatalf("progress = %+v, want 18/30", progress)
}
}
func TestAudioSessionExpires(t *testing.T) {
handler := NewAudioHandler(
stubAudioDetailService{detail: testAudiobookDetail()},
stubAudioFiles{files: []*models.MediaFile{testAudioFile(1, "part-01.mp3", 10, 10)}},
testUserStoreProvider{store: newPlaybackTestStore(t)},
nil,
)
now := time.Date(2026, 6, 8, 12, 0, 0, 0, time.UTC)
handler.now = func() time.Time { return now }
startReq := httptest.NewRequest(http.MethodPost, "/api/v1/audio/playback/start", strings.NewReader(`{"content_id":"book-1"}`))
startReq = startReq.WithContext(newAuthorizedPlaybackContext())
startRec := httptest.NewRecorder()
handler.HandleStartPlayback(startRec, startReq)
var start audioPlaybackResponse
if err := json.Unmarshal(startRec.Body.Bytes(), &start); err != nil {
t.Fatalf("decode start: %v", err)
}
now = now.Add(audioPlaybackSessionTTL + time.Second)
streamReq := httptest.NewRequest(http.MethodHead, "/api/v1/audio/playback/"+start.SessionID+"/tracks/0", nil)
streamReq = withAudioRouteParam(streamReq.WithContext(newAuthorizedPlaybackContext()), "session_id", start.SessionID)
streamReq = withAudioRouteParam(streamReq, "track_index", "0")
streamRec := httptest.NewRecorder()
handler.HandleStreamTrack(streamRec, streamReq)
if streamRec.Code != http.StatusNotFound {
t.Fatalf("expired stream status = %d, want 404", streamRec.Code)
}
}
func testAudiobookDetail() *catalog.ItemDetail {
return &catalog.ItemDetail{
ContentID: "book-1",
Type: "audiobook",
Title: "Book One",
Audiobook: &catalog.AudiobookDetailExtension{
Authors: []catalog.AudiobookPerson{{Name: "Author One"}},
},
}
}
func testAudioFile(id int, name string, partIndex int, duration int) *models.MediaFile {
return &models.MediaFile{
ID: id,
ContentID: "book-1",
FilePath: "/media/" + name,
CodecAudio: "mp3",
Container: "mp3",
Duration: duration,
PresentationPartIndex: partIndex,
Chapters: []models.MediaChapter{{
Index: 0,
Title: name,
StartSeconds: 0,
EndSeconds: float64(duration),
}},
}
}
func withAudioRouteParam(req *http.Request, key, value string) *http.Request {
routeCtx := chi.RouteContext(req.Context())
if routeCtx == nil {
routeCtx = chi.NewRouteContext()
}
routeCtx.URLParams.Add(key, value)
return req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, routeCtx))
}
-531
View File
@@ -1,531 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/models"
)
// HandleGetAudiobook serves GET /api/v1/audiobooks/{id}. Returns the
// audiobook media_item, its media_files (with embedded chapters),
// author/narrator from item_people (kinds 7/8), and the caller's
// per-profile listening progress.
func (h *AudiobookHandler) HandleGetAudiobook(w http.ResponseWriter, r *http.Request) {
contentID := chi.URLParam(r, "id")
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "id is required")
return
}
items, err := h.Items.GetByIDsWithAccess(r.Context(), []string{contentID}, requestAccessFilter(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load audiobook failed")
return
}
if len(items) == 0 {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
item := items[0]
if item == nil || item.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
files, err := h.Files.GetByContentID(r.Context(), contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load files failed")
return
}
author, narrator, err := h.fetchAuthorNarrator(r, contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load people failed")
return
}
progress := h.fetchProgress(r, contentID)
alsoByAuthor := h.fetchAlsoByAuthor(r.Context(), contentID)
inSeries := h.fetchInSeries(r.Context(), contentID)
similar := h.fetchSimilar(r.Context(), contentID)
otherNarrations := h.fetchOtherNarrations(r.Context(), contentID)
resp := audiobookDetailResponse{
Audiobook: audiobookDetailItem{
ContentID: item.ContentID,
Title: item.Title,
Year: item.Year,
Overview: item.Overview,
PosterURL: h.presignAudiobookPoster(r.Context(), item.PosterPath),
Publisher: pickFirstString(item.Studios),
Genres: append([]string(nil), item.Genres...),
},
Author: author,
Narrator: narrator,
Files: audiobookDetailFiles(files),
Progress: progress,
AlsoByAuthor: alsoByAuthor,
InSeries: inSeries,
SimilarAudiobooks: similar,
OtherNarrations: otherNarrations,
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
// fetchAuthorNarrator retrieves the first author (kind=7) and narrator (kind=8)
// name from item_people for the given content ID.
func (h *AudiobookHandler) fetchAuthorNarrator(r *http.Request, contentID string) (author, narrator string, err error) {
people, err := h.Items.GetPeople(r.Context(), contentID)
if err != nil {
return "", "", err
}
for _, p := range people {
switch p.Kind {
case models.PersonKindAuthor:
if author == "" {
author = p.Name
}
case models.PersonKindNarrator:
if narrator == "" {
narrator = p.Name
}
}
if author != "" && narrator != "" {
break
}
}
return author, narrator, nil
}
// fetchAlsoByAuthor returns other audiobooks sharing any author (item_people
// kind=PersonKindAuthor) with the given book. Capped at 12 to keep the rail
// snappy on the client. Returns an empty slice when there is no author or
// no other matching titles.
func (h *AudiobookHandler) fetchAlsoByAuthor(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Files == nil {
return nil
}
const q = `
SELECT m2.content_id, m2.title, COALESCE(m2.year, 0), COALESCE(m2.poster_path, '')
FROM item_people ip1
JOIN item_people ip2
ON ip2.person_id = ip1.person_id
AND ip2.kind = ip1.kind
AND ip2.content_id <> ip1.content_id
JOIN media_items m2
ON m2.content_id = ip2.content_id
AND m2.type = 'audiobook'
WHERE ip1.content_id = $1
AND ip1.kind = $2
ORDER BY m2.year DESC NULLS LAST, LOWER(m2.sort_title)
LIMIT 12
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookRelatedItem, 0, 12)
seen := make(map[string]struct{})
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
if _, dup := seen[it.ContentID]; dup {
continue
}
seen[it.ContentID] = struct{}{}
it.PosterURL = h.presignPoster(ctx, poster)
out = append(out, it)
}
return out
}
// fetchSimilar returns the "You might also like" rail. Prefers embedding-based
// nearest-neighbor search when the source book has an embedding; falls back to
// shared-genre matching otherwise (book is brand new, embedding job hasn't run,
// or the embedding lookup errored). The genre fallback excludes same-author
// books so the rail complements rather than duplicates also_by_author.
func (h *AudiobookHandler) fetchSimilar(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Recs != nil {
if items := h.fetchSimilarByEmbedding(ctx, contentID); items != nil {
return items
}
}
return h.fetchSimilarByGenres(ctx, contentID)
}
// fetchSimilarByEmbedding returns nearest-neighbor audiobooks by cosine
// similarity on the gemini embedding vector. Returns nil (not an empty slice)
// when the source book has no embedding yet, so the caller can fall back to
// the genre-based recommender.
func (h *AudiobookHandler) fetchSimilarByEmbedding(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Recs == nil || h.Files == nil {
return nil
}
emb, err := h.Recs.GetEmbedding(ctx, contentID)
if err != nil || emb == nil {
return nil
}
// Pull extra candidates so we can drop the source book's own author from
// the rail (they belong in also_by_author).
scored, err := h.Recs.FindSimilar(ctx, emb, []string{contentID}, "audiobook", 36)
if err != nil || len(scored) == 0 {
return nil
}
ids := make([]string, 0, len(scored))
for _, s := range scored {
ids = append(ids, s.MediaItemID)
}
const q = `
WITH this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE m.content_id = ANY($3)
AND m.type = 'audiobook'
AND NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor, ids)
if err != nil {
return nil
}
defer rows.Close()
byID := make(map[string]audiobookRelatedItem, len(scored))
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
it.PosterURL = h.presignPoster(ctx, poster)
byID[it.ContentID] = it
}
out := make([]audiobookRelatedItem, 0, 12)
for _, s := range scored {
if it, ok := byID[s.MediaItemID]; ok {
out = append(out, it)
if len(out) >= 12 {
break
}
}
}
if len(out) == 0 {
return nil
}
return out
}
// fetchOtherNarrations returns sibling audiobook items that share the same
// "core" title (after stripping the common "read by X" / "(Read by X)" /
// "(UK Version: Read by X)" suffixes) and the same author. Powers the
// narrator picker dropdown on the detail page when a single book has
// multiple narration editions stored as separate items in the catalog.
func (h *AudiobookHandler) fetchOtherNarrations(ctx context.Context, contentID string) []audiobookNarration {
if h.Files == nil {
return nil
}
// The regex strips a trailing block of the form "[ separator] [( ]read by ...[) ]".
// Tested against patterns in the corpus: "Foo Read By X",
// "Foo - read by X", "Foo (Read by X)", "Foo (UK Version: Read by X)".
const stripRE = `\s*\(?\s*[-:,]?\s*(UK Version:?|US Version:?)?\s*[Rr]ead [Bb]y [^()]+\)?\s*$`
const q = `
WITH this_book AS (
SELECT trim(regexp_replace(title, $1, '', 'g')) AS norm_title
FROM media_items WHERE content_id = $2
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $2 AND kind = $3
)
SELECT
m.content_id,
m.title,
COALESCE(MAX(p.name), '') AS narrator,
COALESCE(m.year, 0)
FROM media_items m
LEFT JOIN item_people ipn ON ipn.content_id = m.content_id AND ipn.kind = $4
LEFT JOIN people p ON p.id = ipn.person_id
WHERE m.type = 'audiobook'
AND m.content_id <> $2
AND trim(regexp_replace(m.title, $1, '', 'g')) = (SELECT norm_title FROM this_book)
AND EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $3
AND ip.person_id IN (SELECT person_id FROM this_author)
)
GROUP BY m.content_id, m.title, m.year
ORDER BY m.year DESC NULLS LAST, m.title
LIMIT 8
`
rows, err := h.Files.Pool().Query(ctx, q,
stripRE, contentID, models.PersonKindAuthor, models.PersonKindNarrator)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookNarration, 0, 4)
for rows.Next() {
var n audiobookNarration
if err := rows.Scan(&n.ContentID, &n.Title, &n.Narrator, &n.Year); err != nil {
return nil
}
out = append(out, n)
}
if len(out) == 0 {
return nil
}
return out
}
// fetchSimilarByGenres returns up to 12 audiobooks sharing genres with the
// given book, excluding books by the same author so the rail complements
// rather than duplicates also_by_author. Used as a fallback when no
// embedding exists for the source yet.
func (h *AudiobookHandler) fetchSimilarByGenres(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Files == nil {
return nil
}
const q = `
WITH this_genres AS (
SELECT unnest(genres) AS g FROM media_items WHERE content_id = $1
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE m.type = 'audiobook'
AND m.content_id <> $1
AND m.genres && (SELECT array_agg(g) FROM this_genres)
AND NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)
ORDER BY
cardinality(ARRAY(SELECT unnest(m.genres) INTERSECT SELECT g FROM this_genres)) DESC,
COALESCE(m.year, 0) DESC,
LOWER(m.sort_title)
LIMIT 12
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookRelatedItem, 0, 12)
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
it.PosterURL = h.presignPoster(ctx, poster)
out = append(out, it)
}
return out
}
// fetchInSeries returns the series the audiobook belongs to (if any) along
// with the ordered list of sibling entries. Driven by the audiobook_series
// table populated by either the scanner (tag-derived) or migration 145's
// best-effort title-pattern backfill.
func (h *AudiobookHandler) fetchInSeries(ctx context.Context, contentID string) *audiobookSeriesGroup {
if h.Files == nil {
return nil
}
const q = `
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(m.poster_path, ''),
s.series_name,
s.series_index
FROM audiobook_series root
JOIN audiobook_series s ON LOWER(s.series_name) = LOWER(root.series_name)
JOIN media_items m ON m.content_id = s.content_id AND m.type = 'audiobook'
WHERE root.content_id = $1
ORDER BY s.series_index NULLS LAST, LOWER(m.sort_title)
LIMIT 30
`
rows, err := h.Files.Pool().Query(ctx, q, contentID)
if err != nil {
return nil
}
defer rows.Close()
var seriesName string
entries := make([]audiobookRelatedItem, 0, 16)
for rows.Next() {
var (
it audiobookRelatedItem
poster string
thisName string
idxNum *float64
)
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster, &thisName, &idxNum); err != nil {
return nil
}
if seriesName == "" {
seriesName = thisName
}
if idxNum != nil {
n := int(*idxNum)
if float64(n) == *idxNum {
it.SeriesIndex = &n
}
}
it.PosterURL = h.presignPoster(ctx, poster)
entries = append(entries, it)
}
// A series with only this book is not useful as a rail.
if len(entries) < 2 {
return nil
}
return &audiobookSeriesGroup{Name: seriesName, Entries: entries}
}
// fetchProgress returns the caller's listening progress for this audiobook, or
// nil when no progress exists or the caller is not authenticated with a profile.
func (h *AudiobookHandler) fetchProgress(r *http.Request, contentID string) *audiobookDetailProgress {
if h.StoreProvider == nil {
return nil
}
userID := apimw.GetUserID(r.Context())
if userID == 0 {
return nil
}
profileID := apimw.GetProfileID(r.Context())
if profileID == "" {
profileID = r.Header.Get("X-Profile-Id")
}
if profileID == "" {
return nil
}
store, err := h.StoreProvider.ForUser(r.Context(), userID)
if err != nil || store == nil {
return nil
}
wp, err := store.GetProgress(r.Context(), profileID, contentID)
if err != nil || wp == nil {
return nil
}
return &audiobookDetailProgress{
PositionSeconds: wp.PositionSeconds,
Completed: wp.Completed,
UpdatedAt: wp.UpdatedAt,
}
}
type audiobookDetailResponse struct {
Audiobook audiobookDetailItem `json:"audiobook"`
Author string `json:"author,omitempty"`
Narrator string `json:"narrator,omitempty"`
Files []audiobookDetailFile `json:"files"`
Progress *audiobookDetailProgress `json:"progress,omitempty"`
AlsoByAuthor []audiobookRelatedItem `json:"also_by_author,omitempty"`
InSeries *audiobookSeriesGroup `json:"in_series,omitempty"`
SimilarAudiobooks []audiobookRelatedItem `json:"similar_audiobooks,omitempty"`
OtherNarrations []audiobookNarration `json:"other_narrations,omitempty"`
}
// audiobookNarration is a sibling narration (same book, different
// narrator/edition). Returned alongside the primary item so the UI can
// surface a narrator picker.
type audiobookNarration struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Narrator string `json:"narrator,omitempty"`
Year int `json:"year,omitempty"`
}
type audiobookRelatedItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
SeriesIndex *int `json:"series_index,omitempty"`
}
type audiobookSeriesGroup struct {
Name string `json:"name,omitempty"`
Entries []audiobookRelatedItem `json:"entries"`
}
type audiobookDetailItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year"`
Overview string `json:"overview,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
Publisher string `json:"publisher,omitempty"`
Genres []string `json:"genres,omitempty"`
}
func (h *AudiobookHandler) presignPoster(ctx context.Context, path string) string {
if path == "" || h.Detail == nil {
return ""
}
return h.Detail.PresignURL(ctx, path, "featured")
}
func pickFirstString(values []string) string {
for _, v := range values {
if v != "" {
return v
}
}
return ""
}
type audiobookDetailFile struct {
ID int `json:"id"`
Path string `json:"path"`
DurationSeconds int `json:"duration_seconds"`
Container string `json:"container,omitempty"`
CodecAudio string `json:"codec_audio,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
AudioChannels int `json:"audio_channels,omitempty"`
Chapters []models.MediaChapter `json:"chapters,omitempty"`
}
type audiobookDetailProgress struct {
PositionSeconds float64 `json:"position_seconds"`
Completed bool `json:"completed"`
UpdatedAt string `json:"updated_at"`
}
func audiobookDetailFiles(files []*models.MediaFile) []audiobookDetailFile {
out := make([]audiobookDetailFile, 0, len(files))
for _, f := range files {
out = append(out, audiobookDetailFile{
ID: f.ID,
Path: f.FilePath,
DurationSeconds: f.Duration,
Container: f.Container,
CodecAudio: f.CodecAudio,
Bitrate: f.Bitrate,
AudioChannels: f.AudioChannels,
Chapters: f.Chapters,
})
}
return out
}
-183
View File
@@ -1,183 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/recommendations"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// AudiobookHandler serves silo-native audiobook endpoints under
// /api/v1/audiobooks/*. Sub-plan 3 implements list/detail/progress;
// sub-plan 4's ABS-compat layer is separate.
type AudiobookHandler struct {
Items *catalog.ItemRepository
Files *scanner.FileRepository
StoreProvider userstore.UserStoreProvider
Detail *catalog.DetailService
// Recs is optional. When set, the detail handler uses embedding-based
// nearest-neighbor search for the "You might also like" rail and falls
// back to shared-genre matching only when no embedding exists for the
// source book yet.
Recs *recommendations.Repo
}
// HandleListAudiobooks serves GET /api/v1/audiobooks?limit=&offset=.
// Paginated catalog of media_items with type='audiobook' scoped to the
// caller's accessible libraries.
func (h *AudiobookHandler) HandleListAudiobooks(w http.ResponseWriter, r *http.Request) {
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
if limit <= 0 || limit > 200 {
limit = 50
}
offset, _ := strconv.Atoi(r.URL.Query().Get("offset"))
if offset < 0 {
offset = 0
}
ctx := r.Context()
access := requestAccessFilter(r)
genre := r.URL.Query().Get("genre")
conditions, args, argIdx, empty := audiobookListConditions(access, genre)
if empty {
writeAudiobookListResponse(w, nil, 0, limit, offset)
return
}
pool := h.Files.Pool()
var total int
countSQL := "SELECT COUNT(*) FROM media_items mi WHERE " + strings.Join(conditions, " AND ")
if err := pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "list audiobooks count failed")
return
}
dataArgs := append([]any(nil), args...)
limitIdx := argIdx
dataArgs = append(dataArgs, limit)
offsetIdx := argIdx + 1
dataArgs = append(dataArgs, offset)
rows, err := pool.Query(ctx, fmt.Sprintf(`
SELECT
mi.content_id,
mi.title,
COALESCE(mi.year, 0),
COALESCE(mi.poster_path, ''),
COALESCE(SUM(mf.duration), 0)::int AS duration_seconds,
COALESCE(MIN(mf.container), '') AS container,
COALESCE(MIN(mf.codec_audio), '') AS codec_audio,
COUNT(mf.id) AS file_count
FROM media_items mi
LEFT JOIN media_files mf ON mf.content_id = mi.content_id
WHERE %s
GROUP BY mi.content_id, mi.title, mi.year, mi.poster_path, mi.sort_title
ORDER BY LOWER(mi.sort_title), LOWER(mi.title)
LIMIT $%d OFFSET $%d
`, strings.Join(conditions, " AND "), limitIdx, offsetIdx), dataArgs...)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "list audiobooks failed")
return
}
defer rows.Close()
items := make([]audiobookListItem, 0, limit)
for rows.Next() {
var it audiobookListItem
var posterPath string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &posterPath, &it.DurationSeconds, &it.Container, &it.CodecAudio, &it.FileCount); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "scan audiobook row failed")
return
}
it.PosterURL = h.presignAudiobookPoster(ctx, posterPath)
items = append(items, it)
}
if err := rows.Err(); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "iterate audiobook rows failed")
return
}
writeAudiobookListResponse(w, items, total, limit, offset)
}
func audiobookListConditions(access catalog.AccessFilter, genre string) ([]string, []any, int, bool) {
conditions := []string{"mi.type = 'audiobook'"}
var args []any
argIdx := 1
if genre != "" {
conditions = append(conditions, fmt.Sprintf("$%d = ANY(mi.genres)", argIdx))
args = append(args, genre)
argIdx++
}
if access.AllowedLibraryIDs != nil {
if len(access.AllowedLibraryIDs) == 0 {
return nil, nil, 1, true
}
conditions = append(conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
AND mil.media_folder_id = ANY($%d)
)`, argIdx))
args = append(args, access.AllowedLibraryIDs)
argIdx++
}
if len(access.DisabledLibraryIDs) > 0 {
if access.AllowedLibraryIDs == nil {
conditions = append(conditions, `
EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
)`)
}
conditions = append(conditions, fmt.Sprintf(`
NOT EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
AND mil.media_folder_id = ANY($%d)
)`, argIdx))
args = append(args, access.DisabledLibraryIDs)
argIdx++
}
catalog.ApplySectionAccessFilter("mi", catalog.AccessFilter{MaxContentRating: access.MaxContentRating}, &conditions, &args, &argIdx)
return conditions, args, argIdx, false
}
func writeAudiobookListResponse(w http.ResponseWriter, items []audiobookListItem, total, limit, offset int) {
if items == nil {
items = []audiobookListItem{}
}
resp := struct {
Items []audiobookListItem `json:"items"`
Total int `json:"total"`
Limit int `json:"limit"`
Offset int `json:"offset"`
}{Items: items, Total: total, Limit: limit, Offset: offset}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
type audiobookListItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year"`
PosterURL string `json:"poster_url,omitempty"`
DurationSeconds int `json:"duration_seconds"`
Container string `json:"container,omitempty"`
CodecAudio string `json:"codec_audio,omitempty"`
FileCount int `json:"file_count"`
}
func (h *AudiobookHandler) presignAudiobookPoster(ctx context.Context, path string) string {
if h == nil || h.Detail == nil || path == "" {
return ""
}
return h.Detail.PresignURL(ctx, cardThumbnailPath(path), "card")
}
@@ -1,113 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type audiobookProgressRequest struct {
PositionSeconds float64 `json:"position_seconds"`
MediaFileID int `json:"media_file_id,omitempty"`
}
// HandleReportAudiobookProgress serves POST /api/v1/audiobooks/{id}/progress.
// UPSERTs into user_watch_progress for the caller's (user_id, profile_id, content_id).
func (h *AudiobookHandler) HandleReportAudiobookProgress(w http.ResponseWriter, r *http.Request) {
contentID := chi.URLParam(r, "id")
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "id is required")
return
}
var req audiobookProgressRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid request body")
return
}
if req.PositionSeconds < 0 {
writeError(w, http.StatusBadRequest, "bad_request", "position_seconds must be >= 0")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if profileID == "" {
profileID = r.Header.Get("X-Profile-Id")
}
if userID == 0 || profileID == "" {
writeError(w, http.StatusUnauthorized, "unauthorized", "authentication required")
return
}
if h.Items == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "catalog unavailable")
return
}
items, err := h.Items.GetByIDsWithAccess(r.Context(), []string{contentID}, requestAccessFilter(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "check access failed")
return
}
if len(items) == 0 || items[0] == nil || items[0].Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
durationSeconds, err := h.audiobookProgressDuration(r.Context(), contentID, req.MediaFileID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load files failed")
return
}
if h.StoreProvider == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "store unavailable")
return
}
store, err := h.StoreProvider.ForUser(r.Context(), userID)
if err != nil || store == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "store unavailable")
return
}
if err := store.SetProgress(r.Context(), profileID, contentID, req.PositionSeconds, durationSeconds, userstore.ProgressThresholds{}); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "save progress failed")
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *AudiobookHandler) audiobookProgressDuration(ctx context.Context, contentID string, mediaFileID int) (float64, error) {
if h == nil || h.Files == nil {
return 0, nil
}
files, err := h.Files.GetByContentID(ctx, contentID)
if err != nil {
return 0, err
}
return audiobookDurationForProgress(files, mediaFileID), nil
}
func audiobookDurationForProgress(files []*models.MediaFile, mediaFileID int) float64 {
var total int
for _, f := range files {
if f == nil || f.Duration <= 0 {
continue
}
if mediaFileID > 0 && f.ID == mediaFileID {
return float64(f.Duration)
}
total += f.Duration
}
if total <= 0 {
return 0
}
return float64(total)
}
-61
View File
@@ -1,61 +0,0 @@
package handlers
import (
"context"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestPresignAudiobookPosterDoesNotExposeRawKeysWithoutPresigner(t *testing.T) {
h := &AudiobookHandler{}
got := h.presignAudiobookPoster(context.Background(), "metadata/audiobooks/book-1/poster/original.webp")
if got != "" {
t.Fatalf("PosterURL = %q, want empty without presigner", got)
}
}
func TestAudiobookListConditionsIncludeAccessPredicates(t *testing.T) {
conditions, args, _, empty := audiobookListConditions(catalog.AccessFilter{
AllowedLibraryIDs: []int{1, 2},
DisabledLibraryIDs: []int{9},
MaxContentRating: "PG-13",
}, "Fantasy")
if empty {
t.Fatal("empty = true, want false")
}
sql := strings.Join(conditions, " AND ")
for _, want := range []string{
"mi.type = 'audiobook'",
"ANY(mi.genres)",
"EXISTS",
"NOT EXISTS",
"mi.content_rating = ANY",
} {
if !strings.Contains(sql, want) {
t.Fatalf("conditions missing %q in:\n%s", want, sql)
}
}
if len(args) != 4 {
t.Fatalf("len(args) = %d, want 4", len(args))
}
}
func TestAudiobookDurationForProgress(t *testing.T) {
files := []*models.MediaFile{
{ID: 10, Duration: 120},
{ID: 11, Duration: 180},
}
if got := audiobookDurationForProgress(files, 10); got != 120 {
t.Fatalf("selected file duration = %v, want 120", got)
}
if got := audiobookDurationForProgress(files, 0); got != 300 {
t.Fatalf("total duration = %v, want 300", got)
}
if got := audiobookDurationForProgress([]*models.MediaFile{{ID: 1}}, 1); got != 0 {
t.Fatalf("unknown duration = %v, want 0", got)
}
}
+94 -57
View File
@@ -36,6 +36,7 @@ type autoscanStore interface {
CountAutoscanScans(ctx context.Context, filter autoscan.ScanListFilter) (int, error)
ListEvents(ctx context.Context, filter autoscan.EventListFilter) ([]autoscan.EventWithRuns, error)
CountEvents(ctx context.Context, filter autoscan.EventListFilter) (int, error)
ListRunningEvents(ctx context.Context) ([]autoscan.Event, error)
GetQueueSummary(ctx context.Context) (autoscan.QueueSummary, error)
LatestEventAt(ctx context.Context) (*time.Time, error)
}
@@ -309,7 +310,7 @@ func (h *AutoscanHandler) HandleDeleteConnection(w http.ResponseWriter, r *http.
// credentials: the connection link is by id only.
type autoscanSourceResponse struct {
ID string `json:"id"`
InstallationID int `json:"installation_id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
@@ -331,7 +332,7 @@ func sourceResponse(s autoscan.Source) autoscanSourceResponse {
config := normalizeSourceConfig(s.SourceConfig)
return autoscanSourceResponse{
ID: s.ID,
InstallationID: s.InstallationID,
PluginID: s.PluginID,
CapabilityID: s.CapabilityID,
ConnectionID: s.ConnectionID,
Enabled: s.Enabled,
@@ -364,10 +365,9 @@ func (h *AutoscanHandler) HandleListSources(w http.ResponseWriter, r *http.Reque
// --- Available scan-source plugins (Add-source picker) ---
type autoscanScanSourcePluginResponse struct {
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
PluginID string `json:"plugin_id"`
DisplayName string `json:"display_name"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
DisplayName string `json:"display_name"`
}
// HandleListAvailableScanSources returns every installed scan_source capability
@@ -381,10 +381,9 @@ func (h *AutoscanHandler) HandleListAvailableScanSources(w http.ResponseWriter,
out := make([]autoscanScanSourcePluginResponse, 0, len(available))
for _, a := range available {
out = append(out, autoscanScanSourcePluginResponse{
InstallationID: a.InstallationID,
CapabilityID: a.CapabilityID,
PluginID: a.PluginID,
DisplayName: a.DisplayName,
PluginID: a.PluginID,
CapabilityID: a.CapabilityID,
DisplayName: a.DisplayName,
})
}
writeJSON(w, http.StatusOK, struct {
@@ -394,12 +393,12 @@ func (h *AutoscanHandler) HandleListAvailableScanSources(w http.ResponseWriter,
// --- Create source ---
// autoscanCreateSourceInput is the create payload: it names the installed
// scan_source capability to bind (installation_id + capability_id) plus the
// initial binding/scheduling fields. Unlike the update payload, identity is
// supplied here rather than read from an existing row.
// autoscanCreateSourceInput is the create payload: it names the scan_source
// capability to bind (plugin_id + capability_id) plus the initial
// binding/scheduling fields. Unlike the update payload, identity is supplied
// here rather than read from an existing row.
type autoscanCreateSourceInput struct {
InstallationID int `json:"installation_id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
@@ -420,6 +419,11 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
return
}
capID := strings.TrimSpace(in.CapabilityID)
pluginID := strings.TrimSpace(in.PluginID)
if pluginID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "plugin_id is required")
return
}
if capID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "capability_id is required")
return
@@ -432,7 +436,7 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
// Validate the (installation_id, capability_id) is a currently-installed
// Validate the (plugin_id, capability_id) is a currently-installed
// scan_source capability — a source may only be created against an installed
// plugin capability.
available, err := h.svc.ListAvailableScanSources(r.Context())
@@ -440,13 +444,13 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeAutoscanError(w, err)
return
}
if !scanSourceInstalled(available, in.InstallationID, capID) {
writeError(w, http.StatusBadRequest, "bad_request", "installation_id + capability_id is not a currently-installed scan_source capability")
if !scanSourceInstalled(available, pluginID, capID) {
writeError(w, http.StatusBadRequest, "bad_request", "plugin_id + capability_id is not a currently-installed scan_source capability")
return
}
connArg := normalizeConnectionID(in.ConnectionID)
created, err := h.repo.CreateSource(r.Context(), autoscan.Source{
InstallationID: in.InstallationID,
PluginID: pluginID,
CapabilityID: capID,
ConnectionID: connArg,
Enabled: in.Enabled,
@@ -462,18 +466,18 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeJSON(w, http.StatusCreated, sourceResponse(created))
}
// scanSourceInstalled reports whether (installationID, capabilityID) is in the
// scanSourceInstalled reports whether (pluginID, capabilityID) is in the
// set of currently-installed scan_source capabilities.
func scanSourceInstalled(available []autoscan.AvailableScanSource, installationID int, capabilityID string) bool {
func scanSourceInstalled(available []autoscan.AvailableScanSource, pluginID, capabilityID string) bool {
for _, a := range available {
if a.InstallationID == installationID && a.CapabilityID == capabilityID {
if a.PluginID == pluginID && a.CapabilityID == capabilityID {
return true
}
}
return false
}
// autoscanSourceInput is the source write payload. The (installation_id,
// autoscanSourceInput is the source write payload. The (plugin_id,
// capability_id) identity is read from the existing source row, so only the
// schedulable/binding fields are accepted.
//
@@ -548,7 +552,7 @@ func (h *AutoscanHandler) HandleUpdateSource(w http.ResponseWriter, r *http.Requ
// connection_id is a full-state field: nil means unbind, a UUID string
// means bind. A whitespace-only string is normalised to nil (unbound).
connArg := normalizeConnectionID(in.ConnectionID)
// Update by id; identity (installation_id, capability_id) is immutable and
// Update by id; identity (plugin_id, capability_id) is immutable and
// preserved by the repo. A missing source maps to 404.
updated, err := h.repo.UpdateSource(r.Context(), autoscan.Source{
ID: id,
@@ -581,9 +585,8 @@ func normalizeConnectionID(in *string) *string {
}
// HandleDeleteSource removes a source row. This is the operator's escape hatch
// for an orphaned source (one whose scan_source plugin was uninstalled): the
// poll loop skips such rows quietly, and this endpoint clears them. An unknown
// id maps to 404 via autoscan.ErrNotFound.
// for an orphaned source (one whose scan_source plugin was uninstalled or no
// longer resolves cleanly). An unknown id maps to 404 via autoscan.ErrNotFound.
func (h *AutoscanHandler) HandleDeleteSource(w http.ResponseWriter, r *http.Request) {
id := strings.TrimSpace(chi.URLParam(r, "id"))
if err := h.repo.DeleteSource(r.Context(), id); err != nil {
@@ -710,7 +713,7 @@ type autoscanEventScanRunResponse struct {
type autoscanEventResponse struct {
ID int64 `json:"id"`
SourceID *string `json:"source_id"`
InstallationID int `json:"installation_id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
StartedAt time.Time `json:"started_at"`
CompletedAt time.Time `json:"completed_at"`
@@ -753,7 +756,7 @@ func eventResponse(event autoscan.EventWithRuns) autoscanEventResponse {
return autoscanEventResponse{
ID: e.ID,
SourceID: e.SourceID,
InstallationID: e.InstallationID,
PluginID: e.PluginID,
CapabilityID: e.CapabilityID,
StartedAt: e.StartedAt,
CompletedAt: e.CompletedAt,
@@ -794,9 +797,9 @@ func (h *AutoscanHandler) HandleListEvents(w http.ResponseWriter, r *http.Reques
filter.Offset = offset
}
switch filter.Status {
case "", autoscan.EventStatusSuccess, autoscan.EventStatusError, autoscan.EventStatusUnresolved:
case "", autoscan.EventStatusRunning, autoscan.EventStatusSuccess, autoscan.EventStatusError, autoscan.EventStatusUnresolved:
default:
writeError(w, http.StatusBadRequest, "bad_request", "status must be success, error, or unresolved")
writeError(w, http.StatusBadRequest, "bad_request", "status must be running, success, error, or unresolved")
return
}
@@ -835,7 +838,7 @@ type autoscanScanResponse struct {
CompletedAt *time.Time `json:"completed_at,omitempty"`
AutoscanEventID *int64 `json:"autoscan_event_id,omitempty"`
SourceID *string `json:"source_id,omitempty"`
InstallationID *int `json:"installation_id,omitempty"`
PluginID string `json:"plugin_id,omitempty"`
CapabilityID string `json:"capability_id,omitempty"`
EventStatus string `json:"event_status,omitempty"`
EventCompletedAt *time.Time `json:"event_completed_at,omitempty"`
@@ -862,7 +865,7 @@ func autoscanScanRunResponse(scan autoscan.ScanWithEvent) autoscanScanResponse {
CompletedAt: scan.CompletedAt,
AutoscanEventID: scan.AutoscanEventID,
SourceID: scan.SourceID,
InstallationID: scan.InstallationID,
PluginID: scan.PluginID,
CapabilityID: scan.CapabilityID,
EventStatus: string(scan.EventStatus),
EventCompletedAt: scan.EventCompletedAt,
@@ -923,24 +926,35 @@ func (h *AutoscanHandler) HandleListScans(w http.ResponseWriter, r *http.Request
// --- Status ---
type autoscanStatusSource struct {
ID string `json:"id"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
Label string `json:"label"`
PathRewrites []autoscan.PathRewrite `json:"path_rewrites"`
LastRunAt *time.Time `json:"last_run_at,omitempty"`
LastError *string `json:"last_error,omitempty"`
ID string `json:"id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
Label string `json:"label"`
PathRewrites []autoscan.PathRewrite `json:"path_rewrites"`
LastRunAt *time.Time `json:"last_run_at,omitempty"`
LastError *string `json:"last_error,omitempty"`
}
type autoscanRunningPollResponse struct {
ID int64 `json:"id"`
SourceID *string `json:"source_id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
StartedAt time.Time `json:"started_at"`
ElapsedMS int64 `json:"elapsed_ms"`
MarkerBefore *string `json:"marker_before,omitempty"`
}
type autoscanStatusResponse struct {
Enabled bool `json:"enabled"`
Sources []autoscanStatusSource `json:"sources"`
ActiveScans int `json:"active_scans"`
AcceptedScans int `json:"accepted_scans"`
RunningScans int `json:"running_scans"`
LatestEventAt *time.Time `json:"latest_event_at,omitempty"`
Enabled bool `json:"enabled"`
Sources []autoscanStatusSource `json:"sources"`
RunningPolls []autoscanRunningPollResponse `json:"running_polls"`
ActiveScans int `json:"active_scans"`
AcceptedScans int `json:"accepted_scans"`
RunningScans int `json:"running_scans"`
LatestEventAt *time.Time `json:"latest_event_at,omitempty"`
}
func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
@@ -960,6 +974,11 @@ func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
writeAutoscanError(w, err)
return
}
runningEvents, err := h.repo.ListRunningEvents(ctx)
if err != nil {
writeAutoscanError(w, err)
return
}
latestEventAt, err := h.repo.LatestEventAt(ctx)
if err != nil {
writeAutoscanError(w, err)
@@ -972,20 +991,38 @@ func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
rewrites = []autoscan.PathRewrite{}
}
trimmed = append(trimmed, autoscanStatusSource{
ID: src.ID,
InstallationID: src.InstallationID,
CapabilityID: src.CapabilityID,
ConnectionID: src.ConnectionID,
Enabled: src.Enabled,
Label: src.Label,
PathRewrites: rewrites,
LastRunAt: src.LastRunAt,
LastError: src.LastError,
ID: src.ID,
PluginID: src.PluginID,
CapabilityID: src.CapabilityID,
ConnectionID: src.ConnectionID,
Enabled: src.Enabled,
Label: src.Label,
PathRewrites: rewrites,
LastRunAt: src.LastRunAt,
LastError: src.LastError,
})
}
now := time.Now()
runningPolls := make([]autoscanRunningPollResponse, 0, len(runningEvents))
for _, event := range runningEvents {
elapsed := now.Sub(event.StartedAt)
if elapsed < 0 {
elapsed = 0
}
runningPolls = append(runningPolls, autoscanRunningPollResponse{
ID: event.ID,
SourceID: event.SourceID,
PluginID: event.PluginID,
CapabilityID: event.CapabilityID,
StartedAt: event.StartedAt,
ElapsedMS: elapsed.Milliseconds(),
MarkerBefore: event.MarkerBefore,
})
}
writeJSON(w, http.StatusOK, autoscanStatusResponse{
Enabled: s.Enabled,
Sources: trimmed,
RunningPolls: runningPolls,
ActiveScans: queue.Active,
AcceptedScans: queue.Accepted,
RunningScans: queue.Running,
+52 -22
View File
@@ -31,6 +31,7 @@ type fakeAutoscanStore struct {
countScansFn func(autoscan.ScanListFilter) (int, error)
listEventsFn func(autoscan.EventListFilter) ([]autoscan.EventWithRuns, error)
countEventsFn func(autoscan.EventListFilter) (int, error)
listRunningFn func() ([]autoscan.Event, error)
queueSummaryFn func() (autoscan.QueueSummary, error)
latestEventAtFn func() (*time.Time, error)
}
@@ -140,6 +141,13 @@ func (f *fakeAutoscanStore) CountEvents(_ context.Context, filter autoscan.Event
return 0, nil
}
func (f *fakeAutoscanStore) ListRunningEvents(context.Context) ([]autoscan.Event, error) {
if f.listRunningFn != nil {
return f.listRunningFn()
}
return nil, nil
}
func (f *fakeAutoscanStore) GetQueueSummary(context.Context) (autoscan.QueueSummary, error) {
if f.queueSummaryFn != nil {
return f.queueSummaryFn()
@@ -273,7 +281,7 @@ func TestAutoscanHandleListSourcesListsOnly(t *testing.T) {
store := &fakeAutoscanStore{
listSourcesFn: func() ([]autoscan.Source, error) {
listed = true
return []autoscan.Source{{ID: "src-1", InstallationID: 1, CapabilityID: "arr"}}, nil
return []autoscan.Source{{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr"}}, nil
},
}
h := NewAutoscanHandler(store, &fakeAutoscanTriggerer{})
@@ -300,7 +308,7 @@ func TestAutoscanHandleListSourcesListsOnly(t *testing.T) {
func TestAutoscanHandleListAvailableScanSources(t *testing.T) {
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{InstallationID: 1, CapabilityID: "arr", PluginID: "sonarr", DisplayName: "Sonarr"},
{PluginID: "sonarr", CapabilityID: "arr", DisplayName: "Sonarr"},
}}
h := NewAutoscanHandler(&fakeAutoscanStore{}, trig)
@@ -331,19 +339,19 @@ func TestAutoscanHandleCreateSourceSucceeds(t *testing.T) {
},
}
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{InstallationID: 1, CapabilityID: "arr"},
{PluginID: "silo.autoscan.arr", CapabilityID: "arr"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"installation_id":1,"capability_id":"arr","connection_id":"conn-1","enabled":true}`, "")
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","connection_id":"conn-1","enabled":true}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
if rec.Code != http.StatusCreated {
t.Fatalf("status = %d, want 201; body=%s", rec.Code, rec.Body.String())
}
if got.InstallationID != 1 || got.CapabilityID != "arr" || got.ConnectionID == nil || *got.ConnectionID != "conn-1" {
if got.PluginID != "silo.autoscan.arr" || got.CapabilityID != "arr" || got.ConnectionID == nil || *got.ConnectionID != "conn-1" {
t.Fatalf("unexpected created source: %+v", got)
}
var body autoscanSourceResponse
@@ -363,14 +371,14 @@ func TestAutoscanHandleCreateSourceRejectsUnknownCapability(t *testing.T) {
return s, nil
},
}
// Installed set does NOT include (1, "arr").
// Installed set does NOT include ("silo.autoscan.arr", "arr").
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{InstallationID: 2, CapabilityID: "other"},
{PluginID: "silo.autoscan.other", CapabilityID: "other"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"installation_id":1,"capability_id":"arr","enabled":false}`, "")
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","enabled":false}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
@@ -394,12 +402,12 @@ func TestAutoscanHandleCreateSourceEnableWithoutConnectionSucceeds(t *testing.T)
},
}
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{InstallationID: 1, CapabilityID: "arr"},
{PluginID: "silo.autoscan.arr", CapabilityID: "arr"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"installation_id":1,"capability_id":"arr","enabled":true}`, "")
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","enabled":true}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
@@ -615,7 +623,7 @@ func TestAutoscanHandleUpdateSourceEnableWithoutConnectionSucceeds(t *testing.T)
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -641,7 +649,7 @@ func TestAutoscanHandleUpdateSourceEnableWithConnectionSucceeds(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -669,7 +677,7 @@ func TestAutoscanHandleUpdateSourceBindConnectionSucceeds(t *testing.T) {
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
// Source starts with no connection.
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -697,7 +705,7 @@ func TestAutoscanHandleUpdateSourceUnbindConnectionSucceeds(t *testing.T) {
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
// Source starts with a connection already bound.
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -727,7 +735,7 @@ func TestAutoscanHandleUpdateSourceUnbindWhileEnabledSucceeds(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -925,7 +933,7 @@ func TestAutoscanHandleUpdateSourceRoundTripsPathRewrites(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -990,7 +998,7 @@ func TestAutoscanHandleUpdateSourceRejectsBlankRewrite(t *testing.T) {
upserted := false
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
upserted = true
@@ -1020,7 +1028,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
eventCompleted := completed.Add(-1 * time.Second)
eventID := int64(42)
sourceID := "src-1"
installationID := 7
pluginID := "silo.autoscan.cephfs"
var gotFilter autoscan.ScanListFilter
var gotCountFilter autoscan.ScanListFilter
store := &fakeAutoscanStore{
@@ -1040,7 +1048,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
},
AutoscanEventID: &eventID,
SourceID: &sourceID,
InstallationID: &installationID,
PluginID: pluginID,
CapabilityID: "cephfs",
EventStatus: autoscan.EventStatusSuccess,
EventCompletedAt: &eventCompleted,
@@ -1084,7 +1092,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
if scan.AutoscanEventID == nil || *scan.AutoscanEventID != eventID || scan.EventStatus != "success" {
t.Fatalf("event context = %+v", scan)
}
if scan.SourceID == nil || *scan.SourceID != sourceID || scan.InstallationID == nil || *scan.InstallationID != installationID || scan.CapabilityID != "cephfs" {
if scan.SourceID == nil || *scan.SourceID != sourceID || scan.PluginID != pluginID || scan.CapabilityID != "cephfs" {
t.Fatalf("source context = %+v", scan)
}
}
@@ -1128,7 +1136,7 @@ func TestAutoscanHandleListEventsSerializesFiltersAndRuns(t *testing.T) {
Event: autoscan.Event{
ID: 42,
SourceID: &sourceID,
InstallationID: 7,
PluginID: "silo.autoscan.cephfs",
CapabilityID: "cephfs",
StartedAt: started,
CompletedAt: completed,
@@ -1258,18 +1266,30 @@ func TestAutoscanHandleUpdateSourceNormalizesLabel(t *testing.T) {
func TestAutoscanHandleStatusReturnsTrimmedSources(t *testing.T) {
latestEventAt := time.Date(2026, 6, 4, 14, 30, 0, 0, time.UTC)
runningStartedAt := time.Now().Add(-90 * time.Second)
sourceID := "src-1"
store := &fakeAutoscanStore{
getSettingsFn: func() (autoscan.Settings, error) {
return autoscan.Settings{Enabled: true}, nil
},
listSourcesFn: func() ([]autoscan.Source, error) {
return []autoscan.Source{
{ID: "src-1", InstallationID: 7, CapabilityID: "scan_source", ConnectionID: ptr("conn-1"), Enabled: true},
{ID: "src-1", PluginID: "silo.autoscan.cephfs", CapabilityID: "scan_source", ConnectionID: ptr("conn-1"), Enabled: true},
}, nil
},
queueSummaryFn: func() (autoscan.QueueSummary, error) {
return autoscan.QueueSummary{Active: 3, Accepted: 2, Running: 1}, nil
},
listRunningFn: func() ([]autoscan.Event, error) {
return []autoscan.Event{{
ID: 44,
SourceID: &sourceID,
PluginID: "silo.autoscan.cephfs",
CapabilityID: "cephfs",
StartedAt: runningStartedAt,
Status: autoscan.EventStatusRunning,
}}, nil
},
latestEventAtFn: func() (*time.Time, error) {
return &latestEventAt, nil
},
@@ -1296,6 +1316,16 @@ func TestAutoscanHandleStatusReturnsTrimmedSources(t *testing.T) {
if body.ActiveScans != 3 || body.AcceptedScans != 2 || body.RunningScans != 1 {
t.Errorf("queue summary = %+v", body)
}
if len(body.RunningPolls) != 1 {
t.Fatalf("running_polls = %+v", body.RunningPolls)
}
poll := body.RunningPolls[0]
if poll.ID != 44 || poll.SourceID == nil || *poll.SourceID != sourceID || poll.PluginID != "silo.autoscan.cephfs" || poll.CapabilityID != "cephfs" {
t.Fatalf("running poll identity = %+v", poll)
}
if poll.ElapsedMS <= 0 {
t.Fatalf("running poll elapsed_ms = %d, want > 0", poll.ElapsedMS)
}
if body.LatestEventAt == nil || !body.LatestEventAt.Equal(latestEventAt) {
t.Errorf("latest_event_at = %v, want %v", body.LatestEventAt, latestEventAt)
}
+3
View File
@@ -221,6 +221,9 @@ func (h *CatalogHandler) HandleGetCatalogFacetSearch(w http.ResponseWriter, r *h
writeError(w, http.StatusBadRequest, "bad_request", "limit must be a positive integer")
return
}
if n > 100 {
n = 100
}
limit = n
}
+1 -1
View File
@@ -497,7 +497,7 @@ func (h *CatalogResourceHandler) enrichItemDetail(r *http.Request, detail *catal
detail.SeasonUserData = h.items.getAggregateUserData(r, episodes)
}
}
case "movie", "episode":
case "movie", "episode", "audiobook":
detail.SeasonUserData = h.items.getLeafUserData(r, detail.ContentID)
applyEffectiveEditionPreference(detail.SeasonUserData, &detail.EffectiveVersionEditionKey)
}
+3 -1
View File
@@ -2036,8 +2036,10 @@ func (h *LibraryHandler) seedDefaultChain(ctx context.Context, libraryType strin
levels = []string{"series", "season", "episode"}
case "movies", "movie":
levels = []string{"movie"}
case "audiobooks", "audiobook":
levels = []string{"audiobook"}
case "mixed":
levels = []string{"movie", "series", "season", "episode"}
levels = []string{"movie", "series", "season", "episode", "audiobook"}
default:
return nil
}
+1 -1
View File
@@ -32,7 +32,7 @@ type SectionHandler struct {
StoreProvider userstore.UserStoreProvider
UserRepo *auth.UserRepository
DetailSvc *catalog.DetailService
Settings *catalog.ServerSettingsRepo
Settings catalog.SettingsStore
CollectionRepo *catalog.LibraryCollectionRepository
}
+1 -1
View File
@@ -13,7 +13,7 @@ const SectionsAllowProfileCustomSettingKey = "sections.allow_profile_custom_sect
// SectionSettingsHandler exposes GET/PUT for the sections-related server setting.
type SectionSettingsHandler struct {
Settings *catalog.ServerSettingsRepo
Settings catalog.SettingsStore
}
type sectionsSettingResponse struct {
+139
View File
@@ -0,0 +1,139 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"log/slog"
"net/http"
"time"
"github.com/google/uuid"
"github.com/Silo-Server/silo-server/internal/playback"
)
var ErrServerRestartAlreadyRequested = errors.New("server restart already requested")
type ServerControlHandler struct {
requestRestart func(ctx context.Context) error
commands *playback.CommandDispatcher
}
type serverRestartRequest struct {
Reason string `json:"reason"`
Title string `json:"title"`
Message string `json:"message"`
}
type serverRestartResponse struct {
Status string `json:"status"`
Message string `json:"message"`
NotifiedSessions int `json:"notified_sessions"`
}
func NewServerControlHandler(
requestRestart func(ctx context.Context) error,
commands *playback.CommandDispatcher,
) *ServerControlHandler {
return &ServerControlHandler{
requestRestart: requestRestart,
commands: commands,
}
}
// HandleRestart handles POST /admin/server/restart.
func (h *ServerControlHandler) HandleRestart(w http.ResponseWriter, r *http.Request) {
if h == nil || h.requestRestart == nil {
writeError(w, http.StatusServiceUnavailable, "service_unavailable", "Server restart is unavailable")
return
}
var req serverRestartRequest
if err := decodeOptionalJSONBody(r, &req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
notifiedSessions := h.notifyPlaybackSessions(req)
if err := h.requestRestart(context.Background()); err != nil {
if errors.Is(err, ErrServerRestartAlreadyRequested) {
writeJSON(w, http.StatusAccepted, serverRestartResponse{
Status: "already_requested",
Message: "Server restart is already in progress.",
NotifiedSessions: notifiedSessions,
})
return
}
slog.Error("server restart request failed", "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to request server restart")
return
}
writeJSON(w, http.StatusAccepted, serverRestartResponse{
Status: "restart_requested",
Message: "Server restart requested. The process will shut down gracefully.",
NotifiedSessions: notifiedSessions,
})
}
func (h *ServerControlHandler) notifyPlaybackSessions(req serverRestartRequest) int {
if h == nil || h.commands == nil {
return 0
}
title := req.Title
if title == "" {
title = "Server restarting"
}
message := req.Message
if message == "" {
message = "The server is restarting now. Playback may reconnect shortly."
}
payload, err := json.Marshal(map[string]string{
"title": title,
"message": message,
})
if err != nil {
slog.Warn("server restart: failed to build realtime payload", "error", err)
return 0
}
results := h.commands.DispatchToAll(func(session *playback.Session) (playback.CommandEnvelope, time.Duration, func(), error) {
if session == nil {
return playback.CommandEnvelope{}, 0, nil, playback.ErrCommandDispatchUnavailable
}
command, err := playback.NewCommandEnvelope(
session.ID,
uuid.NewString(),
playback.CommandServerRestarting,
payload,
)
if err != nil {
return playback.CommandEnvelope{}, 0, nil, err
}
command.Reason = req.Reason
if command.Reason == "" {
command.Reason = "server_restart_requested"
}
command.IssuedBy = &playback.CommandIssuedBy{Kind: "admin"}
return command, 0, nil, nil
})
delivered := 0
for _, result := range results {
if result.Delivered {
delivered++
continue
}
if result.DispatchErr != nil && !errors.Is(result.DispatchErr, playback.ErrRealtimeConnectionNotFound) {
slog.Warn("server restart: failed to notify playback session",
"session_id", result.SessionID,
"error", result.DispatchErr,
)
}
}
return delivered
}
@@ -0,0 +1,156 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/playback"
)
type serverControlTestConn struct {
messages []any
}
func (c *serverControlTestConn) WriteJSON(v any) error {
c.messages = append(c.messages, v)
return nil
}
func TestServerControlRestartRequestsShutdown(t *testing.T) {
t.Parallel()
called := 0
handler := NewServerControlHandler(func(context.Context) error {
called++
return nil
}, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
if called != 1 {
t.Fatalf("restart calls = %d, want 1", called)
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.Status != "restart_requested" {
t.Fatalf("status = %q, want restart_requested", resp.Status)
}
if resp.NotifiedSessions != 0 {
t.Fatalf("notified_sessions = %d, want 0", resp.NotifiedSessions)
}
}
func TestServerControlRestartUnavailable(t *testing.T) {
t.Parallel()
handler := NewServerControlHandler(nil, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
}
func TestServerControlRestartAlreadyRequested(t *testing.T) {
t.Parallel()
handler := NewServerControlHandler(func(context.Context) error {
return ErrServerRestartAlreadyRequested
}, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.Status != "already_requested" {
t.Fatalf("status = %q, want already_requested", resp.Status)
}
}
func TestServerControlRestartNotifiesPlaybackSessions(t *testing.T) {
t.Parallel()
sessionMgr := playback.NewSessionManager(0, 0)
session, err := sessionMgr.StartSession(1, "profile-1", 100, playback.PlayDirect, false)
if err != nil {
t.Fatalf("StartSession: %v", err)
}
realtimeHub := playback.NewRealtimeHub()
conn := &serverControlTestConn{}
registration := realtimeHub.Register(session.ID, conn)
if registration == nil {
t.Fatal("expected realtime registration")
}
defer realtimeHub.Unregister(registration)
dispatcher := playback.NewCommandDispatcher(sessionMgr, realtimeHub, nil)
handler := NewServerControlHandler(func(context.Context) error {
return nil
}, dispatcher)
req := httptest.NewRequest(
http.MethodPost,
"/admin/server/restart",
strings.NewReader(`{"reason":"maintenance","message":"Restarting for maintenance"}`),
)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.NotifiedSessions != 1 {
t.Fatalf("notified_sessions = %d, want 1", resp.NotifiedSessions)
}
if len(conn.messages) != 1 {
t.Fatalf("messages = %d, want 1", len(conn.messages))
}
command, ok := conn.messages[0].(playback.CommandEnvelope)
if !ok {
t.Fatalf("message type = %T, want playback.CommandEnvelope", conn.messages[0])
}
if command.Name != playback.CommandServerRestarting {
t.Fatalf("command name = %q, want %q", command.Name, playback.CommandServerRestarting)
}
if command.Reason != "maintenance" {
t.Fatalf("reason = %q, want maintenance", command.Reason)
}
var payload map[string]string
if err := json.Unmarshal(command.Payload, &payload); err != nil {
t.Fatalf("decode payload: %v", err)
}
if payload["message"] != "Restarting for maintenance" {
t.Fatalf("payload message = %q, want custom message", payload["message"])
}
}
+4 -2
View File
@@ -17,14 +17,16 @@ import (
type SystemHandler struct {
transcodePool *nodepool.TranscodePool
jwtSecret string
ffmpegPath string
buildInfo buildinfo.Info
}
// NewSystemHandler creates a SystemHandler.
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string) *SystemHandler {
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string, ffmpegPath string) *SystemHandler {
return &SystemHandler{
transcodePool: transcodePool,
jwtSecret: jwtSecret,
ffmpegPath: ffmpegPath,
buildInfo: buildinfo.Current(),
}
}
@@ -45,7 +47,7 @@ func (h *SystemHandler) HandleHWAccel(w http.ResponseWriter, r *http.Request) {
}
}
info := playback.DetectHWAccel()
info := playback.DetectHWAccelWithFFmpeg(h.ffmpegPath)
writeJSON(w, http.StatusOK, info)
}
+5 -5
View File
@@ -32,11 +32,11 @@ type tasteSeedSubmitResponse struct {
// HandleTasteSeedItems handles GET /recommendations/taste-seed/items.
//
// Returns a paginated, hydrated list of "popular on this server" posters used
// for the new-user taste-seeding picker. Blends server-watched popularity with
// TMDB rating so fresh servers (no watch history yet) still surface meaningful
// content. The user_state field carries the existing is_favorite flag, so the
// UI can pre-select items the profile already favorited.
// Returns a paginated, hydrated list of posters used for the new-user
// taste-seeding picker. Blends server-watched popularity with rating reliability
// and recency so fresh servers (no watch history yet) still surface recognizable
// content. The user_state field carries the existing is_favorite flag, so the UI
// can pre-select items the profile already favorited.
func (h *RecommendationsHandler) HandleTasteSeedItems(w http.ResponseWriter, r *http.Request) {
if h.recsRepo == nil || h.Fetcher == nil {
writeJSON(w, http.StatusOK, tasteSeedItemsResponse{Items: []sectionItemResponse{}})
+46 -27
View File
@@ -53,6 +53,7 @@ import (
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/subtitles"
subtitleai "github.com/Silo-Server/silo-server/internal/subtitles/ai"
@@ -76,6 +77,7 @@ type Dependencies struct {
BootstrapSensitiveValues map[string]string
AppContext context.Context
DB *pgxpool.Pool
SecretCipher *secret.Cipher // at-rest credential cipher (required when DB is set)
FrontendFS fs.FS
S3Public *s3client.Client // public assets bucket client (may be nil)
S3Private *s3client.Client // private internal bucket client (may be nil)
@@ -143,6 +145,7 @@ type Dependencies struct {
PlaybackRealtimeHub *playback.RealtimeHub
OnUserSessionsRevoked func(ctx context.Context, userID int)
OnServerSettingUpdated func(ctx context.Context, key, value string)
RequestServerRestart func(ctx context.Context) error
// UserCollectionSync handles per-profile imported collections (TMDB /
// Trakt / MDBList) — the user-facing analogue of CollectionService.
@@ -164,9 +167,6 @@ type Dependencies struct {
// clients hitting /login, /api/*, /abs/api/*, and /abs/socket.io/* all
// resolve correctly. May be nil; no ABS routes are registered in that case.
ABSHandler absHandler
// AudiobooksEnabled gates native audiobook admin/UI routes behind the same
// server setting as the ABS compatibility listener.
AudiobooksEnabled bool
}
// absHandler is the narrow interface the router needs from the ABS handler.
@@ -225,9 +225,11 @@ func NewRouter(deps Dependencies) chi.Router {
healthHandler := handlers.NewHealthHandler(healthServerName, healthServerID)
// Build server settings repo if DB is available (needed by auth and admin).
var settingsRepo *catalog.ServerSettingsRepo
// Wrap it in the encrypting decorator so sensitive keys rest as ciphertext
// and every consumer transparently reads plaintext.
var settingsRepo catalog.SettingsStore
if deps.DB != nil {
settingsRepo = catalog.NewServerSettingsRepo(deps.DB)
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(deps.DB), deps.SecretCipher)
}
// Build auth handler and auth middleware if DB and config are available.
@@ -433,27 +435,25 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.Config != nil {
tmdbAPIKey = deps.Config.TMDBAPIKey
}
requestsRepo := mediarequests.NewRepository(deps.DB)
requestsRepo := mediarequests.NewRepository(deps.DB, deps.SecretCipher)
requestSvc := mediarequests.NewService(
requestsRepo,
tmdb.NewClient(tmdbAPIKey, 40),
mediarequests.NewCatalogPresence(itemRepo, providerIDRepo),
)
requestSvc.SetSecretResolver(settingsRepo)
requestSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if viewerResolver != nil {
requestSvc.SetEntitlementResolver(mediarequests.NewAccessEntitlements(viewerResolver))
}
requestHandler = handlers.NewRequestsHandler(requestSvc)
autoscanRepo := autoscan.NewRepository(deps.DB)
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && deps.PluginService != nil {
autoscanSvc := BuildAutoscanService(
autoscanRepo,
deps.PluginService,
plugins.NewInstallationStore(deps.DB),
requestsRepo,
settingsRepo,
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -568,7 +568,7 @@ func NewRouter(deps Dependencies) chi.Router {
// Create subtitleRepo early — only needs DB, shared with playback handler and subtitle search handler.
var subtitleRepo *subtitles.PgRepository
if deps.DB != nil {
subtitleRepo = subtitles.NewPgRepository(deps.DB)
subtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
}
// Notifier that pushes "subtitle ready" events to active sessions when an AI
@@ -579,8 +579,10 @@ func NewRouter(deps Dependencies) chi.Router {
// Build playback handler if session manager is available.
var playbackHandler *handlers.PlaybackHandler
var adminPlaybackControlHandler *handlers.AdminPlaybackControlHandler
var playbackCommandDispatcher *playback.CommandDispatcher
var streamHandler *handlers.StreamHandler
var watchTogetherHandler *handlers.WatchTogetherHandler
var audioHandler *handlers.AudioHandler
if deps.SessionMgr != nil {
var playbackAdminStore handlers.PlaybackAdminStore
if deps.DB != nil {
@@ -662,6 +664,7 @@ func NewRouter(deps Dependencies) chi.Router {
playbackHandler.RealtimeHub = realtimeHub
playbackHandler.CommandTracker = commandTracker
playbackHandler.CommandDispatcher = playback.NewCommandDispatcher(deps.SessionMgr, realtimeHub, commandTracker)
playbackCommandDispatcher = playbackHandler.CommandDispatcher
playbackHandler.IntroAnalyzer = deps.IntroAnalyzer
playbackHandler.IntroRepository = deps.IntroRepository
playbackHandler.MarkerRegistry = deps.MarkerRegistry
@@ -689,6 +692,14 @@ func NewRouter(deps Dependencies) chi.Router {
)
}
}
if detailSvc != nil && deps.FileRepo != nil && deps.UserStoreProvider != nil {
audioHandler = handlers.NewAudioHandler(
detailSvc,
deps.FileRepo,
deps.UserStoreProvider,
handlers.NewPGAudioBookmarkStore(deps.DB),
)
}
// Wire subtitle repo and S3 client onto streamHandler for S3-stored subtitle serving.
if streamHandler != nil && subtitleRepo != nil && deps.S3Public != nil {
@@ -700,6 +711,8 @@ func NewRouter(deps Dependencies) chi.Router {
streamHandler.FFmpegPath = deps.Config.Playback.FFmpegPath
}
serverControlHandler := handlers.NewServerControlHandler(deps.RequestServerRestart, playbackCommandDispatcher)
// Build admin handler if we have a user repo.
var adminHandler *handlers.AdminHandler
var catalogSeedHandler *handlers.CatalogSeedHandler
@@ -998,6 +1011,7 @@ func NewRouter(deps Dependencies) chi.Router {
libraryCollectionService.TraktTokenResolver = &traktCollectionTokenResolver{
pool: deps.DB,
settings: settingsRepo,
cipher: deps.SecretCipher,
provider: watchtrakt.NewProvider(nil, ""),
}
}
@@ -1157,7 +1171,7 @@ func NewRouter(deps Dependencies) chi.Router {
var historyImportHandler *handlers.HistoryImportHandler
var historyImportSvc *historyimport.Service
if deps.DB != nil {
historyRepo := historyimport.NewRepository(deps.DB)
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyImportSvc = historyimport.NewService(deps.AppContext, historyRepo, deps.UserStoreProvider)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, providerIDRepo)
historyImportSvc.SetStableIdentityResolver(historyIdentity)
@@ -1166,7 +1180,7 @@ func NewRouter(deps Dependencies) chi.Router {
}
historyImportHandler = handlers.NewHistoryImportHandler(historyImportSvc)
if deps.UserStoreProvider != nil {
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB), historyRepo, deps.UserStoreProvider)
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB, deps.SecretCipher), historyRepo, deps.UserStoreProvider)
webhookSyncSvc.SetStableIdentityResolver(historyIdentity)
webhookSyncHandler = handlers.NewWebhookSyncHandler(webhookSyncSvc)
}
@@ -1765,6 +1779,22 @@ func NewRouter(deps Dependencies) chi.Router {
})
}
if audioHandler != nil {
r.Route("/audio", func(r chi.Router) {
r.Group(func(r chi.Router) {
r.Use(apimw.RequireProfile)
r.Post("/playback/start", audioHandler.HandleStartPlayback)
r.Patch("/playback/{session_id}/sync", audioHandler.HandleSyncPlayback)
r.Post("/playback/{session_id}/close", audioHandler.HandleClosePlayback)
r.Get("/items/{content_id}/bookmarks", audioHandler.HandleListBookmarks)
r.Post("/items/{content_id}/bookmarks", audioHandler.HandleCreateBookmark)
r.Delete("/items/{content_id}/bookmarks/{time_seconds}", audioHandler.HandleDeleteBookmark)
})
r.Get("/playback/{session_id}/tracks/{track_index}", audioHandler.HandleStreamTrack)
r.Head("/playback/{session_id}/tracks/{track_index}", audioHandler.HandleStreamTrack)
})
}
if watchTogetherHandler != nil {
r.Route("/watch-together", func(r chi.Router) {
r.Get("/rooms/{room_id}/ws", watchTogetherHandler.HandleRoomWebSocket)
@@ -1809,20 +1839,6 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/sections/recipes", recipeHandler.HandleList)
r.Get("/sections/recipes/{type}/candidates", recipeHandler.HandleCandidates)
// Audiobook endpoints (no profile required — catalog-level list/detail).
if deps.AudiobooksEnabled && itemRepo != nil {
audiobookHandler := &handlers.AudiobookHandler{
Items: itemRepo,
Files: deps.FileRepo,
Detail: detailSvc,
StoreProvider: deps.UserStoreProvider,
Recs: recsRepoForStale,
}
r.Get("/audiobooks", audiobookHandler.HandleListAudiobooks)
r.Get("/audiobooks/{id}", audiobookHandler.HandleGetAudiobook)
r.Post("/audiobooks/{id}/progress", audiobookHandler.HandleReportAudiobookProgress)
}
// Section endpoints (profile-scoped).
if sectionHandler != nil {
r.Group(func(r chi.Router) {
@@ -1920,6 +1936,7 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/playback-history", adminHandler.HandleListPlaybackHistory)
r.Get("/unmatched", adminHandler.HandleListUnmatched)
r.Get("/stats", adminHandler.HandleGetStats)
r.Post("/server/restart", serverControlHandler.HandleRestart)
r.Get("/settings/sensitive-status", adminHandler.HandleGetSensitiveStatus)
r.Post("/settings/check/{kind}", adminHandler.HandleCheckSettingsConnection)
if sectionSettingsHandler != nil {
@@ -2113,10 +2130,12 @@ func NewRouter(deps Dependencies) chi.Router {
// System inspection.
{
sysJWTSecret := ""
sysFFmpegPath := ""
if deps.Config != nil {
sysJWTSecret = deps.Config.Auth.JWTSecret
sysFFmpegPath = deps.Config.Playback.FFmpegPath
}
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret)
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret, sysFFmpegPath)
r.Route("/system", func(r chi.Router) {
r.Get("/build", systemHandler.HandleBuildInfo)
r.Get("/hw-accel", systemHandler.HandleHWAccel)
@@ -11,13 +11,15 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/watchsync"
watchtrakt "github.com/Silo-Server/silo-server/internal/watchsync/providers/trakt"
)
type traktCollectionTokenResolver struct {
pool *pgxpool.Pool
settings *catalog.ServerSettingsRepo
settings catalog.SettingsStore // encrypting decorator: decrypts trakt client_secret
cipher *secret.Cipher // decrypts watch_provider_connections tokens read via raw SQL
provider *watchtrakt.Provider
}
@@ -97,7 +99,7 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
LIMIT 1
`, profileID)
var conn watchsync.Connection
if err := row.Scan(
err := row.Scan(
&conn.ID,
&conn.Provider,
&conn.UserID,
@@ -107,24 +109,45 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
&conn.AccessToken,
&conn.RefreshToken,
&conn.TokenExpiresAt,
); err != nil {
)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return watchsync.Connection{}, errors.New("trakt connection not found for profile")
}
return watchsync.Connection{}, fmt.Errorf("load trakt connection: %w", err)
}
// This resolver reads watch_provider_connections directly (bypassing the
// watchsync repo), so it must apply the same at-rest decryption the repo
// does, bound to the same connection identity (watchsync.TokenAAD).
if conn.AccessToken, err = r.cipher.DecryptIfEncrypted(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt access token: %w", err)
}
if conn.RefreshToken, err = r.cipher.DecryptIfEncrypted(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt refresh token: %w", err)
}
return conn, nil
}
func (r *traktCollectionTokenResolver) updateTokens(ctx context.Context, conn watchsync.Connection) error {
_, err := r.pool.Exec(ctx, `
// Encrypt the refreshed tokens inline, bound to the connection identity, so
// the raw-SQL write matches what the watchsync repo (and this resolver's read
// path) store.
accessToken, err := r.cipher.Encrypt(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt refresh token: %w", err)
}
_, err = r.pool.Exec(ctx, `
UPDATE watch_provider_connections
SET access_token = $2,
refresh_token = $3,
token_expires_at = $4,
updated_at = now()
WHERE id = $1::uuid
`, conn.ID, conn.AccessToken, conn.RefreshToken, conn.TokenExpiresAt)
`, conn.ID, accessToken, refreshToken, conn.TokenExpiresAt)
if err != nil {
return fmt.Errorf("update refreshed trakt connection tokens: %w", err)
}
@@ -18,7 +18,8 @@ func (h *Handler) handleAuthorDetail(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
slog.Warn("abs author access resolution failed", "err", err, "id", id)
http.Error(w, "forbidden", http.StatusForbidden)
return
}
author, err := h.deps.MediaStore.GetAuthorByID(r.Context(), id, access)
@@ -47,7 +48,8 @@ func (h *Handler) handleSeriesDetail(w http.ResponseWriter, r *http.Request) {
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
slog.Warn("abs series access resolution failed", "err", err, "id", id)
http.Error(w, "forbidden", http.StatusForbidden)
return
}
series, err := h.deps.MediaStore.GetSeriesByName(r.Context(), id, access)
@@ -44,7 +44,9 @@ func TestAuthor_Detail_ReturnsBooks(t *testing.T) {
t.Fatalf("status = %d", rec.Code)
}
var got map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &got)
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
if got["name"] != "Brandon Sanderson" {
t.Errorf("name = %v", got["name"])
}
@@ -78,7 +80,9 @@ func TestSeries_Detail_ReturnsBooks(t *testing.T) {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &got)
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
if got["name"] != "Mistborn" {
t.Errorf("name = %v", got["name"])
}
+6 -1
View File
@@ -61,7 +61,12 @@ func (h *Handler) handleUpsertBookmark(reason string) http.HandlerFunc {
return
}
item, err := h.deps.MediaStore.GetAudiobookByID(r.Context(), itemID, access)
if err != nil || item == nil {
if err != nil {
slog.Error("abs bookmark item lookup failed", "err", err, "user", a.UserID, "item", itemID)
http.Error(w, "item lookup failed", http.StatusInternalServerError)
return
}
if item == nil {
http.Error(w, "item not found", http.StatusNotFound)
return
}
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"sort"
@@ -157,10 +158,14 @@ func (p *recordingPublisher) snapshot() []publishedEvent {
// handler tests can drive both the 200 and 404 branches.
type stubMediaStore struct {
noopMediaStore
known map[string]*models.MediaItem // itemID → row (nil means "exists but no row needed")
known map[string]*models.MediaItem // itemID → row (nil means "exists but no row needed")
lookupErr error
}
func (s *stubMediaStore) GetAudiobookByID(_ context.Context, id string, _ catalog.AccessFilter) (*models.MediaItem, error) {
if s.lookupErr != nil {
return nil, s.lookupErr
}
if it, ok := s.known[id]; ok {
if it == nil {
return &models.MediaItem{ContentID: id}, nil
@@ -411,6 +416,19 @@ func TestMissingItem_404(t *testing.T) {
}
}
func TestItemLookup_ErrorFromMediaStore_500(t *testing.T) {
hb := newBookmarksHarness(t, "book-1")
hb.H.deps.MediaStore = &stubMediaStore{
known: map[string]*models.MediaItem{"book-1": nil},
lookupErr: errors.New("media lookup failed"),
}
rec := dispatchBookmark(hb.H, http.MethodPost, "/api/me/item/book-1/bookmark", "book-1", "", []byte(`{"title":"x","time":1}`), "1", "", hb.H.handleUpsertBookmark("bookmark_created"))
if rec.Code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500; body=%s", rec.Code, rec.Body.String())
}
}
func TestInvalidBody_400(t *testing.T) {
hb := newBookmarksHarness(t, "book-1")
rec := dispatchBookmark(hb.H, http.MethodPost, "/api/me/item/book-1/bookmark", "book-1", "", []byte(`{not json`), "1", "", hb.H.handleUpsertBookmark("bookmark_created"))
+14 -3
View File
@@ -6,6 +6,7 @@ import (
"io"
"log/slog"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/oklog/ulid/v2"
@@ -39,7 +40,12 @@ func (h *Handler) handleCreateCollection(w http.ResponseWriter, r *http.Request)
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
if body.Name == nil || *body.Name == "" {
if body.Name == nil {
http.Error(w, "name required", http.StatusBadRequest)
return
}
name := strings.TrimSpace(*body.Name)
if name == "" {
http.Error(w, "name required", http.StatusBadRequest)
return
}
@@ -48,7 +54,7 @@ func (h *Handler) handleCreateCollection(w http.ResponseWriter, r *http.Request)
ID: ulid.Make().String(),
UserID: a.UserID,
ProfileID: a.ProfileID,
Name: *body.Name,
Name: name,
}
if body.Description != nil {
c.Description = *body.Description
@@ -267,7 +273,12 @@ func (h *Handler) handleUpdateCollection(w http.ResponseWriter, r *http.Request)
return
}
if body.Name != nil {
c.Name = *body.Name
name := strings.TrimSpace(*body.Name)
if name == "" {
http.Error(w, "name required", http.StatusBadRequest)
return
}
c.Name = name
}
if body.Description != nil {
c.Description = *body.Description
@@ -25,7 +25,7 @@ import (
// and profile_id; we mirror that so List can filter correctly).
type memCollectionStore struct {
mu sync.Mutex
rows map[string]Collection // id -> row
rows map[string]Collection // id -> row
items map[string][]CollectionItem // collection_id -> items
}
@@ -281,8 +281,16 @@ func TestCollection_List_DoesNotLeakOtherUsers(t *testing.T) {
// User 2 lists.
rec := dispatchABSWithParams(http.MethodGet, "/api/collections", nil, nil, "2", "", hb.H.handleListCollections)
var env map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &env)
list, _ := env["collections"].([]any)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if err := json.Unmarshal(rec.Body.Bytes(), &env); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
list, ok := env["collections"].([]any)
if !ok {
t.Fatalf("response missing 'collections' key; body=%s", rec.Body.String())
}
if len(list) != 0 {
t.Errorf("user 2 sees %d collections, want 0", len(list))
}
@@ -295,8 +303,16 @@ func TestCollection_List_ProfileIsolation(t *testing.T) {
_ = dispatchABSWithParams(http.MethodPost, "/api/collections", nil, []byte(`{"name":"A"}`), "1", pA, hb.H.handleCreateCollection)
rec := dispatchABSWithParams(http.MethodGet, "/api/collections", nil, nil, "1", pB, hb.H.handleListCollections)
var env map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &env)
list, _ := env["collections"].([]any)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if err := json.Unmarshal(rec.Body.Bytes(), &env); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
list, ok := env["collections"].([]any)
if !ok {
t.Fatalf("response missing 'collections' key; body=%s", rec.Body.String())
}
if len(list) != 0 {
t.Errorf("profile B sees %d collections, want 0", len(list))
}
@@ -32,7 +32,12 @@ func (h *Handler) setHideFromContinue(w http.ResponseWriter, r *http.Request, hi
return
}
item, err := h.deps.MediaStore.GetAudiobookByID(r.Context(), itemID, access)
if err != nil || item == nil {
if err != nil {
slog.Error("abs continue item lookup failed", "err", err, "user", a.UserID, "item", itemID)
http.Error(w, "item lookup failed", http.StatusInternalServerError)
return
}
if item == nil {
http.Error(w, "item not found", http.StatusNotFound)
return
}
@@ -3,6 +3,7 @@ package abs
import (
"context"
"encoding/json"
"errors"
"net/http"
"sync"
"testing"
@@ -73,3 +74,18 @@ func TestContinue_UnknownItem_404(t *testing.T) {
t.Errorf("status = %d, want 404", rec.Code)
}
}
func TestContinue_MediaLookupError_500(t *testing.T) {
prog := &recordingProgressFake{}
media := &stubMediaStore{
known: map[string]*models.MediaItem{"book-1": nil},
lookupErr: errors.New("media lookup failed"),
}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
rec := dispatchABSWithParams(http.MethodGet, "/api/me/progress/book-1/remove-from-continue-listening",
map[string]string{"itemId": "book-1"}, nil, "1", "", h.handleRemoveFromContinueListening)
if rec.Code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500; body=%s", rec.Code, rec.Body.String())
}
}
+13 -9
View File
@@ -151,14 +151,18 @@ func (h *Handler) handleDeleteItemProgress(w http.ResponseWriter, r *http.Reques
// returns the empty progress shape so the client can store offline state
// without raising an error.
func (h *Handler) handleSetEpisodeProgress(w http.ResponseWriter, r *http.Request) {
if a, ok := absAuthFrom(r); !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
writeJSON(w, http.StatusOK, map[string]any{
"libraryItemId": chi.URLParam(r, "libraryItemId"),
"episodeId": chi.URLParam(r, "episodeId"),
"currentTime": 0.0,
"duration": 0.0,
"isFinished": false,
"progress": 0.0,
"lastUpdate": 0,
"libraryItemId": chi.URLParam(r, "libraryItemId"),
"episodeId": chi.URLParam(r, "episodeId"),
"currentTime": 0.0,
"duration": 0.0,
"isFinished": false,
"progress": 0.0,
"lastUpdate": 0,
})
}
@@ -182,8 +186,8 @@ func (h *Handler) handleEbookFile(w http.ResponseWriter, _ *http.Request) {
// optimistic update succeeds.
func (h *Handler) handleEbookStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{
"libraryItemId": chi.URLParam(r, "id"),
"fileId": chi.URLParam(r, "fileid"),
"libraryItemId": chi.URLParam(r, "id"),
"fileId": chi.URLParam(r, "fileid"),
"isSupplementary": false,
})
}
+8 -4
View File
@@ -13,16 +13,20 @@ import (
)
const (
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
)
// ABSConfigProvider implements abs.ConfigProvider using silo's server_settings
// table. The ABS JWT secret is generated once on first read and persisted for
// the lifetime of the deployment.
type ABSConfigProvider struct {
Settings *catalog.ServerSettingsRepo
// Settings is the encrypting settings decorator in production, so the ABS
// JWT secret (a SensitiveSettingKey) rests as ciphertext and is transparently
// decrypted here. Typed as the interface so both the raw repo and the
// decorator satisfy it.
Settings catalog.SettingsStore
// secretCache holds the generated secret so we only hit the DB once per
// process lifetime. Protected by mu.
+10 -13
View File
@@ -21,10 +21,8 @@ type SettingsReader interface {
GetString(ctx context.Context, key string) (string, error)
}
// Service is the audiobooks feature's top-level orchestrator. Sub-plan 1
// exposes only Enabled(); subsequent sub-plans hang additional methods
// off Service as new capabilities (scanner branches, ABS handlers, etc.)
// come online.
// Service owns the Audiobookshelf compatibility adapters. Native audiobook
// libraries are normal catalog media_items and do not depend on this service.
type Service struct {
settings SettingsReader
ABSHandler *abs.Handler
@@ -44,8 +42,8 @@ type ABSHandlerDeps struct {
Pool *pgxpool.Pool
Items *catalog.ItemRepository
Files *scanner.FileRepository
Settings *catalog.ServerSettingsRepo
Auth absAuthAdapter // see BuildABSHandler below
Settings catalog.SettingsStore // encrypting decorator in production
Auth absAuthAdapter // see BuildABSHandler below
AccessResolver abs.AccessResolver
// Recs is the recommendations repository used to power the
// /items/{id}/similar endpoint via embedding nearest-neighbor search.
@@ -181,17 +179,16 @@ func buildABSRecommender(deps ABSHandlerDeps) abs.Recommender {
return &ABSRecommender{Pool: deps.Pool, Recs: deps.Recs}
}
// Enabled reports whether the audiobooks feature flag (set by
// 160_audiobooks_feature_flag and toggled by operators) is currently true.
// Any value other than the literal string "true" reads as false; this matches how silo
// treats other boolean server_settings rows.
func (s *Service) Enabled(ctx context.Context) (bool, error) {
// ABSCompatEnabled reports whether the Audiobookshelf compatibility listener
// should be started. Native audiobook libraries intentionally do not read this
// setting.
func (s *Service) ABSCompatEnabled(ctx context.Context) (bool, error) {
if s == nil || s.settings == nil {
return false, nil
}
value, err := s.settings.GetString(ctx, "audiobooks.enabled")
value, err := s.settings.GetString(ctx, "audiobookshelf_compat.enabled")
if err != nil {
return false, fmt.Errorf("read audiobooks.enabled: %w", err)
return false, fmt.Errorf("read audiobookshelf_compat.enabled: %w", err)
}
return value == "true", nil
}
+16 -16
View File
@@ -12,13 +12,13 @@ type fakeSettingsReader struct {
}
func (f *fakeSettingsReader) GetString(_ context.Context, key string) (string, error) {
if key != "audiobooks.enabled" {
if key != "audiobookshelf_compat.enabled" {
return "", errors.New("unexpected key: " + key)
}
return f.value, f.err
}
func TestServiceEnabledReadsFlag(t *testing.T) {
func TestServiceABSCompatEnabledReadsFlag(t *testing.T) {
cases := []struct {
name string
stored string
@@ -33,44 +33,44 @@ func TestServiceEnabledReadsFlag(t *testing.T) {
tc := tc
t.Run(tc.name, func(t *testing.T) {
svc := New(&fakeSettingsReader{value: tc.stored})
got, err := svc.Enabled(context.Background())
got, err := svc.ABSCompatEnabled(context.Background())
if err != nil {
t.Fatalf("Enabled returned error: %v", err)
t.Fatalf("ABSCompatEnabled returned error: %v", err)
}
if got != tc.want {
t.Fatalf("Enabled = %v, want %v", got, tc.want)
t.Fatalf("ABSCompatEnabled = %v, want %v", got, tc.want)
}
})
}
}
func TestServiceEnabledPropagatesError(t *testing.T) {
func TestServiceABSCompatEnabledPropagatesError(t *testing.T) {
wantErr := errors.New("db down")
svc := New(&fakeSettingsReader{err: wantErr})
_, err := svc.Enabled(context.Background())
_, err := svc.ABSCompatEnabled(context.Background())
if !errors.Is(err, wantErr) {
t.Fatalf("Enabled error = %v, want %v wrapped", err, wantErr)
t.Fatalf("ABSCompatEnabled error = %v, want %v wrapped", err, wantErr)
}
}
func TestServiceEnabledNilReceiverReturnsFalse(t *testing.T) {
func TestServiceABSCompatEnabledNilReceiverReturnsFalse(t *testing.T) {
var svc *Service
got, err := svc.Enabled(context.Background())
got, err := svc.ABSCompatEnabled(context.Background())
if err != nil {
t.Fatalf("Enabled returned error: %v", err)
t.Fatalf("ABSCompatEnabled returned error: %v", err)
}
if got {
t.Fatal("Enabled = true, want false")
t.Fatal("ABSCompatEnabled = true, want false")
}
}
func TestServiceEnabledNilSettingsReturnsFalse(t *testing.T) {
func TestServiceABSCompatEnabledNilSettingsReturnsFalse(t *testing.T) {
svc := New(nil)
got, err := svc.Enabled(context.Background())
got, err := svc.ABSCompatEnabled(context.Background())
if err != nil {
t.Fatalf("Enabled returned error: %v", err)
t.Fatalf("ABSCompatEnabled returned error: %v", err)
}
if got {
t.Fatal("Enabled = true, want false")
t.Fatal("ABSCompatEnabled = true, want false")
}
}
+13 -27
View File
@@ -13,31 +13,27 @@ type ResolvedConnection struct {
}
// RequestIntegrationLookup resolves a soft-linked Requests integration to its
// base URL and encrypted api-key reference.
// base URL and (already-decrypted) api key.
type RequestIntegrationLookup interface {
Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error)
}
// SecretResolver resolves an encrypted api-key reference to its plaintext value.
type SecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error)
}
type ConnectionResolver struct {
requests RequestIntegrationLookup
secrets SecretResolver
}
func NewConnectionResolver(r RequestIntegrationLookup, s SecretResolver) *ConnectionResolver {
return &ConnectionResolver{requests: r, secrets: s}
func NewConnectionResolver(r RequestIntegrationLookup) *ConnectionResolver {
return &ConnectionResolver{requests: r}
}
// Resolve turns a stored Connection into concrete credentials. When the
// connection is linked to a Requests integration the live base URL + key ref are
// read from there; otherwise the connection's own fields are used. The api-key
// ref is then resolved to plaintext via the secrets resolver.
// connection is linked to a Requests integration the live base URL + key are
// read from there (the Requests repo decrypts it); otherwise the connection's
// own fields are used (the autoscan repo already decrypted api_key_ref on read).
// Either way the returned key is plaintext — there is no longer a separate
// secret-resolution step.
func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (ResolvedConnection, error) {
baseURL, apiKeyRef := c.BaseURL, c.APIKeyRef
baseURL, apiKey := c.BaseURL, c.APIKeyRef
// A pointer-to-empty-string request_integration_id is NOT a live link (it can
// arise from a both-NULL orphan or a stripped link). Treat empty/whitespace as
// "no link" so we don't call requests.Get("") and use the connection's own
@@ -46,21 +42,11 @@ func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (Resolv
if cr.requests == nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: no lookup configured", *c.RequestIntegrationID)
}
u, ref, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
u, key, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: %w", *c.RequestIntegrationID, err)
}
baseURL, apiKeyRef = u, ref
baseURL, apiKey = u, key
}
apiKey := strings.TrimSpace(apiKeyRef)
if cr.secrets != nil && apiKey != "" {
resolved, err := cr.secrets.Get(ctx, apiKey)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: resolve api key: %w", err)
}
if resolved = strings.TrimSpace(resolved); resolved != "" {
apiKey = resolved
}
}
return ResolvedConnection{BaseURL: baseURL, APIKey: apiKey}, nil
return ResolvedConnection{BaseURL: baseURL, APIKey: strings.TrimSpace(apiKey)}, nil
}
+20 -36
View File
@@ -6,9 +6,10 @@ import (
"testing"
)
// fakeRequestLookup implements RequestIntegrationLookup from a static map.
// fakeRequestLookup implements RequestIntegrationLookup from a static map. The
// returned apiKey is already plaintext (the Requests repo decrypts on read).
type fakeRequestLookup struct {
entries map[string]struct{ baseURL, apiKeyRef string }
entries map[string]struct{ baseURL, apiKey string }
err error
}
@@ -20,30 +21,17 @@ func (f fakeRequestLookup) Get(_ context.Context, id string) (string, string, er
if !ok {
return "", "", errors.New("integration not found: " + id)
}
return e.baseURL, e.apiKeyRef, nil
}
// fakeSecrets implements SecretResolver from a static map; unknown refs resolve
// to empty (so the ref itself is used).
type fakeSecrets struct {
values map[string]string
err error
}
func (f fakeSecrets) Get(_ context.Context, ref string) (string, error) {
if f.err != nil {
return "", f.err
}
return f.values[ref], nil
return e.baseURL, e.apiKey, nil
}
func TestResolveConnectionOwnCredentials(t *testing.T) {
secrets := fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
// The autoscan repo already decrypted api_key_ref, so the connection carries
// the literal key; Resolve returns it verbatim (trimmed).
r := NewConnectionResolver(fakeRequestLookup{})
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "ownref",
APIKeyRef: "OWNKEY",
})
if err != nil {
t.Fatalf("Resolve own: %v", err)
@@ -54,11 +42,10 @@ func TestResolveConnectionOwnCredentials(t *testing.T) {
}
func TestResolveConnectionLinked(t *testing.T) {
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKeyRef string }{
"req-1": {baseURL: "http://req:7878", apiKeyRef: "rk"},
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKey string }{
"req-1": {baseURL: "http://req:7878", apiKey: "REQKEY"},
}}
secrets := fakeSecrets{values: map[string]string{"rk": "REQKEY"}}
r := NewConnectionResolver(lookup, secrets)
r := NewConnectionResolver(lookup)
id := "req-1"
got, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id})
@@ -71,7 +58,7 @@ func TestResolveConnectionLinked(t *testing.T) {
}
func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{}, fakeSecrets{})
r := NewConnectionResolver(fakeRequestLookup{})
id := "gone"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when linked Requests integration is missing")
@@ -79,7 +66,7 @@ func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
}
func TestResolveConnectionLinkedLookupErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")}, fakeSecrets{})
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")})
id := "req-1"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when the lookup itself fails")
@@ -92,13 +79,12 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
// requests.Get(""). The lookup here errors on any call, so reaching it fails.
r := NewConnectionResolver(
fakeRequestLookup{err: errors.New("lookup must not be called")},
fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}},
)
for _, empty := range []string{"", " "} {
id := empty
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "ownref",
APIKeyRef: "OWNKEY",
RequestIntegrationID: &id,
})
if err != nil {
@@ -110,20 +96,18 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
}
}
func TestResolveConnectionTrimsAndFallsBackOnEmptySecret(t *testing.T) {
// Whitespace-padded ref; secret resolves to whitespace-only -> fall back to
// the trimmed ref (parity with requests.resolveAPIKey).
secrets := fakeSecrets{values: map[string]string{"ownref": " "}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
func TestResolveConnectionTrimsKey(t *testing.T) {
// Resolve trims surrounding whitespace from the (decrypted) key.
r := NewConnectionResolver(fakeRequestLookup{})
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: " ownref ",
APIKeyRef: " OWNKEY ",
})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if got.APIKey != "ownref" {
t.Fatalf("expected trimmed ref fallback, got %q", got.APIKey)
if got.APIKey != "OWNKEY" {
t.Fatalf("expected trimmed key, got %q", got.APIKey)
}
}
+9 -39
View File
@@ -9,19 +9,17 @@ import (
// enriched with the metadata the Add-source picker needs (plugin id + a
// human-friendly display name).
type DiscoveredSource struct {
InstallationID int
CapabilityID string
// PluginID is the installation's plugin id (e.g. "sonarr"); empty when the
// lister cannot supply it.
PluginID string
PluginID string
CapabilityID string
// DisplayName is a human-friendly label for the capability (from the
// capability's manifest display_name, falling back to plugin/capability ids).
DisplayName string
}
// ScanSourceLister enumerates every installed scan_source.v1 capability so the
// engine can (a) offer them in the Add-source picker and (b) detect orphaned
// source rows whose plugin has been uninstalled.
// engine can offer them in the Add-source picker.
type ScanSourceLister interface {
// ListScanSources returns one entry per installed scan_source.v1 capability,
// enriched with plugin id + display name.
@@ -31,17 +29,9 @@ type ScanSourceLister interface {
// AvailableScanSource is one installed scan_source capability an operator can
// create a source against (the Add-source picker list).
type AvailableScanSource struct {
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
PluginID string `json:"plugin_id"`
DisplayName string `json:"display_name"`
}
// installedKey identifies an installed scan_source capability for set
// membership tests (orphan detection in PollOnce).
type installedKey struct {
InstallationID int
CapabilityID string
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
DisplayName string `json:"display_name"`
}
// ListAvailableScanSources enumerates every installed scan_source capability so
@@ -59,30 +49,10 @@ func (s *Service) ListAvailableScanSources(ctx context.Context) ([]AvailableScan
out := make([]AvailableScanSource, 0, len(discovered))
for _, d := range discovered {
out = append(out, AvailableScanSource{
InstallationID: d.InstallationID,
CapabilityID: d.CapabilityID,
PluginID: d.PluginID,
DisplayName: d.DisplayName,
PluginID: d.PluginID,
CapabilityID: d.CapabilityID,
DisplayName: d.DisplayName,
})
}
return out, nil
}
// installedScanSources returns the set of currently-installed scan_source
// capabilities so PollOnce can skip orphaned source rows (their plugin is
// gone). A nil set means the set is unavailable (no lister) — callers must treat
// that as "discovery unavailable" and NOT prune.
func (s *Service) installedScanSources(ctx context.Context) (map[installedKey]struct{}, error) {
if s.lister == nil {
return nil, nil
}
discovered, err := s.lister.ListScanSources(ctx)
if err != nil {
return nil, fmt.Errorf("list scan sources: %w", err)
}
present := make(map[installedKey]struct{}, len(discovered))
for _, d := range discovered {
present[installedKey{InstallationID: d.InstallationID, CapabilityID: d.CapabilityID}] = struct{}{}
}
return present, nil
}
+3 -32
View File
@@ -14,8 +14,8 @@ func (f fakeLister) ListScanSources(context.Context) ([]DiscoveredSource, error)
func TestListAvailableScanSourcesEnumeratesInstalled(t *testing.T) {
lister := fakeLister{sources: []DiscoveredSource{
{InstallationID: 1, CapabilityID: "arr-a", PluginID: "sonarr", DisplayName: "Sonarr"},
{InstallationID: 2, CapabilityID: "arr-b", PluginID: "radarr", DisplayName: "Radarr"},
{PluginID: "sonarr", CapabilityID: "arr-a", DisplayName: "Sonarr"},
{PluginID: "radarr", CapabilityID: "arr-b", DisplayName: "Radarr"},
}}
svc := &Service{lister: lister}
@@ -26,7 +26,7 @@ func TestListAvailableScanSourcesEnumeratesInstalled(t *testing.T) {
if len(available) != 2 {
t.Fatalf("expected 2 available, got %d: %+v", len(available), available)
}
if available[0] != (AvailableScanSource{InstallationID: 1, CapabilityID: "arr-a", PluginID: "sonarr", DisplayName: "Sonarr"}) {
if available[0] != (AvailableScanSource{PluginID: "sonarr", CapabilityID: "arr-a", DisplayName: "Sonarr"}) {
t.Fatalf("unexpected first available: %+v", available[0])
}
}
@@ -41,32 +41,3 @@ func TestListAvailableScanSourcesNilListerEmpty(t *testing.T) {
t.Fatalf("nil lister must return empty, got %+v", available)
}
}
func TestInstalledScanSourcesSetMembership(t *testing.T) {
lister := fakeLister{sources: []DiscoveredSource{
{InstallationID: 1, CapabilityID: "arr-a"},
}}
svc := &Service{lister: lister}
present, err := svc.installedScanSources(context.Background())
if err != nil {
t.Fatalf("installedScanSources: %v", err)
}
if _, ok := present[installedKey{1, "arr-a"}]; !ok {
t.Fatalf("expected 1/arr-a present: %+v", present)
}
if _, ok := present[installedKey{2, "arr-b"}]; ok {
t.Fatalf("did not expect 2/arr-b present: %+v", present)
}
}
func TestInstalledScanSourcesNilListerNilSet(t *testing.T) {
svc := &Service{lister: nil}
present, err := svc.installedScanSources(context.Background())
if err != nil {
t.Fatalf("installedScanSources: %v", err)
}
if present != nil {
t.Fatalf("nil lister must return a nil set, got %+v", present)
}
}
+4
View File
@@ -5,3 +5,7 @@ import "errors"
// ErrNotFound is returned when an autoscan operation references a connection or
// source that does not exist. The admin API surfaces it as a 404.
var ErrNotFound = errors.New("autoscan: not found")
// ErrPollAlreadyRunning is returned when a source already has an active poll
// event. Callers should treat it as a no-op skip, not a source failure.
var ErrPollAlreadyRunning = errors.New("autoscan: source poll already running")
+2 -2
View File
@@ -69,7 +69,7 @@ func TestTestConnectionNoProbeConfigured(t *testing.T) {
func TestSuggestRewritesMatchesArrRootsToSiloFolders(t *testing.T) {
store := &fakeStore{
sources: []Source{
{ID: "src-1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1")},
{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1")},
},
}
arr := &fakeArrClient{roots: []string{"/data/tv"}}
@@ -87,7 +87,7 @@ func TestSuggestRewritesMatchesArrRootsToSiloFolders(t *testing.T) {
func TestSuggestRewritesNoConnection(t *testing.T) {
store := &fakeStore{
sources: []Source{{ID: "src-1", InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}},
sources: []Source{{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}},
}
svc := &Service{store: store, connres: passthroughConnRes{}, rootFolders: &fakeArrClient{}, folders: fakeFolderLister{}}
+5 -5
View File
@@ -9,7 +9,7 @@ import (
// ScanSourceProvider yields changed paths for one source. The engine calls
// PollChanges; production wraps the plugins.Service scan_source resolver.
type ScanSourceProvider interface {
PollChanges(ctx context.Context, installationID int, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) (changes []Change, nextMarker string, err error)
PollChanges(ctx context.Context, pluginID, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) (changes []Change, nextMarker string, err error)
}
// PollChangesClient is the slice of *pluginhost.ScanSourceClient used here. It
@@ -20,10 +20,10 @@ type PollChangesClient interface {
PollChanges(ctx context.Context, req *pluginv1.PollChangesRequest) (*pluginv1.PollChangesResponse, error)
}
// ScanSourceResolver yields a per-(installation, capability) scan-source client.
// ScanSourceResolver yields a per-(plugin, capability) scan-source client.
// Exported for the same cross-package adapter reason as PollChangesClient.
type ScanSourceResolver interface {
ScanSourceClient(ctx context.Context, installationID int, capabilityID string) (PollChangesClient, error)
ScanSourceClient(ctx context.Context, pluginID, capabilityID string) (PollChangesClient, error)
}
type pluginProvider struct{ resolver ScanSourceResolver }
@@ -34,8 +34,8 @@ func NewPluginProvider(resolver ScanSourceResolver) ScanSourceProvider {
return &pluginProvider{resolver: resolver}
}
func (p *pluginProvider) PollChanges(ctx context.Context, installationID int, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
client, err := p.resolver.ScanSourceClient(ctx, installationID, capabilityID)
func (p *pluginProvider) PollChanges(ctx context.Context, pluginID, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
client, err := p.resolver.ScanSourceClient(ctx, pluginID, capabilityID)
if err != nil {
return nil, "", err
}
+18 -7
View File
@@ -17,10 +17,17 @@ func (c *capturingClient) PollChanges(_ context.Context, req *pluginv1.PollChang
return &pluginv1.PollChangesResponse{SourcePaths: []string{"/mnt/media/x"}, NextMarker: "m2"}, nil
}
// capturingResolver yields a fixed PollChangesClient.
type capturingResolver struct{ client PollChangesClient }
// capturingResolver yields a fixed PollChangesClient and records the requested
// stable scan-source identity.
type capturingResolver struct {
client PollChangesClient
lastPluginID string
lastCapID string
}
func (r capturingResolver) ScanSourceClient(context.Context, int, string) (PollChangesClient, error) {
func (r *capturingResolver) ScanSourceClient(_ context.Context, pluginID, capabilityID string) (PollChangesClient, error) {
r.lastPluginID = pluginID
r.lastCapID = capabilityID
return r.client, nil
}
@@ -28,11 +35,12 @@ func (r capturingResolver) ScanSourceClient(context.Context, int, string) (PollC
// delivered to the plugin on the PollChangesRequest.
func TestPluginProviderPopulatesConnection(t *testing.T) {
client := &capturingClient{}
prov := NewPluginProvider(capturingResolver{client})
resolver := &capturingResolver{client: client}
prov := NewPluginProvider(resolver)
conn := ResolvedConnection{BaseURL: "https://arr.example", APIKey: "secret-key"}
sourceConfig := map[string]string{"exclusions": ".downloads"}
changes, next, err := prov.PollChanges(context.Background(), 1, "cap", "m1", conn, sourceConfig)
changes, next, err := prov.PollChanges(context.Background(), "silo.autoscan.arr", "cap", "m1", conn, sourceConfig)
if err != nil {
t.Fatalf("PollChanges: %v", err)
}
@@ -45,6 +53,9 @@ func TestPluginProviderPopulatesConnection(t *testing.T) {
if client.last.GetCapabilityId() != "cap" || client.last.GetMarker() != "m1" {
t.Fatalf("unexpected request fields: cap=%q marker=%q", client.last.GetCapabilityId(), client.last.GetMarker())
}
if resolver.lastPluginID != "silo.autoscan.arr" || resolver.lastCapID != "cap" {
t.Fatalf("resolver identity = %q/%q", resolver.lastPluginID, resolver.lastCapID)
}
rc := client.last.GetConnection()
if rc == nil {
t.Fatal("expected connection to be populated on the request")
@@ -59,9 +70,9 @@ func TestPluginProviderPopulatesConnection(t *testing.T) {
func TestPluginProviderPrefersStructuredChanges(t *testing.T) {
client := &capturingStructuredClient{}
prov := NewPluginProvider(capturingResolver{client: client})
prov := NewPluginProvider(&capturingResolver{client: client})
changes, next, err := prov.PollChanges(context.Background(), 1, "cap", "m1", ResolvedConnection{}, nil)
changes, next, err := prov.PollChanges(context.Background(), "silo.autoscan.arr", "cap", "m1", ResolvedConnection{}, nil)
if err != nil {
t.Fatalf("PollChanges: %v", err)
}
+181 -37
View File
@@ -9,14 +9,38 @@ import (
"time"
"unicode/utf8"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
type Repository struct{ pool *pgxpool.Pool }
type Repository struct {
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository { return &Repository{pool: pool} }
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// connectionAPIKeyAAD binds an autoscan_connections api_key_ref ciphertext to
// its row id.
func connectionAPIKeyAAD(id string) string {
return secret.RowAAD("autoscan_connections", "api_key_ref", id)
}
// encryptAPIKey encrypts a non-empty, trimmed key bound to the connection id;
// an empty key returns "" so NULL/keep-existing semantics are preserved.
func (r *Repository) encryptAPIKey(id, apiKey string) (string, error) {
apiKey = strings.TrimSpace(apiKey)
if apiKey == "" {
return "", nil
}
return r.cipher.Encrypt(apiKey, connectionAPIKeyAAD(id))
}
// --- Settings ---
@@ -64,7 +88,7 @@ func (r *Repository) UpdateSettings(ctx context.Context, s Settings) (Settings,
const connectionColumns = `id, name, kind, base_url, api_key_ref, request_integration_id`
func scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
func (r *Repository) scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
var c Connection
var baseURL, apiKeyRef, reqIntegrationID *string
if err := row.Scan(&c.ID, &c.Name, &c.Kind, &baseURL, &apiKeyRef, &reqIntegrationID); err != nil {
@@ -77,6 +101,13 @@ func scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
c.APIKeyRef = *apiKeyRef
}
c.RequestIntegrationID = reqIntegrationID
// Decrypt the stored key (read-path contract): legacy plaintext passes
// through, enc:v1: decrypts, corrupt ciphertext errors.
apiKey, err := r.cipher.DecryptIfEncrypted(c.APIKeyRef, connectionAPIKeyAAD(c.ID))
if err != nil {
return Connection{}, fmt.Errorf("decrypt autoscan connection %s api key: %w", c.ID, err)
}
c.APIKeyRef = apiKey
return c, nil
}
@@ -89,12 +120,22 @@ func nullable(s string) any {
}
func (r *Repository) CreateConnection(ctx context.Context, c Connection) (Connection, error) {
// Generate the id in Go (rather than the DB default) so the api_key_ref
// ciphertext can be AAD-bound to the row id at insert time.
id := strings.TrimSpace(c.ID)
if id == "" {
id = uuid.NewString()
}
apiKeyRef, err := r.encryptAPIKey(id, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_connections (name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5)
INSERT INTO autoscan_connections (id, name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING `+connectionColumns,
c.Name, c.Kind, nullable(c.BaseURL), nullable(c.APIKeyRef), c.RequestIntegrationID)
out, err := scanConnection(row)
id, c.Name, c.Kind, nullable(c.BaseURL), nullable(apiKeyRef), c.RequestIntegrationID)
out, err := r.scanConnection(row)
if err != nil {
return Connection{}, fmt.Errorf("create autoscan connection: %w", err)
}
@@ -108,6 +149,11 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
// next poll. Mirrors requests.UpdateIntegration's CASE-WHEN keep-semantics.
// Pass the raw trimmed string (not nullable()) so the empty-string sentinel
// reaches the CASE.
// Encrypt the incoming key; an empty result preserves the keep-existing CASE.
apiKeyRef, err := r.encryptAPIKey(c.ID, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
UPDATE autoscan_connections
SET name = $2, kind = $3, base_url = $4,
@@ -115,8 +161,8 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
request_integration_id = $6, updated_at = now()
WHERE id = $1
RETURNING `+connectionColumns,
c.ID, c.Name, c.Kind, nullable(c.BaseURL), c.APIKeyRef, c.RequestIntegrationID)
out, err := scanConnection(row)
c.ID, c.Name, c.Kind, nullable(c.BaseURL), apiKeyRef, c.RequestIntegrationID)
out, err := r.scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, c.ID)
@@ -151,7 +197,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
defer rows.Close()
var out []Connection
for rows.Next() {
c, err := scanConnection(rows)
c, err := r.scanConnection(rows)
if err != nil {
return nil, err
}
@@ -163,7 +209,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
func (r *Repository) GetConnection(ctx context.Context, id string) (Connection, error) {
row := r.pool.QueryRow(ctx, `SELECT `+connectionColumns+`
FROM autoscan_connections WHERE id = $1`, id)
c, err := scanConnection(row)
c, err := r.scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, id)
@@ -175,14 +221,14 @@ func (r *Repository) GetConnection(ctx context.Context, id string) (Connection,
// --- Sources ---
const sourceColumns = `id, installation_id, capability_id, connection_id, enabled,
const sourceColumns = `id, plugin_id, capability_id, connection_id, enabled,
poll_interval_seconds, path_rewrites, source_config, label, marker, last_run_at, last_error`
func scanSource(row interface{ Scan(...any) error }) (Source, error) {
var s Source
var pathRewrites []byte
var sourceConfig []byte
if err := row.Scan(&s.ID, &s.InstallationID, &s.CapabilityID, &s.ConnectionID,
if err := row.Scan(&s.ID, &s.PluginID, &s.CapabilityID, &s.ConnectionID,
&s.Enabled, &s.PollIntervalSeconds, &pathRewrites, &sourceConfig, &s.Label, &s.Marker, &s.LastRunAt, &s.LastError); err != nil {
return Source{}, err
}
@@ -283,11 +329,11 @@ func (r *Repository) CreateSource(ctx context.Context, s Source) (Source, error)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_sources (
installation_id, capability_id, connection_id, enabled, poll_interval_seconds, path_rewrites, source_config, label
plugin_id, capability_id, connection_id, enabled, poll_interval_seconds, path_rewrites, source_config, label
)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING `+sourceColumns,
s.InstallationID, s.CapabilityID, connectionIDArg(s.ConnectionID), s.Enabled, s.PollIntervalSeconds, rewrites, sourceConfig, s.Label)
s.PluginID, s.CapabilityID, connectionIDArg(s.ConnectionID), s.Enabled, s.PollIntervalSeconds, rewrites, sourceConfig, s.Label)
out, err := scanSource(row)
if err != nil {
if connID, ok := connectionFKViolation(err, s.ConnectionID); ok {
@@ -299,7 +345,7 @@ func (r *Repository) CreateSource(ctx context.Context, s Source) (Source, error)
}
// UpdateSource updates a source's binding/scheduling fields by id. Identity
// (installation_id, capability_id) and bookkeeping fields (marker/last_run_at/
// (plugin_id, capability_id) and bookkeeping fields (marker/last_run_at/
// last_error) are left untouched. An unknown id maps to ErrNotFound; a
// non-existent connection trips the FK constraint and also maps to ErrNotFound.
func (r *Repository) UpdateSource(ctx context.Context, s Source) (Source, error) {
@@ -353,7 +399,7 @@ func connectionFKViolation(err error, connectionID *string) (string, bool) {
func (r *Repository) ListSources(ctx context.Context) ([]Source, error) {
rows, err := r.pool.Query(ctx, `SELECT `+sourceColumns+`
FROM autoscan_sources ORDER BY installation_id, capability_id`)
FROM autoscan_sources ORDER BY plugin_id, capability_id`)
if err != nil {
return nil, fmt.Errorf("list autoscan sources: %w", err)
}
@@ -364,7 +410,7 @@ func (r *Repository) ListSources(ctx context.Context) ([]Source, error) {
func (r *Repository) ListEnabledSources(ctx context.Context) ([]Source, error) {
rows, err := r.pool.Query(ctx, `SELECT `+sourceColumns+`
FROM autoscan_sources WHERE enabled = true
ORDER BY installation_id, capability_id`)
ORDER BY plugin_id, capability_id`)
if err != nil {
return nil, fmt.Errorf("list enabled autoscan sources: %w", err)
}
@@ -462,7 +508,7 @@ func (r *Repository) RecordError(ctx context.Context, sourceID, msg string) erro
return nil
}
const eventColumns = `id, source_id, installation_id, capability_id, started_at, completed_at,
const eventColumns = `id, source_id, plugin_id, capability_id, started_at, completed_at,
duration_ms, status, changes_returned, changes_resolved, targets_claimed, scans_created,
scans_reused, scans_suppressed, error_message, marker_before, marker_after`
@@ -472,7 +518,7 @@ func scanEvent(row interface{ Scan(...any) error }) (Event, error) {
if err := row.Scan(
&e.ID,
&e.SourceID,
&e.InstallationID,
&e.PluginID,
&e.CapabilityID,
&e.StartedAt,
&e.CompletedAt,
@@ -499,25 +545,80 @@ func (r *Repository) CreateEvent(ctx context.Context, in EventCreate) (int64, er
if started.IsZero() {
started = time.Now()
}
row := r.pool.QueryRow(ctx, `
sourceID := strings.TrimSpace(in.SourceID)
if sourceID == "" {
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_events (
source_id, plugin_id, capability_id, started_at, completed_at,
duration_ms, status, error_message, marker_before
)
VALUES ($1, $2, $3, $4, $4, 0, $5, $6, $7)
RETURNING id`,
nil,
in.PluginID,
in.CapabilityID,
started,
string(EventStatusRunning),
"",
nullable(in.MarkerBefore),
)
var id int64
if err := row.Scan(&id); err != nil {
return 0, fmt.Errorf("create autoscan event: %w", err)
}
return id, nil
}
tx, err := r.pool.Begin(ctx)
if err != nil {
return 0, fmt.Errorf("begin autoscan event: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock(900173, hashtext($1))`, sourceID); err != nil {
return 0, fmt.Errorf("lock autoscan event: %w", err)
}
var runningID int64
err = tx.QueryRow(ctx, `
SELECT id
FROM autoscan_events
WHERE source_id = $1
AND status = $2
ORDER BY started_at ASC, id ASC
LIMIT 1`,
sourceID,
string(EventStatusRunning),
).Scan(&runningID)
if err == nil {
return 0, fmt.Errorf("%w: source %s event %d", ErrPollAlreadyRunning, sourceID, runningID)
}
if !errors.Is(err, pgx.ErrNoRows) {
return 0, fmt.Errorf("check running autoscan event: %w", err)
}
row := tx.QueryRow(ctx, `
INSERT INTO autoscan_events (
source_id, installation_id, capability_id, started_at, completed_at,
source_id, plugin_id, capability_id, started_at, completed_at,
duration_ms, status, error_message, marker_before
)
VALUES ($1, $2, $3, $4, $4, 0, $5, $6, $7)
RETURNING id`,
nullable(in.SourceID),
in.InstallationID,
sourceID,
in.PluginID,
in.CapabilityID,
started,
string(EventStatusError),
"poll started but did not finish",
string(EventStatusRunning),
"",
nullable(in.MarkerBefore),
)
var id int64
if err := row.Scan(&id); err != nil {
return 0, fmt.Errorf("create autoscan event: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return 0, fmt.Errorf("commit autoscan event: %w", err)
}
return id, nil
}
@@ -565,6 +666,25 @@ func (r *Repository) FinishEvent(ctx context.Context, in EventFinish) error {
return nil
}
func (r *Repository) MarkInterruptedEvents(ctx context.Context) error {
msg := "poll started but did not finish"
_, err := r.pool.Exec(ctx, `
UPDATE autoscan_events
SET completed_at = now(),
duration_ms = GREATEST(0, EXTRACT(EPOCH FROM (now() - started_at)) * 1000)::bigint,
status = $1,
error_message = $2
WHERE status = $3`,
string(EventStatusError),
msg,
string(EventStatusRunning),
)
if err != nil {
return fmt.Errorf("mark interrupted autoscan events: %w", err)
}
return nil
}
// clampAutoscanLimit bounds a requested page size to a sane window: the
// default page when unset, capped so a single query can never fan out.
func clampAutoscanLimit(limit int) int {
@@ -758,6 +878,30 @@ func (r *Repository) ListEvents(ctx context.Context, filter EventListFilter) ([]
return events, runRows.Err()
}
func (r *Repository) ListRunningEvents(ctx context.Context) ([]Event, error) {
rows, err := r.pool.Query(ctx, `
SELECT `+eventColumns+`
FROM autoscan_events
WHERE status = $1
ORDER BY started_at ASC, id ASC`,
string(EventStatusRunning),
)
if err != nil {
return nil, fmt.Errorf("list running autoscan events: %w", err)
}
defer rows.Close()
events := make([]Event, 0)
for rows.Next() {
event, scanErr := scanEvent(rows)
if scanErr != nil {
return nil, scanErr
}
events = append(events, event)
}
return events, rows.Err()
}
func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilter) ([]ScanWithEvent, error) {
limit := clampAutoscanLimit(filter.Limit)
offset := filter.Offset
@@ -780,15 +924,15 @@ func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilte
sr.trigger,
sr.status,
COALESCE(sr.error_message, ''),
sr.requested_at,
sr.started_at,
sr.completed_at,
sr.autoscan_event_id,
e.source_id,
e.installation_id,
COALESCE(e.capability_id, ''),
COALESCE(e.status, ''),
e.completed_at
sr.requested_at,
sr.started_at,
sr.completed_at,
sr.autoscan_event_id,
e.source_id,
COALESCE(e.plugin_id, ''),
COALESCE(e.capability_id, ''),
COALESCE(e.status, ''),
e.completed_at
FROM scan_runs sr
LEFT JOIN autoscan_events e ON e.id = sr.autoscan_event_id
WHERE `+strings.Join(clauses, " AND ")+`
@@ -818,7 +962,7 @@ func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilte
&scan.CompletedAt,
&scan.AutoscanEventID,
&scan.SourceID,
&scan.InstallationID,
&scan.PluginID,
&scan.CapabilityID,
&eventStatus,
&scan.EventCompletedAt,
@@ -853,7 +997,7 @@ func (r *Repository) GetQueueSummary(ctx context.Context) (QueueSummary, error)
func (r *Repository) LatestEventAt(ctx context.Context) (*time.Time, error) {
var latest *time.Time
err := r.pool.QueryRow(ctx, `SELECT max(completed_at) FROM autoscan_events`).Scan(&latest)
err := r.pool.QueryRow(ctx, `SELECT max(completed_at) FROM autoscan_events WHERE status <> $1`, string(EventStatusRunning)).Scan(&latest)
if err != nil {
return nil, fmt.Errorf("get latest autoscan event time: %w", err)
}
+22 -35
View File
@@ -91,8 +91,7 @@ func (s *Service) SetSuggesterDeps(rootFolders RootFolderClient, folders FolderL
// from scan_source plugins; connres resolves a source's connection to concrete
// credentials; resolver/queue/suppress drive the resolve→suppress→enqueue loop.
// lister enumerates installed scan_source capabilities so the Add-source picker
// can offer them and PollOnce can skip orphaned source rows; it may be nil
// (the picker then returns an empty list and orphan-pruning is disabled).
// can offer them; it may be nil (the picker then returns an empty list).
func NewService(
store Store,
provider ScanSourceProvider,
@@ -113,24 +112,15 @@ func NewService(
}
}
// PollOnce runs one autoscan cycle. Per-source failures are logged and skipped;
// only settings/listing errors propagate. The opaque next marker returned by the
// PollOnce runs one autoscan cycle. Per-source failures are logged, recorded on
// the source/event, and the loop continues; only settings/listing errors
// propagate. The opaque next marker returned by the
// provider is stored verbatim, but only when the cycle's work is genuinely
// consumed: the provider returned no paths, or it returned paths and at least one
// resolved+enqueued. When paths come back but NONE resolve to a library folder
// (e.g. a freshly-enabled source with unconfigured rewrites) the marker is held
// and an error recorded, so those imports aren't skipped forever.
func (s *Service) PollOnce(ctx context.Context) error {
// Fetch the set of currently-installed scan_source capabilities so we can skip
// orphaned source rows (their plugin was uninstalled). Listing failures are
// non-fatal: a nil set means the installed set is unavailable, in which case
// we do NOT prune orphans — every enabled source is assumed present.
present, derr := s.installedScanSources(ctx)
if derr != nil {
slog.WarnContext(ctx, "autoscan: list installed scan sources failed", "err", derr)
present = nil
}
settings, err := s.store.GetSettings(ctx)
if err != nil {
return err
@@ -146,16 +136,6 @@ func (s *Service) PollOnce(ctx context.Context) error {
now := time.Now()
for _, src := range sources {
// Skip orphaned sources: an enabled source whose scan_source plugin has
// been uninstalled/disabled is no longer in the discovered set, so polling
// it would error every cycle. Skip it quietly (no RecordError) so the spam
// stops; the operator can delete it via the source delete endpoint. A nil
// `present` set means discovery is unavailable — don't prune in that case.
if present != nil {
if _, ok := present[installedKey{InstallationID: src.InstallationID, CapabilityID: src.CapabilityID}]; !ok {
continue
}
}
// Honor the per-source poll interval as a "poll at most every N seconds"
// floor: the global task fires at the default cadence, so a source with a
// longer interval is skipped until enough time has elapsed.
@@ -170,7 +150,10 @@ func (s *Service) PollOnce(ctx context.Context) error {
if src.Marker != nil {
marker = *src.Marker
}
eventID := s.createEvent(ctx, src, marker, time.Now())
eventID, started := s.createEvent(ctx, src, marker, time.Now())
if !started {
continue
}
// A connection is OPTIONAL. Server-based providers (Sonarr/Radarr) bind a
// connection and the resolved {base_url, api_key} is handed to the plugin.
// Other providers (e.g. a filesystem/CephFS watcher) need none and get an
@@ -195,7 +178,7 @@ func (s *Service) PollOnce(ctx context.Context) error {
}
conn = resolved
}
changes, next, perr := s.provider.PollChanges(ctx, src.InstallationID, src.CapabilityID, marker, conn, src.SourceConfig)
changes, next, perr := s.provider.PollChanges(ctx, src.PluginID, src.CapabilityID, marker, conn, src.SourceConfig)
if perr != nil {
slog.WarnContext(ctx, "autoscan: poll changes failed", "source_id", src.ID, "err", perr)
if rerr := s.store.RecordError(ctx, src.ID, perr.Error()); rerr != nil {
@@ -295,22 +278,26 @@ func (s *Service) PollOnce(ctx context.Context) error {
return nil
}
func (s *Service) createEvent(ctx context.Context, src Source, marker string, startedAt time.Time) int64 {
func (s *Service) createEvent(ctx context.Context, src Source, marker string, startedAt time.Time) (int64, bool) {
if s == nil || s.store == nil {
return 0
return 0, true
}
id, err := s.store.CreateEvent(ctx, EventCreate{
SourceID: src.ID,
InstallationID: src.InstallationID,
CapabilityID: src.CapabilityID,
StartedAt: startedAt,
MarkerBefore: marker,
SourceID: src.ID,
PluginID: src.PluginID,
CapabilityID: src.CapabilityID,
StartedAt: startedAt,
MarkerBefore: marker,
})
if err != nil {
if errors.Is(err, ErrPollAlreadyRunning) {
slog.DebugContext(ctx, "autoscan: source poll already running", "source_id", src.ID)
return 0, false
}
slog.WarnContext(ctx, "autoscan: create event failed", "source_id", src.ID, "err", err)
return 0
return 0, true
}
return id
return id, true
}
func (s *Service) finishEvent(ctx context.Context, eventID int64, finish EventFinish) {
+85 -38
View File
@@ -2,6 +2,7 @@ package autoscan
import (
"context"
"errors"
"path/filepath"
"strings"
"testing"
@@ -12,13 +13,14 @@ import (
)
type fakeStore struct {
settings Settings
sources []Source
connection Connection
advanced map[string]string // source ID -> marker
recorded map[string]string // source ID -> error message
createdEvents []EventCreate
events []EventFinish
settings Settings
sources []Source
connection Connection
advanced map[string]string // source ID -> marker
recorded map[string]string // source ID -> error message
createdEvents []EventCreate
events []EventFinish
createEventErr error
}
func (f *fakeStore) GetSettings(context.Context) (Settings, error) { return f.settings, nil }
@@ -51,6 +53,9 @@ func (f *fakeStore) RecordError(_ context.Context, sourceID, msg string) error {
return nil
}
func (f *fakeStore) CreateEvent(_ context.Context, event EventCreate) (int64, error) {
if f.createEventErr != nil {
return 0, f.createEventErr
}
f.createdEvents = append(f.createdEvents, event)
return int64(len(f.createdEvents)), nil
}
@@ -65,14 +70,20 @@ type fakeProvider struct {
changes map[string][]Change // key: capabilityID, structured changes
nextMarker string
err error
errByCap map[string]error
lastConfig map[string]string
calls int
}
func (f *fakeProvider) PollChanges(_ context.Context, _ int, capabilityID, _ string, _ ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
func (f *fakeProvider) PollChanges(_ context.Context, _ string, capabilityID, _ string, _ ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
f.calls++
f.lastConfig = sourceConfig
if f.err != nil {
return nil, "", f.err
}
if err := f.errByCap[capabilityID]; err != nil {
return nil, "", err
}
if changes, ok := f.changes[capabilityID]; ok {
return changes, f.nextMarker, nil
}
@@ -87,7 +98,7 @@ func TestPollOncePassesSourceConfigToProvider(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
SourceConfig: map[string]string{"exclusions": ".downloads\n.recyclebin"},
}},
}
@@ -101,6 +112,38 @@ func TestPollOncePassesSourceConfigToProvider(t *testing.T) {
}
}
func TestPollOnceSkipsSourceWhenPollAlreadyRunning(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
createEventErr: ErrPollAlreadyRunning,
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"cephfs": {"/mnt/media/Movie/movie.mkv"}}, nextMarker: "m1"}
q := &recordingQueuer{}
svc := newService(store, prov, q, allowSuppressor{})
if err := svc.PollOnce(context.Background()); err != nil {
t.Fatalf("PollOnce: %v", err)
}
if prov.calls != 0 {
t.Fatalf("provider calls = %d, want 0", prov.calls)
}
if len(q.enqueued) != 0 {
t.Fatalf("enqueued scans = %+v, want none", q.enqueued)
}
if len(store.events) != 0 {
t.Fatalf("finished events = %+v, want none", store.events)
}
if len(store.recorded) != 0 {
t.Fatalf("recorded source errors = %+v, want none", store.recorded)
}
if len(store.advanced) != 0 {
t.Fatalf("advanced markers = %+v, want none", store.advanced)
}
}
// passthroughConnRes resolves to empty credentials; the engine doesn't inspect
// them in tests (the fake provider ignores conn).
type passthroughConnRes struct{}
@@ -210,7 +253,7 @@ func TestPollOnceEnqueuesDedupedFolders(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -241,7 +284,7 @@ func TestPollOnceRecordsSuccessfulEvent(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true, Marker: &marker,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true, Marker: &marker,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -283,7 +326,7 @@ func TestPollOnceRecordsReusedScanCounts(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -318,7 +361,7 @@ func TestPollOnceAppliesSourceRewritesBeforeEnqueue(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PathRewrites: []PathRewrite{{From: "/data/tv", To: "/mnt/media/tv"}},
}},
}
@@ -347,7 +390,7 @@ func TestPollOnceStructuredFileChangeEnqueuesExactFile(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
PathRewrites: []PathRewrite{{From: "/ceph/tv", To: "/mnt/media/tv"}},
}},
}
@@ -380,7 +423,7 @@ func TestPollOnceStructuredSubtreeChangeEnqueuesExactSubtree(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
PathRewrites: []PathRewrite{{From: "/ceph/movies", To: "/mnt/media/movies"}},
}},
}
@@ -416,7 +459,7 @@ func TestPollOnceScansDistinctPathsUnderSameFolder(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -446,7 +489,7 @@ func TestPollOnceStoresOpaqueMarkerVerbatim(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
const opaque = "eyJjdXJzb3IiOiJhYmMxMjMifQ==|2026-06-02T14:10:00Z"
@@ -470,7 +513,7 @@ func TestPollOnceHoldsMarkerWhenPathsReturnedButNoneResolve(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -523,7 +566,7 @@ func TestPollOnceAdvancesMarkerWhenResolvedButAllSuppressed(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
// rewrite /data/tv -> /mnt/media/tv so fakeResolver resolves the paths.
PathRewrites: []PathRewrite{{From: "/data/tv", To: "/mnt/media/tv"}},
}},
@@ -556,7 +599,7 @@ func TestPollOnceAdvancesMarkerWhenZeroPathsReturned(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {}}, nextMarker: "m1"}
@@ -588,7 +631,7 @@ func TestPollOnceDisabledNoop(t *testing.T) {
func TestPollOnceProviderErrorKeepsMarker(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
sources: []Source{{ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
}
prov := &fakeProvider{err: context.DeadlineExceeded}
q := &recordingQueuer{}
@@ -607,7 +650,7 @@ func TestPollOnceProviderErrorKeepsMarker(t *testing.T) {
func TestPollOnceReleasesClaimOnEnqueueFailure(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
sources: []Source{{ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {"/mnt/media/Show/S01/E01.mkv"}}, nextMarker: "m1"}
sup := &recordingSuppressor{}
@@ -640,7 +683,7 @@ func TestPollOncePollsConnectionlessSource(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: nil, Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil, Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {"/mnt/media/Show/S01/E01.mkv"}}, nextMarker: "m1"}
@@ -666,7 +709,7 @@ func TestPollOnceSkipsSourcePolledTooRecently(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PollIntervalSeconds: &interval, LastRunAt: &recent,
}},
}
@@ -690,7 +733,7 @@ func TestPollOnceRunsSourcePastItsInterval(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PollIntervalSeconds: &interval, LastRunAt: &old,
}},
}
@@ -708,25 +751,23 @@ func TestPollOnceRunsSourcePastItsInterval(t *testing.T) {
}
}
func TestPollOnceSkipsOrphanedSourceQuietly(t *testing.T) {
// Two enabled sources: only "arr-live" is still discovered; "arr-gone"'s
// plugin has been uninstalled. The orphan must be skipped quietly — no poll,
// no RecordError spam — while the live source still polls normally.
func TestPollOnceRecordsScanSourceResolutionFailure(t *testing.T) {
// Two enabled sources: one polls normally, while the other simulates the
// production resolver failing because the plugin cannot be resolved.
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{
{ID: "live", InstallationID: 1, CapabilityID: "arr-live", ConnectionID: strptr("c1"), Enabled: true},
{ID: "gone", InstallationID: 2, CapabilityID: "arr-gone", ConnectionID: strptr("c1"), Enabled: true},
{ID: "live", PluginID: "silo.autoscan.live", CapabilityID: "arr-live", ConnectionID: strptr("c1"), Enabled: true},
{ID: "gone", PluginID: "silo.autoscan.gone", CapabilityID: "arr-gone", ConnectionID: strptr("c1"), Enabled: true},
},
}
prov := &fakeProvider{paths: map[string][]string{
"arr-live": {"/mnt/media/Show/S01/E01.mkv"},
"arr-gone": {"/mnt/media/Other/S01/E01.mkv"},
}, errByCap: map[string]error{
"arr-gone": errors.New("scan source plugin \"silo.autoscan.gone\" is not installed"),
}, nextMarker: "m1"}
q := &recordingQueuer{}
// Lister reports only the live capability; the orphan is absent.
lister := fakeLister{sources: []DiscoveredSource{{InstallationID: 1, CapabilityID: "arr-live"}}}
svc := NewService(store, prov, passthroughConnRes{}, fakeResolver{}, q, allowSuppressor{}, lister)
svc := newService(store, prov, q, allowSuppressor{})
if err := svc.PollOnce(context.Background()); err != nil {
t.Fatalf("PollOnce: %v", err)
@@ -738,9 +779,15 @@ func TestPollOnceSkipsOrphanedSourceQuietly(t *testing.T) {
t.Fatalf("expected live source marker advanced")
}
if _, ok := store.advanced["gone"]; ok {
t.Fatalf("orphaned source must NOT poll/advance")
t.Fatalf("failed source must NOT advance")
}
if _, ok := store.recorded["gone"]; ok {
t.Fatalf("orphaned source must be skipped quietly, but an error was recorded: %q", store.recorded["gone"])
if got := store.recorded["gone"]; !strings.Contains(got, "not installed") {
t.Fatalf("failed source error = %q", got)
}
if len(store.events) != 2 {
t.Fatalf("expected one event per source, got %+v", store.events)
}
if store.events[1].Status != EventStatusError || !strings.Contains(store.events[1].ErrorMessage, "not installed") {
t.Fatalf("failed source event = %+v", store.events[1])
}
}
+10 -9
View File
@@ -45,11 +45,11 @@ type Change struct {
Scope ChangeScope
}
// Source ties a scan_source plugin capability instance to a connection plus the
// Source ties a scan_source plugin capability to a connection plus the
// host-owned scheduling/bookkeeping state.
type Source struct {
ID string
InstallationID int
PluginID string
CapabilityID string
ConnectionID *string // nil until an operator binds a connection
Enabled bool
@@ -65,6 +65,7 @@ type Source struct {
type EventStatus string
const (
EventStatusRunning EventStatus = "running"
EventStatusSuccess EventStatus = "success"
EventStatusError EventStatus = "error"
EventStatusUnresolved EventStatus = "unresolved"
@@ -73,7 +74,7 @@ const (
type Event struct {
ID int64
SourceID *string
InstallationID int
PluginID string
CapabilityID string
StartedAt time.Time
CompletedAt time.Time
@@ -91,11 +92,11 @@ type Event struct {
}
type EventCreate struct {
SourceID string
InstallationID int
CapabilityID string
StartedAt time.Time
MarkerBefore string
SourceID string
PluginID string
CapabilityID string
StartedAt time.Time
MarkerBefore string
}
type EventFinish struct {
@@ -154,7 +155,7 @@ type ScanWithEvent struct {
ScanRunSummary
AutoscanEventID *int64
SourceID *string
InstallationID *int
PluginID string
CapabilityID string
EventStatus EventStatus
EventCompletedAt *time.Time
+3 -2
View File
@@ -1089,8 +1089,9 @@ func validateCatalogOverlayQuery(searchQuery string, def QueryDefinition, ruleFi
if def.MediaScope != "" &&
def.MediaScope != "movie" &&
def.MediaScope != "series" &&
def.MediaScope != "episode" {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', or 'episode'", ErrInvalidCatalogRequest)
def.MediaScope != "episode" &&
def.MediaScope != "audiobook" {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', 'episode', or 'audiobook'", ErrInvalidCatalogRequest)
}
if def.Match != "" && def.Match != "all" && def.Match != "any" {
return fmt.Errorf("%w: match must be 'all' or 'any'", ErrInvalidCatalogRequest)
+423
View File
@@ -154,6 +154,51 @@ type ItemDetail struct {
EffectiveVersionHDR *bool `json:"effective_version_hdr,omitempty"`
EffectiveVersionCodecVideo *string `json:"effective_version_codec_video,omitempty"`
EffectiveVersionEditionKey *string `json:"effective_version_edition_key,omitempty"`
// Audiobook-specific detail. Present only when Type == "audiobook".
Audiobook *AudiobookDetailExtension `json:"audiobook,omitempty"`
}
type AudiobookDetailExtension struct {
Authors []AudiobookPerson `json:"authors"`
Narrators []AudiobookPerson `json:"narrators"`
Publisher string `json:"publisher,omitempty"`
TotalDurationSeconds int `json:"total_duration_seconds"`
Series *AudiobookSeriesGroup `json:"series,omitempty"`
OtherNarrations []AudiobookNarration `json:"other_narrations"`
Related AudiobookRelatedContent `json:"related"`
}
type AudiobookPerson struct {
PersonID string `json:"person_id,omitempty"`
Name string `json:"name"`
PhotoURL string `json:"photo_url,omitempty"`
PhotoThumbhash string `json:"photo_thumbhash,omitempty"`
}
type AudiobookRelatedContent struct {
AlsoByAuthor []AudiobookRelatedItem `json:"also_by_author"`
Similar []AudiobookRelatedItem `json:"similar"`
}
type AudiobookRelatedItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
SeriesIndex *int `json:"series_index,omitempty"`
}
type AudiobookSeriesGroup struct {
Name string `json:"name,omitempty"`
Entries []AudiobookRelatedItem `json:"entries"`
}
type AudiobookNarration struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
Narrators []string `json:"narrators"`
}
// ItemUserState is per-profile viewer state included in item detail responses.
@@ -224,6 +269,7 @@ type FileVersion struct {
PresentationKind string `json:"presentation_kind,omitempty"`
PresentationGroupKey string `json:"presentation_group_key,omitempty"`
PresentationPartIndex int `json:"presentation_part_index,omitempty"`
PresentationPartTotal int `json:"presentation_part_total,omitempty"`
MultiEpisodeStart int `json:"multi_episode_start,omitempty"`
MultiEpisodeEnd int `json:"multi_episode_end,omitempty"`
EffectiveAudioTrackIndex *int `json:"effective_audio_track_index,omitempty"`
@@ -833,6 +879,9 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
}
files = FilterMediaFilesByAccess(files, filter)
if item.Type == "audiobook" {
sortAudiobookMediaFiles(files)
}
files = s.preparePlaybackFiles(ctx, files)
detail.Versions, detail.PlaybackVariants, detail.Subtitles, detail.Intro, detail.Credits, detail.Recap, detail.Preview = s.buildPlaybackInfo(
ctx,
@@ -843,6 +892,10 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
detail.OverlaySummary = overlays.BuildSummary(files)
}
if item.Type == "audiobook" {
detail.Audiobook = s.buildAudiobookExtension(ctx, item, detail.Versions, crewCredits, filter)
}
// Series folder paths from confirmed claims when available, otherwise from
// the file links that currently belong to the item. This keeps provisional
// root-scoped series visible for manual match/admin flows without implying
@@ -952,6 +1005,351 @@ func splitCastCrew(credits []PersonCredit) ([]CastCredit, []CrewCredit) {
return cast, crew
}
func (s *DetailService) buildAudiobookExtension(
ctx context.Context,
item *models.MediaItem,
versions []FileVersion,
crew []CrewCredit,
filter AccessFilter,
) *AudiobookDetailExtension {
if item == nil {
return nil
}
totalDuration := 0
for _, version := range versions {
if version.Duration > 0 {
totalDuration += version.Duration
}
}
return &AudiobookDetailExtension{
Authors: audiobookPeopleFromCrew(crew, models.PersonKindAuthor.String()),
Narrators: audiobookPeopleFromCrew(crew, models.PersonKindNarrator.String()),
Publisher: firstNonEmptyString(item.Studios),
TotalDurationSeconds: totalDuration,
Series: s.fetchAudiobookSeries(ctx, item.ContentID, filter),
OtherNarrations: s.fetchAudiobookOtherNarrations(ctx, item.ContentID, filter),
Related: AudiobookRelatedContent{
AlsoByAuthor: s.fetchAudiobookAlsoByAuthor(ctx, item.ContentID, filter),
Similar: s.fetchAudiobookSimilarByGenres(ctx, item.ContentID, filter),
},
}
}
func audiobookPeopleFromCrew(crew []CrewCredit, job string) []AudiobookPerson {
out := make([]AudiobookPerson, 0)
for _, credit := range crew {
if credit.Job != job || strings.TrimSpace(credit.Name) == "" {
continue
}
out = append(out, AudiobookPerson{
PersonID: credit.PersonID,
Name: credit.Name,
PhotoURL: credit.PhotoURL,
PhotoThumbhash: credit.PhotoThumbhash,
})
}
return out
}
func firstNonEmptyString(values []string) string {
for _, value := range values {
if strings.TrimSpace(value) != "" {
return value
}
}
return ""
}
func appendAudiobookItemAccessConditions(
alias string,
filter AccessFilter,
conditions *[]string,
args *[]any,
argIdx *int,
) bool {
if filter.AllowedLibraryIDs != nil {
if len(filter.AllowedLibraryIDs) == 0 {
return false
}
*conditions = append(*conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil_allowed
WHERE mil_allowed.content_id = %s.content_id
AND mil_allowed.media_folder_id = ANY($%d)
)`, alias, *argIdx))
*args = append(*args, filter.AllowedLibraryIDs)
*argIdx = *argIdx + 1
}
if len(filter.DisabledLibraryIDs) > 0 {
if filter.AllowedLibraryIDs == nil {
*conditions = append(*conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil_visible
WHERE mil_visible.content_id = %s.content_id
)`, alias))
}
*conditions = append(*conditions, fmt.Sprintf(`
NOT EXISTS (
SELECT 1 FROM media_item_libraries mil_disabled
WHERE mil_disabled.content_id = %s.content_id
AND mil_disabled.media_folder_id = ANY($%d)
)`, alias, *argIdx))
*args = append(*args, filter.DisabledLibraryIDs)
*argIdx = *argIdx + 1
}
ApplySectionAccessFilter(alias, AccessFilter{MaxContentRating: filter.MaxContentRating}, conditions, args, argIdx)
return true
}
func (s *DetailService) fetchAudiobookAlsoByAuthor(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
conditions := []string{
"ip1.content_id = $1",
"ip1.kind = $2",
"m2.type = 'audiobook'",
}
if !appendAudiobookItemAccessConditions("m2", filter, &conditions, &args, &argIdx) {
return []AudiobookRelatedItem{}
}
query := fmt.Sprintf(`
SELECT m2.content_id, m2.title, COALESCE(m2.year, 0), COALESCE(m2.poster_path, '')
FROM item_people ip1
JOIN item_people ip2
ON ip2.person_id = ip1.person_id
AND ip2.kind = ip1.kind
AND ip2.content_id <> ip1.content_id
JOIN media_items m2 ON m2.content_id = ip2.content_id
WHERE %s
ORDER BY m2.year DESC NULLS LAST, LOWER(m2.sort_title)
LIMIT 12
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookRelatedItem{}
}
defer rows.Close()
out := make([]AudiobookRelatedItem, 0, 12)
seen := make(map[string]struct{})
for rows.Next() {
var item AudiobookRelatedItem
var posterPath string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &posterPath); err != nil {
return []AudiobookRelatedItem{}
}
if _, ok := seen[item.ContentID]; ok {
continue
}
seen[item.ContentID] = struct{}{}
item.PosterURL = s.PresignURL(ctx, posterPath, "featured")
out = append(out, item)
}
return out
}
func (s *DetailService) fetchAudiobookSimilarByGenres(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
conditions := []string{
"m.type = 'audiobook'",
"m.content_id <> $1",
"m.genres && (SELECT array_agg(g) FROM this_genres)",
`NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)`,
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return []AudiobookRelatedItem{}
}
query := fmt.Sprintf(`
WITH this_genres AS (
SELECT unnest(genres) AS g FROM media_items WHERE content_id = $1
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE %s
ORDER BY
(
SELECT COUNT(*)
FROM unnest(m.genres) AS genre(value)
WHERE genre.value IN (SELECT g FROM this_genres)
) DESC,
COALESCE(m.year, 0) DESC,
LOWER(m.sort_title)
LIMIT 12
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookRelatedItem{}
}
defer rows.Close()
out := make([]AudiobookRelatedItem, 0, 12)
for rows.Next() {
var item AudiobookRelatedItem
var posterPath string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &posterPath); err != nil {
return []AudiobookRelatedItem{}
}
item.PosterURL = s.PresignURL(ctx, posterPath, "featured")
out = append(out, item)
}
return out
}
func (s *DetailService) fetchAudiobookSeries(ctx context.Context, contentID string, filter AccessFilter) *AudiobookSeriesGroup {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return nil
}
args := []any{contentID}
argIdx := 2
conditions := []string{
"root.content_id = $1",
"m.type = 'audiobook'",
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return nil
}
query := fmt.Sprintf(`
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(m.poster_path, ''),
s.series_name,
s.series_index
FROM audiobook_series root
JOIN audiobook_series s ON LOWER(s.series_name) = LOWER(root.series_name)
JOIN media_items m ON m.content_id = s.content_id
WHERE %s
ORDER BY s.series_index NULLS LAST, LOWER(m.sort_title)
LIMIT 30
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return nil
}
defer rows.Close()
var seriesName string
entries := make([]AudiobookRelatedItem, 0, 16)
for rows.Next() {
var (
item AudiobookRelatedItem
poster string
name string
index *float64
)
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &poster, &name, &index); err != nil {
return nil
}
if seriesName == "" {
seriesName = name
}
if index != nil {
n := int(*index)
if float64(n) == *index {
item.SeriesIndex = &n
}
}
item.PosterURL = s.PresignURL(ctx, poster, "featured")
entries = append(entries, item)
}
if len(entries) < 2 {
return nil
}
return &AudiobookSeriesGroup{Name: seriesName, Entries: entries}
}
func (s *DetailService) fetchAudiobookOtherNarrations(ctx context.Context, contentID string, filter AccessFilter) []AudiobookNarration {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookNarration{}
}
const stripRE = `\s*\(?\s*[-:,]?\s*(UK Version:?|US Version:?)?\s*[Rr]ead [Bb]y [^()]+\)?\s*$`
args := []any{stripRE, contentID, models.PersonKindAuthor, models.PersonKindNarrator}
argIdx := 5
conditions := []string{
"m.type = 'audiobook'",
"m.content_id <> $2",
"trim(regexp_replace(m.title, $1, '', 'g')) = (SELECT norm_title FROM this_book)",
`EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $3
AND ip.person_id IN (SELECT person_id FROM this_author)
)`,
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return []AudiobookNarration{}
}
query := fmt.Sprintf(`
WITH this_book AS (
SELECT trim(regexp_replace(title, $1, '', 'g')) AS norm_title
FROM media_items WHERE content_id = $2
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $2 AND kind = $3
)
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(string_agg(DISTINCT NULLIF(BTRIM(p.name), ''), '|||'), '') AS narrators
FROM media_items m
LEFT JOIN item_people ipn ON ipn.content_id = m.content_id AND ipn.kind = $4
LEFT JOIN people p ON p.id = ipn.person_id
WHERE %s
GROUP BY m.content_id, m.title, m.year
ORDER BY m.year DESC NULLS LAST, m.title
LIMIT 8
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookNarration{}
}
defer rows.Close()
out := make([]AudiobookNarration, 0, 4)
for rows.Next() {
var item AudiobookNarration
var narrators string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &narrators); err != nil {
return []AudiobookNarration{}
}
item.Narrators = splitDelimitedNames(narrators)
out = append(out, item)
}
return out
}
func splitDelimitedNames(raw string) []string {
if strings.TrimSpace(raw) == "" {
return []string{}
}
parts := strings.Split(raw, "|||")
out := make([]string, 0, len(parts))
for _, part := range parts {
if value := strings.TrimSpace(part); value != "" {
out = append(out, value)
}
}
return out
}
func seriesFolderPathsFromFiles(files []*models.MediaFile) []string {
if len(files) == 0 {
return nil
@@ -1567,6 +1965,30 @@ func markerFromRange(start, end *float64) *Marker {
return &Marker{Start: *start, End: *end}
}
func sortAudiobookMediaFiles(files []*models.MediaFile) {
sort.SliceStable(files, func(i, j int) bool {
a, b := files[i], files[j]
if a == nil || b == nil {
return a != nil
}
if a.PresentationPartIndex > 0 || b.PresentationPartIndex > 0 {
if a.PresentationPartIndex != b.PresentationPartIndex {
if a.PresentationPartIndex == 0 {
return false
}
if b.PresentationPartIndex == 0 {
return true
}
return a.PresentationPartIndex < b.PresentationPartIndex
}
}
if a.FilePath != b.FilePath {
return strings.Compare(a.FilePath, b.FilePath) < 0
}
return a.ID < b.ID
})
}
func (s *DetailService) buildPlaybackInfo(
ctx context.Context,
files []*models.MediaFile,
@@ -1622,6 +2044,7 @@ func (s *DetailService) buildPlaybackInfo(
PresentationKind: f.PresentationKind,
PresentationGroupKey: f.PresentationGroupKey,
PresentationPartIndex: f.PresentationPartIndex,
PresentationPartTotal: f.PresentationPartTotal,
MultiEpisodeStart: f.MultiEpisodeStart,
MultiEpisodeEnd: f.MultiEpisodeEnd,
EffectiveAudioTrackIndex: intPtr(effectiveAudioSelection.Index),
+42
View File
@@ -0,0 +1,42 @@
package catalog
import (
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestSortAudiobookMediaFilesPreservesPresentationPartOrder(t *testing.T) {
files := []*models.MediaFile{
{ID: 30, FilePath: "/books/book/03.mp3", PresentationPartIndex: 3},
{ID: 10, FilePath: "/books/book/01.mp3", PresentationPartIndex: 1},
{ID: 20, FilePath: "/books/book/02.mp3", PresentationPartIndex: 2},
}
sortAudiobookMediaFiles(files)
got := []int{files[0].ID, files[1].ID, files[2].ID}
want := []int{10, 20, 30}
for i := range want {
if got[i] != want[i] {
t.Fatalf("sorted IDs = %v, want %v", got, want)
}
}
}
func TestSortAudiobookMediaFilesFallsBackToPath(t *testing.T) {
files := []*models.MediaFile{
{ID: 20, FilePath: "/books/book/02.mp3"},
{ID: 10, FilePath: "/books/book/01.mp3"},
}
sortAudiobookMediaFiles(files)
got := []int{files[0].ID, files[1].ID}
want := []int{10, 20}
for i := range want {
if got[i] != want[i] {
t.Fatalf("sorted IDs = %v, want %v", got, want)
}
}
}
+138 -136
View File
@@ -28,65 +28,35 @@ type RatingFilter struct {
Min float64
// Limit caps the number of rows returned. Zero or negative means no limit.
Limit int
// LibraryID, when non-nil, restricts results to items in that library via
// the media_item_libraries junction table.
// LibraryID, when non-nil, restricts results to items in that library.
LibraryID *int
// Filter carries viewer-level access constraints (content rating ceiling,
// allowed/disabled library sets).
Filter AccessFilter
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
// buildRatingThresholdQuery builds the SQL statement and bind args for ListByRatingThreshold.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold — NULL ratings are excluded implicitly by >=.
// IMDb rating threshold - NULL ratings are excluded implicitly by >=.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.Min)
argIdx++
// Optional single-library pin.
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id = $%d", argIdx))
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, f.LibraryID, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
@@ -95,6 +65,18 @@ func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f Ratin
args = append(args, f.Limit)
}
return query, args
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
query, args := buildRatingThresholdQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing items by rating threshold: %w", err)
@@ -122,6 +104,50 @@ type UnplayedFilter struct {
Filter AccessFilter
}
// buildUnplayedHighRatedQuery builds the SQL statement and bind args for ListUnplayedHighRated.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// Items the user has any watch history entry for are excluded.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
}
return query, args
}
// ListUnplayedHighRated returns high-rated items that the given user/profile has
// never started watching. "Never started" means no row exists in
// user_watch_history for (user_id, profile_id, media_item_id), regardless of
@@ -133,67 +159,9 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
return nil, fmt.Errorf("ListUnplayedHighRated: UserID and ProfileID are required")
}
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// LEFT JOIN exclusion: items the user has any watch history entry for are excluded.
// We use a NOT EXISTS subquery rather than an outer join so it composes cleanly
// with library access conditions on the main query.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
query, args := buildUnplayedHighRatedQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
@@ -225,13 +193,10 @@ type ForgottenFavoritesFilter struct {
Filter AccessFilter
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
// buildForgottenFavoritesQuery builds the SQL statement and bind args for ListForgottenFavorites.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
@@ -255,39 +220,15 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
@@ -296,6 +237,21 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
args = append(args, f.Limit)
}
return query, args
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
query, args := buildForgottenFavoritesQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing forgotten favorites: %w", err)
@@ -308,3 +264,49 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
}
return items, nil
}
func appendDiscoveryLibraryScope(
conditions *[]string,
args *[]any,
argIdx *int,
libraryID *int,
filter AccessFilter,
) bool {
if libraryID != nil {
whereSQL, scopeArgs, ok := buildLibraryScopeJoin([]int{*libraryID}, nil, *argIdx, "", "mi.content_id")
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
if filter.AllowedLibraryIDs != nil && len(filter.AllowedLibraryIDs) == 0 {
return false
}
// buildLibraryScopeJoin uses semi-joins: disabled libraries are an item-level
// exclusion, matching the rest of catalog access filtering and avoiding row fanout.
if filter.AllowedLibraryIDs == nil && len(filter.DisabledLibraryIDs) > 0 {
// In deny-only mode, require at least one library membership. Without this,
// stale or provisional media_items rows with no membership pass NOT EXISTS.
*conditions = append(*conditions,
"EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)",
)
}
whereSQL, scopeArgs, ok := buildLibraryScopeJoin(
filter.AllowedLibraryIDs,
filter.DisabledLibraryIDs,
*argIdx,
"",
"mi.content_id",
)
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
+146 -224
View File
@@ -4,139 +4,12 @@ package catalog
// discovery_repo.go using the same pure-unit approach used throughout this
// package: build a query string and assert that key fragments are present,
// without needing a live database.
//
// The helpers themselves execute against a real pool; integration coverage is
// provided by whichever recipe resolver exercises them in higher-level tests.
import (
"strings"
"testing"
)
// buildRatingThresholdQuery is the testable extraction of the SQL generation
// logic from ListByRatingThreshold so we can inspect it without a pool.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.Min)
argIdx++
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, "mil.media_folder_id = $2")
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// buildUnplayedHighRatedQuery mirrors the SQL-building portion of
// ListUnplayedHighRated for unit testing.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.MinRating)
argIdx++
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t)")
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// itoa is a local helper to avoid importing strconv.
func itoa(n int) string {
if n == 0 {
return "0"
}
buf := [20]byte{}
pos := len(buf)
for n > 0 {
pos--
buf[pos] = byte('0' + n%10)
n /= 10
}
return string(buf[pos:])
}
// ---------------------------------------------------------------------------
// ListByRatingThreshold SQL generation tests
// ---------------------------------------------------------------------------
@@ -147,6 +20,7 @@ func TestRatingThreshold_BasicQuery(t *testing.T) {
Limit: 20,
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -181,25 +55,24 @@ func TestRatingThreshold_NoLimitOmitsClause(t *testing.T) {
}
}
func TestRatingThreshold_LibraryIDJoin(t *testing.T) {
func TestRatingThreshold_LibraryIDUsesSemiJoin(t *testing.T) {
libID := 42
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
LibraryID: &libID,
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id = $2") {
t.Fatalf("expected media_folder_id predicate at $2, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected library id predicate at $2, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args (min + library id), got %d", len(args))
}
if args[1] != 42 {
t.Fatalf("expected library id 42 at args[1], got %v", args[1])
t.Fatalf("expected 2 args (min + library id slice), got %d", len(args))
}
assertIntSliceArg(t, args, 1, []int{42})
}
func TestRatingThreshold_AllowedLibraries(t *testing.T) {
@@ -210,18 +83,20 @@ func TestRatingThreshold_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($2, $3)") {
t.Fatalf("expected allowed library IN clause, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected allowed library ANY clause, got:\n%s", query)
}
if len(args) != 3 {
t.Fatalf("expected 3 args, got %d: %v", len(args), args)
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{1, 2})
}
func TestRatingThreshold_DisabledLibraries(t *testing.T) {
func TestRatingThreshold_DisabledLibrariesUseItemLevelExclusion(t *testing.T) {
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
@@ -229,19 +104,24 @@ func TestRatingThreshold_DisabledLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "mil.media_folder_id NOT IN ($2)") {
t.Fatalf("expected disabled library NOT IN clause, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_out.media_folder_id = ANY($2)") {
t.Fatalf("expected disabled library ANY clause, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{99})
}
func TestRatingThreshold_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
query, _ := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
AllowedLibraryIDs: []int{}, // empty slice → early return
AllowedLibraryIDs: []int{},
},
})
if query != "" {
@@ -261,6 +141,7 @@ func TestUnplayedHighRated_BasicQuery(t *testing.T) {
ProfileID: "profile-abc",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -301,14 +182,14 @@ func TestUnplayedHighRated_NoLimitOmitsClause(t *testing.T) {
}
}
func TestUnplayedHighRated_NoLibraryJoin(t *testing.T) {
func TestUnplayedHighRated_NoLibraryScope(t *testing.T) {
query, _ := buildUnplayedHighRatedQuery(UnplayedFilter{
MinRating: 7.0,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
}
}
@@ -322,15 +203,17 @@ func TestUnplayedHighRated_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
}
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestUnplayedHighRated_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -359,8 +242,6 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
t.Fatalf("expected content_rating = ANY filter, got:\n%s", query)
}
// args: minRating, userID, profileID, then content rating slice (one arg).
// Task 5.1 converted the rating ladder from IN ($N, $N+1, ...) to
// = ANY($N) with a single bound slice arg.
if len(args) != 4 {
t.Fatalf("expected exactly 4 args (minRating, userID, profileID, rating slice); got %d: %v", len(args), args)
}
@@ -370,62 +251,6 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
// ListForgottenFavorites SQL generation tests
// ---------------------------------------------------------------------------
// buildForgottenFavoritesQuery is the testable extraction of the SQL
// generation logic from ListForgottenFavorites.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= 7.0")
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t\t AND uwh.watched_at >= NOW() - ($"+itoa(argIdx+2)+" || ' days')::interval\n\t)")
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
func TestForgottenFavorites_BasicQuery(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
@@ -434,6 +259,7 @@ func TestForgottenFavorites_BasicQuery(t *testing.T) {
ProfileID: "profile-xyz",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= 7.0") {
t.Fatalf("expected 7.0 rating floor, got:\n%s", query)
}
@@ -477,14 +303,14 @@ func TestForgottenFavorites_NoLimitOmitsClause(t *testing.T) {
}
}
func TestForgottenFavorites_NoLibraryJoin(t *testing.T) {
func TestForgottenFavorites_NoLibraryScope(t *testing.T) {
query, _ := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
}
}
@@ -498,15 +324,17 @@ func TestForgottenFavorites_AllowedLibraries(t *testing.T) {
},
})
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
}
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -525,14 +353,13 @@ func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T)
func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 0, // should default to 365
LookbackDays: 0,
UserID: 1,
ProfileID: "p",
})
if !strings.Contains(query, "NOT EXISTS") {
t.Fatalf("expected NOT EXISTS subquery, got:\n%s", query)
}
// args[2] should be the defaulted lookback value (365).
if len(args) < 3 {
t.Fatalf("expected at least 3 args, got %d", len(args))
}
@@ -540,3 +367,98 @@ func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
t.Fatalf("expected lookback default 365, got %v", args[2])
}
}
func TestDiscoveryQueries_QualifyContentIDAndAvoidLibraryJoin(t *testing.T) {
restricted := AccessFilter{AllowedLibraryIDs: []int{1, 2}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: restricted}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: restricted}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: restricted}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertQualifiedItemSelect(t, tc.query)
assertNoLibraryJoin(t, tc.query)
if !strings.Contains(tc.query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", tc.query)
}
if strings.Contains(tc.query, "SELECT content_id,") {
t.Fatalf("SELECT list uses an unqualified content_id, got:\n%s", tc.query)
}
})
}
}
func TestDiscoveryQueries_DisabledLibrariesUseNotExists(t *testing.T) {
filter := AccessFilter{DisabledLibraryIDs: []int{9}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: filter}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: filter}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: filter}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertNoLibraryJoin(t, tc.query)
assertDenyOnlyRequiresLibraryMembership(t, tc.query)
if !strings.Contains(tc.query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", tc.query)
}
})
}
}
func assertQualifiedItemSelect(t *testing.T, query string) {
t.Helper()
if !strings.HasPrefix(query, "SELECT mi.content_id,") {
t.Fatalf("SELECT list is not qualified with the mi alias, got:\n%s", query)
}
}
func assertNoLibraryJoin(t *testing.T, query string) {
t.Helper()
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("plain library JOIN can fan out discovery rows, got:\n%s", query)
}
}
func assertDenyOnlyRequiresLibraryMembership(t *testing.T, query string) {
t.Helper()
membershipPredicate := "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)"
if !strings.Contains(query, membershipPredicate) {
t.Fatalf("deny-only library filters must require positive library membership, got:\n%s", query)
}
if strings.Contains(query, "mil_scope_any.media_folder_id") {
t.Fatalf("positive membership predicate should not bind a specific library, got:\n%s", query)
}
}
func assertIntSliceArg(t *testing.T, args []any, idx int, want []int) {
t.Helper()
if idx >= len(args) {
t.Fatalf("missing arg %d in %v", idx, args)
}
got, ok := args[idx].([]int)
if !ok {
t.Fatalf("arg %d has type %T, want []int", idx, args[idx])
}
if len(got) != len(want) {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
}
}
func mustQuery(query string, _ []any) string { return query }
+194
View File
@@ -0,0 +1,194 @@
package catalog
import (
"context"
"errors"
"fmt"
"sort"
"github.com/Silo-Server/silo-server/internal/secret"
)
// SettingsStore is the read/write surface over server_settings shared by the
// raw *ServerSettingsRepo and the *EncryptedSettingsRepo decorator. Consumers
// depend on this interface (rather than the concrete repo) so they
// transparently gain at-rest encryption for sensitive keys once the decorator
// is wired in.
type SettingsStore interface {
Get(ctx context.Context, key string) (string, error)
Set(ctx context.Context, key, value string) error
GetAll(ctx context.Context) (map[string]string, error)
}
// SensitiveSettingKeys is the single source of truth for which server_settings
// keys hold secrets. It drives BOTH at-rest encryption (the decorator encrypts
// these on write, decrypts them on read) AND admin-API redaction, so the two
// can never drift apart.
//
// It was audited from the config loader's real inputs
// (internal/config/db_loader.go), the admin redaction map, and the other known
// server_settings consumers — the Audiobookshelf-compat JWT secret
// (internal/audiobooks/config.go), the watch-sync OAuth client config
// (internal/watchsync/service.go), and the legacy single-instance arr keys —
// including the legacy aliases still read as fallbacks (s3.operational_*,
// recommendations.openai_api_key) and redis.sentinel_password, which the old
// redaction map omitted.
//
// Adding a key here is only safe when every reader of that key goes through a
// cipher-aware path (the decorator, or a repo that calls
// secret.Cipher.DecryptIfEncrypted with secret.SettingsAAD). A raw SQL reader
// would otherwise see ciphertext after the backfill.
var SensitiveSettingKeys = map[string]bool{
// Core auth signing secret — no longer the encryption root (SECRET_KEY is),
// and now itself encrypted at rest under SECRET_KEY.
"auth.jwt_secret": true,
// Audiobookshelf-compat HMAC signing key (generated + persisted as hex).
"audiobooks.abs.jwt_secret": true,
// S3 — public assets, private internal, user DB, plus the legacy
// "operational" aliases db_loader still reads as fallbacks (L155-192;
// migration 086 copied but did not delete them).
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"s3.operational_access_key": true,
"s3.operational_secret_key": true,
"s3.operational_token_secret": true,
// Redis — url may embed credentials (redis://:pass@host); sentinel_password
// is read at db_loader L334 and was MISSING from the old redaction map.
"redis.url": true,
"redis.sentinel_password": true,
// Metadata / list-provider API keys.
"tmdb.api_key": true,
"mdblist.api_key": true,
"introdb.api_key": true,
// Subtitle AI (on-demand translation) API key.
"subtitle_ai.api_key": true,
// Recommendations embedding auth (+ legacy openai alias at db_loader L409).
"recommendations.embedding_auth_token": true,
"recommendations.openai_api_key": true,
// Watch-sync OAuth client credentials. The client_id entries are low-value
// but kept as a harmless safe superset and to match existing redaction.
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
// Legacy single-instance arr keys (superseded by per-integration rows, but
// still referenced by older request_integrations rows until backfilled).
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
}
// EncryptedSettingsRepo decorates a raw settings store, transparently
// encrypting the SensitiveSettingKeys on write and decrypting them on read.
// config.LoadFromDB and every other consumer keep seeing plaintext while the
// values rest as ciphertext in the database, GCM-bound to their key.
type EncryptedSettingsRepo struct {
// inner is the RAW store (no encryption). It must never be another
// EncryptedSettingsRepo, or writes would double-encrypt.
inner SettingsStore
cipher *secret.Cipher
}
// NewEncryptedSettingsRepo wraps a raw settings store with the cipher. Pass the
// concrete *ServerSettingsRepo (or an in-memory fake in tests) as inner.
func NewEncryptedSettingsRepo(inner SettingsStore, cipher *secret.Cipher) *EncryptedSettingsRepo {
return &EncryptedSettingsRepo{inner: inner, cipher: cipher}
}
var _ SettingsStore = (*EncryptedSettingsRepo)(nil)
// Set encrypts a sensitive, non-empty value (AAD-bound to its key) before
// delegating to the raw store; non-sensitive keys and empty values delegate
// unchanged. Encrypting an empty value is a no-op, so an absent secret stays an
// empty column rather than an envelope.
func (r *EncryptedSettingsRepo) Set(ctx context.Context, key, value string) error {
if SensitiveSettingKeys[key] && value != "" {
ct, err := r.cipher.Encrypt(value, secret.SettingsAAD(key))
if err != nil {
return fmt.Errorf("encrypt setting %q: %w", key, err)
}
value = ct
}
return r.inner.Set(ctx, key, value)
}
// Get reads a value and applies the read-path contract: legacy plaintext passes
// through, an enc:v1: value is decrypted, and a corrupt ciphertext errors.
func (r *EncryptedSettingsRepo) Get(ctx context.Context, key string) (string, error) {
value, err := r.inner.Get(ctx, key)
if err != nil {
return "", err
}
out, err := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if err != nil {
return "", fmt.Errorf("decrypt setting %q: %w", key, err)
}
return out, nil
}
// GetAll reads every setting and decrypts any enc:v1: value in place, so
// callers (notably config.LoadFromDB) receive a fully plaintext map.
func (r *EncryptedSettingsRepo) GetAll(ctx context.Context) (map[string]string, error) {
all, err := r.inner.GetAll(ctx)
if err != nil {
return nil, err
}
for key, value := range all {
out, derr := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if derr != nil {
return nil, fmt.Errorf("decrypt setting %q: %w", key, derr)
}
all[key] = out
}
return all, nil
}
// BackfillSensitiveSettings encrypts any plaintext value currently stored under
// a sensitive key. It is idempotent (already-encrypted and empty values are
// skipped) and best-effort: per-key failures are collected and returned but do
// not abort the sweep, so one unreadable row cannot block startup. Returns the
// number of keys newly encrypted.
func (r *EncryptedSettingsRepo) BackfillSensitiveSettings(ctx context.Context) (int, error) {
keys := make([]string, 0, len(SensitiveSettingKeys))
for key := range SensitiveSettingKeys {
keys = append(keys, key)
}
sort.Strings(keys) // deterministic order for logs and tests
encrypted := 0
var errs []error
for _, key := range keys {
// Read the RAW inner value (not through this decorator) so we can tell
// plaintext from an already-encrypted envelope.
raw, err := r.inner.Get(ctx, key)
if err != nil {
errs = append(errs, fmt.Errorf("read %q: %w", key, err))
continue
}
if raw == "" || secret.IsEncrypted(raw) {
continue
}
ct, err := r.cipher.Encrypt(raw, secret.SettingsAAD(key))
if err != nil {
errs = append(errs, fmt.Errorf("encrypt %q: %w", key, err))
continue
}
if err := r.inner.Set(ctx, key, ct); err != nil {
errs = append(errs, fmt.Errorf("write %q: %w", key, err))
continue
}
encrypted++
}
return encrypted, errors.Join(errs...)
}
@@ -0,0 +1,210 @@
package catalog
import (
"context"
"crypto/rand"
"testing"
"github.com/Silo-Server/silo-server/internal/secret"
)
// memSettings is an in-memory raw SettingsStore for DB-free decorator tests.
type memSettings struct{ m map[string]string }
func newMemSettings() *memSettings { return &memSettings{m: map[string]string{}} }
func (s *memSettings) Get(_ context.Context, key string) (string, error) { return s.m[key], nil }
func (s *memSettings) Set(_ context.Context, key, value string) error {
s.m[key] = value
return nil
}
func (s *memSettings) GetAll(_ context.Context) (map[string]string, error) {
out := make(map[string]string, len(s.m))
for k, v := range s.m {
out[k] = v
}
return out, nil
}
func newCipher(t *testing.T) *secret.Cipher {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return c
}
func pickSensitiveKey() string {
for k := range SensitiveSettingKeys {
return k
}
return "auth.jwt_secret"
}
func TestEncryptedSettings_SensitiveRoundTrip(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "tmdb.api_key"
const plain = "tmdb-secret-key-value"
if err := dec.Set(ctx, key, plain); err != nil {
t.Fatalf("Set: %v", err)
}
// Raw store must hold ciphertext.
if got := raw.m[key]; !secret.IsEncrypted(got) {
t.Fatalf("raw store value = %q, want enc:v1: ciphertext", got)
}
// Decorator Get must return plaintext.
got, err := dec.Get(ctx, key)
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != plain {
t.Fatalf("Get = %q, want %q", got, plain)
}
}
func TestEncryptedSettings_NonSensitivePassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "server.log_level"
if SensitiveSettingKeys[key] {
t.Fatalf("precondition: %q must be non-sensitive", key)
}
if err := dec.Set(ctx, key, "debug"); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m[key]; got != "debug" {
t.Fatalf("non-sensitive raw value = %q, want plaintext 'debug'", got)
}
}
func TestEncryptedSettings_LegacyPlaintextPassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
// Simulate a pre-encryption row: a sensitive key holding raw plaintext.
raw.m["tmdb.api_key"] = "legacy-plaintext"
got, err := dec.Get(ctx, "tmdb.api_key")
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != "legacy-plaintext" {
t.Fatalf("legacy plaintext Get = %q, want pass-through", got)
}
}
func TestEncryptedSettings_EmptySensitiveStaysEmpty(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", ""); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m["tmdb.api_key"]; got != "" {
t.Fatalf("empty sensitive value stored as %q, want empty (no envelope)", got)
}
}
func TestEncryptedSettings_GetAllDecrypts(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", "k1"); err != nil {
t.Fatalf("Set: %v", err)
}
if err := dec.Set(ctx, "server.mode", "integrated"); err != nil {
t.Fatalf("Set: %v", err)
}
all, err := dec.GetAll(ctx)
if err != nil {
t.Fatalf("GetAll: %v", err)
}
if all["tmdb.api_key"] != "k1" {
t.Fatalf("GetAll sensitive = %q, want decrypted 'k1'", all["tmdb.api_key"])
}
if all["server.mode"] != "integrated" {
t.Fatalf("GetAll non-sensitive = %q", all["server.mode"])
}
}
// TestSensitiveSettingKeys_Audited locks the audited allowlist: a dropped key is
// a plaintext leak (and breaks redaction), so the critical secrets must stay
// present, while values that are NOT secrets must stay absent (encrypting them
// would corrupt config reads).
func TestSensitiveSettingKeys_Audited(t *testing.T) {
mustHave := []string{
"auth.jwt_secret",
"audiobooks.abs.jwt_secret",
"s3.public_secret_key",
"s3.operational_secret_key", // legacy alias still read as a fallback
"redis.sentinel_password", // was missing from the old redaction map
"recommendations.openai_api_key",
"tmdb.api_key",
"requests.radarr.api_key",
"requests.sonarr.api_key",
"watchsync.trakt.client_secret",
}
for _, k := range mustHave {
if !SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys missing critical secret key %q", k)
}
}
mustNotHave := []string{
"server.mode",
"server.log_level",
"database.max_connections",
"jellyfin_compat.server_id",
}
for _, k := range mustNotHave {
if SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys must not contain non-secret key %q", k)
}
}
}
func TestEncryptedSettings_BackfillIdempotent(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
key := pickSensitiveKey()
raw.m[key] = "plaintext-secret" // legacy row
raw.m["server.mode"] = "integrated" // non-sensitive, must be left alone
n, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill: %v", err)
}
if n != 1 {
t.Fatalf("backfill encrypted %d, want 1", n)
}
if !secret.IsEncrypted(raw.m[key]) {
t.Fatalf("after backfill raw[%q] = %q, want ciphertext", key, raw.m[key])
}
if raw.m["server.mode"] != "integrated" {
t.Fatalf("backfill mutated non-sensitive key: %q", raw.m["server.mode"])
}
// Decryptable back to the original.
if got, _ := dec.Get(ctx, key); got != "plaintext-secret" {
t.Fatalf("post-backfill Get = %q, want original", got)
}
// Second run is a no-op.
n2, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill 2: %v", err)
}
if n2 != 0 {
t.Fatalf("second backfill encrypted %d, want 0 (no double-encrypt)", n2)
}
}
+13
View File
@@ -3,6 +3,7 @@ package catalog
import (
"context"
"errors"
"strings"
"testing"
"time"
@@ -136,3 +137,15 @@ func TestDeleteInBatchesReturnsError(t *testing.T) {
t.Fatalf("expected sentinel, got %v", err)
}
}
func TestLateOrphanCleanupUsesProvisionalSafetyPredicate(t *testing.T) {
conditions := normalizePredicateSQL(orphanedProvisionalMediaItemConditions)
for name, query := range map[string]string{
"collect": collectOrphanedProvisionalIDsByContentIDSQL,
"delete": deleteOrphanedProvisionalIDsByContentIDSQL,
} {
if !strings.Contains(normalizePredicateSQL(query), conditions) {
t.Fatalf("%s late orphan query does not use provisional safety predicate", name)
}
}
}
+142 -22
View File
@@ -563,16 +563,6 @@ func (r *FolderRepository) DeleteWithStats(
}
}
// Filter accumulated image dirs once, now that orphans are gone, against
// any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
}
stats.OrphanedImageDirs = filtered
}
// Phase 2: delete this folder's media_files (folder-tied, not item-tied).
if progress != nil {
progress(orphanTotal, orphanTotal, "Deleting media files")
@@ -592,23 +582,30 @@ func (r *FolderRepository) DeleteWithStats(
}
// Phase 3: delete remaining folder memberships (shared items kept; only the
// membership in this folder is removed).
// membership in this folder is removed). Deleting the memberships with
// RETURNING lets us catch skeleton items that a matcher created after the
// phase-1 orphan sweep but before the delete job reached this phase.
if progress != nil {
progress(orphanTotal, orphanTotal, "Removing library memberships")
}
if _, err := deleteInBatches(ctx, folderChildDeleteBatch, func(ctx context.Context) (int64, error) {
tag, e := r.pool.Exec(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)`, id, folderChildDeleteBatch)
if e != nil {
return 0, e
}
return tag.RowsAffected(), nil
}); err != nil {
lateOrphans, lateImageDirs, err := r.deleteFolderMemberships(ctx, id)
if err != nil {
return nil, fmt.Errorf("deleting media item links: %w", err)
}
stats.OrphanedItems += lateOrphans
for _, d := range lateImageDirs {
rawDirs[d] = struct{}{}
}
// Filter accumulated image dirs once, now that all item orphans are gone,
// against any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
}
stats.OrphanedImageDirs = filtered
}
// Phase 4: delete the now-lightweight folder row. Tolerate 0 rows so a
// resumed run that already removed it still succeeds.
@@ -656,6 +653,129 @@ func (r *FolderRepository) collectOrphanBatch(ctx context.Context, folderID, lim
return ids, nil
}
// deleteFolderMemberships removes this folder's remaining item memberships and
// deletes any content that becomes orphaned as a result. This is intentionally
// separate from the phase-1 orphan sweep because in-flight matchers can create
// new skeleton memberships after phase 1 has already drained.
func (r *FolderRepository) deleteFolderMemberships(ctx context.Context, folderID int) (int, []string, error) {
var orphaned int
var imageDirs []string
for {
var contentIDs []string
if err := retryOnDeadlock(ctx, func() error {
ids, err := r.deleteFolderMembershipBatch(ctx, folderID, folderChildDeleteBatch)
if err != nil {
return err
}
contentIDs = ids
return nil
}); err != nil {
return orphaned, imageDirs, err
}
if len(contentIDs) > 0 {
deleted, dirs, err := r.deleteOrphanedItemsByContentID(ctx, contentIDs)
if err != nil {
return orphaned, imageDirs, err
}
orphaned += deleted
imageDirs = append(imageDirs, dirs...)
}
if len(contentIDs) < folderChildDeleteBatch {
return orphaned, imageDirs, nil
}
}
}
func (r *FolderRepository) deleteFolderMembershipBatch(ctx context.Context, folderID, limit int) ([]string, error) {
rows, err := r.pool.Query(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)
RETURNING content_id`, folderID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var contentIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning deleted membership content_id: %w", err)
}
contentIDs = append(contentIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return contentIDs, nil
}
const collectOrphanedProvisionalIDsByContentIDSQL = `
SELECT mi.content_id
FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
const deleteOrphanedProvisionalIDsByContentIDSQL = `
DELETE FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
func (r *FolderRepository) deleteOrphanedItemsByContentID(ctx context.Context, contentIDs []string) (int, []string, error) {
if len(contentIDs) == 0 {
return 0, nil, nil
}
orphanIDs, err := r.collectOrphanedProvisionalIDsByContentID(ctx, contentIDs)
if err != nil {
return 0, nil, err
}
if len(orphanIDs) == 0 {
return 0, nil, nil
}
imageDirs, err := collectImageDirs(ctx, r.pool, orphanIDs)
if err != nil {
return 0, nil, err
}
var deleted int64
if err := retryOnDeadlock(ctx, func() error {
tag, err := r.pool.Exec(ctx, deleteOrphanedProvisionalIDsByContentIDSQL, orphanIDs)
if err != nil {
return err
}
deleted = tag.RowsAffected()
return nil
}); err != nil {
return 0, nil, err
}
return int(deleted), imageDirs, nil
}
func (r *FolderRepository) collectOrphanedProvisionalIDsByContentID(ctx context.Context, contentIDs []string) ([]string, error) {
rows, err := r.pool.Query(ctx, collectOrphanedProvisionalIDsByContentIDSQL, contentIDs)
if err != nil {
return nil, err
}
defer rows.Close()
var orphanIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning orphan content_id: %w", err)
}
orphanIDs = append(orphanIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return orphanIDs, nil
}
// dirSetToSlice returns the keys of set as a slice (nil if empty).
func dirSetToSlice(set map[string]struct{}) []string {
if len(set) == 0 {
+200
View File
@@ -0,0 +1,200 @@
package catalog
import (
"context"
"fmt"
"github.com/jackc/pgx/v5/pgxpool"
)
const defaultOrphanedProvisionalCleanupBatchSize = 1000
const orphanedMediaItemSafetyConditions = `NOT EXISTS (
SELECT 1 FROM public.media_item_libraries mil
WHERE mil.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.media_files mf
WHERE mf.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.episodes e
WHERE e.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.seasons s
WHERE s.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.library_collection_items lci
WHERE lci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_bookmarks ab
WHERE ab.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_playback_sessions aps
WHERE aps.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_rss_feeds arf
WHERE arf.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.downloads d
WHERE d.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.playback_history_admin pha
WHERE pha.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_bindings psib
WHERE psib.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_state psis
WHERE psis.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.podcast_feeds pf
WHERE pf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_favorites uf
WHERE uf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_downloads ud
WHERE ud.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_history_hidden_items uhhi
WHERE uhhi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid
WHERE uhid.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid_series
WHERE uhid_series.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_audio_preferences uap
WHERE uap.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_personal_collection_items upci
WHERE upci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_ratings ur
WHERE ur.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_series_playback_preferences uspp
WHERE uspp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_subtitle_preferences usp
WHERE usp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_history uwh
WHERE uwh.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_progress uwp
WHERE uwp.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watchlist uwl
WHERE uwl.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_favorite_items wpfi
WHERE wpfi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_history_exports wphe
WHERE wphe.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_scrobble_sessions wpss
WHERE wpss.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_rooms wtr
WHERE wtr.selected_content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_suggestions wts
WHERE wts.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.webhook_sync_item_state wsis
WHERE wsis.media_item_id = mi.content_id
)`
const orphanedProvisionalMediaItemConditions = `mi.status IN ('pending', 'unmatched', 'ambiguous')
AND ` + orphanedMediaItemSafetyConditions
const orphanedProvisionalMediaItemPredicate = `
WHERE ` + orphanedProvisionalMediaItemConditions
type OrphanedProvisionalCleanupStats struct {
Candidates int
Deleted int
}
type OrphanedProvisionalCleaner struct {
pool *pgxpool.Pool
}
func NewOrphanedProvisionalCleaner(pool *pgxpool.Pool) *OrphanedProvisionalCleaner {
return &OrphanedProvisionalCleaner{pool: pool}
}
func (c *OrphanedProvisionalCleaner) Cleanup(ctx context.Context, batchSize int) (OrphanedProvisionalCleanupStats, error) {
var stats OrphanedProvisionalCleanupStats
if c == nil || c.pool == nil {
return stats, fmt.Errorf("orphaned provisional cleanup is not configured")
}
if batchSize <= 0 {
batchSize = defaultOrphanedProvisionalCleanupBatchSize
}
if err := c.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM public.media_items mi `+orphanedProvisionalMediaItemPredicate,
).Scan(&stats.Candidates); err != nil {
return stats, fmt.Errorf("counting orphaned provisional media items: %w", err)
}
for {
tag, err := c.pool.Exec(ctx, `
WITH candidates AS (
SELECT mi.content_id
FROM public.media_items mi
`+orphanedProvisionalMediaItemPredicate+`
ORDER BY mi.content_id ASC
LIMIT $1
)
DELETE FROM public.media_items mi
USING candidates c
WHERE mi.content_id = c.content_id
`, batchSize)
if err != nil {
return stats, fmt.Errorf("deleting orphaned provisional media items: %w", err)
}
deleted := int(tag.RowsAffected())
stats.Deleted += deleted
if deleted < batchSize {
break
}
}
return stats, nil
}
+41
View File
@@ -0,0 +1,41 @@
package catalog
import (
"strings"
"testing"
)
func normalizePredicateSQL(s string) string {
return strings.Join(strings.Fields(s), " ")
}
func TestOrphanedProvisionalPredicatePreservesDurableMediaItemReferences(t *testing.T) {
predicate := normalizePredicateSQL(orphanedProvisionalMediaItemPredicate)
for _, want := range []string{
"public.abs_bookmarks ab WHERE ab.library_item_id = mi.content_id",
"public.abs_playback_sessions aps WHERE aps.content_id = mi.content_id",
"public.abs_rss_feeds arf WHERE arf.library_item_id = mi.content_id",
"public.episodes e WHERE e.series_id = mi.content_id",
"public.podcast_feeds pf WHERE pf.media_item_id = mi.content_id",
"public.seasons s WHERE s.series_id = mi.content_id",
"public.user_audio_preferences uap WHERE uap.series_id = mi.content_id",
"public.user_home_item_dismissals uhid WHERE uhid.media_item_id = mi.content_id",
"public.user_home_item_dismissals uhid_series WHERE uhid_series.series_id = mi.content_id",
"public.user_personal_collection_items upci WHERE upci.media_item_id = mi.content_id",
"public.user_series_playback_preferences uspp WHERE uspp.series_id = mi.content_id",
"public.user_subtitle_preferences usp WHERE usp.series_id = mi.content_id",
} {
if !strings.Contains(predicate, normalizePredicateSQL(want)) {
t.Fatalf("cleanup predicate missing durable reference guard %q", want)
}
}
for _, droppedTable := range []string{
"abs_collection_items",
"abs_playlist_items",
"abs_playlists",
} {
if strings.Contains(predicate, droppedTable) {
t.Fatalf("cleanup predicate must not reference dropped legacy table %q", droppedTable)
}
}
}
+3 -2
View File
@@ -185,8 +185,9 @@ func (q QueryDefinition) ValidateWithOptions(allowPersonalizedSorts, allowPerson
if normalized.MediaScope != "" &&
normalized.MediaScope != "movie" &&
normalized.MediaScope != "series" &&
normalized.MediaScope != "episode" {
return fmt.Errorf("media_scope must be 'movie', 'series', or 'episode'")
normalized.MediaScope != "episode" &&
normalized.MediaScope != "audiobook" {
return fmt.Errorf("media_scope must be 'movie', 'series', 'episode', or 'audiobook'")
}
if normalized.Match != "all" && normalized.Match != "any" {
+12
View File
@@ -60,6 +60,18 @@ func TestValidate_EpisodeMediaScope(t *testing.T) {
}
}
func TestValidate_AudiobookMediaScope(t *testing.T) {
qd := QueryDefinition{
MediaScope: "audiobook",
Match: "all",
Groups: []QueryGroup{},
Sort: QuerySort{Field: "title", Order: "asc"},
}
if err := qd.Validate(); err != nil {
t.Fatalf("expected audiobook media scope to be valid, got %v", err)
}
}
func TestValidateWithSortScope_RejectsPersonalizedSortWithoutProfileScope(t *testing.T) {
qd := QueryDefinition{
Match: "all",
+1 -1
View File
@@ -32,7 +32,7 @@ func ExtractFrame(ctx context.Context, opts FrameExtractOptions) ([]byte, string
runExtract = runFFmpegFrameExtract
}
resolvedAccel := playback.ResolveHWAccel(opts.HWAccel)
resolvedAccel := playback.ResolveHWAccelWithFFmpeg(opts.HWAccel, ffmpegPath)
resolvedDevice := opts.HWDevice
if resolvedDevice == "" && (resolvedAccel == "qsv" || resolvedAccel == "vaapi") {
resolvedDevice = playback.PickRenderDevice("")
+1 -1
View File
@@ -652,7 +652,7 @@ func (s *Service) extractFrameLocal(ctx context.Context, inputPath string, seekS
func (s *Service) resolveHWConfig() (string, string) {
s.hwResolveOnce.Do(func() {
s.resolvedHWAccel = playback.ResolveHWAccel(s.hwAccel)
s.resolvedHWAccel = playback.ResolveHWAccelWithFFmpeg(s.hwAccel, s.ffmpegPath)
s.resolvedHWDevice = s.hwDevice
if s.resolvedHWDevice == "" && (s.resolvedHWAccel == "qsv" || s.resolvedHWAccel == "vaapi") {
s.resolvedHWDevice = playback.PickRenderDevice("")
+19
View File
@@ -7,6 +7,11 @@ import (
"github.com/joho/godotenv"
)
// minSecretKeyLen is the minimum acceptable SECRET_KEY length (in characters).
// It mirrors secret.MinMasterKeyLen; the package is not imported here to keep
// the bootstrap loader dependency-free, so the two constants must stay in sync.
const minSecretKeyLen = 32
// BootstrapConfig holds the minimal config needed before database connection.
type BootstrapConfig struct {
DatabaseURL string
@@ -14,6 +19,10 @@ type BootstrapConfig struct {
Listen string
JFListen string
Mode string
// SecretKey is the master key (raw SECRET_KEY env value) from which the
// at-rest credential cipher derives its data key. It lives outside Postgres
// so encrypted secrets survive a full database compromise/dump.
SecretKey []byte
}
// LoadBootstrap loads bootstrap configuration from a .env file (if it exists)
@@ -28,6 +37,15 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
return nil, fmt.Errorf("DATABASE_URL is required (set in .env or environment)")
}
// SECRET_KEY is the at-rest encryption master key. It is required: the server
// must never fall back to a zero key or a key derived from another value, or
// encrypted credentials would not survive a database dump. godotenv has
// already loaded .env above, so dev sets it there.
secretKey := os.Getenv("SECRET_KEY")
if len(secretKey) < minSecretKeyLen {
return nil, fmt.Errorf("SECRET_KEY is required (>=%d chars); generate one with: openssl rand -base64 48", minSecretKeyLen)
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
@@ -51,5 +69,6 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
Listen: ":" + port,
JFListen: ":" + jfPort,
Mode: mode,
SecretKey: []byte(secretKey),
}, nil
}
+3 -4
View File
@@ -198,13 +198,12 @@ type authConfigRaw struct {
RefreshTokenExpiry string `yaml:"refresh_token_expiry"`
}
// AudiobookshelfCompatConfig holds the dedicated ABS-compat listener
// settings. The listener binds its own port (default :13378, ABS's
// AudiobookshelfCompatConfig holds the dedicated ABS-compat listener setting.
// The listener binds its own port (:13378, ABS's
// conventional port) and serves the full ABS protocol — login, libraries,
// items, sessions — without colliding with silo's SPA on the main listener.
type AudiobookshelfCompatConfig struct {
Listen string `yaml:"listen"`
PublicURL string `yaml:"public_url"`
Listen string `yaml:"-"`
}
// JellyfinCompatConfig holds compatibility proxy settings.
+4 -6
View File
@@ -356,15 +356,13 @@ func LoadFromDB(m map[string]string) (*Config, error) {
cfg.Auth.RefreshTokenExpiry = refreshTokenExpiry
// AudiobookshelfCompat — dedicated listener for ABS client apps. Keep the
// listener off unless the audiobooks feature flag is explicitly enabled.
audiobooksEnabled, err := boolOr(m, "audiobooks.enabled", false)
// listener off unless ABS compatibility is explicitly enabled.
absCompatEnabled, err := boolOr(m, "audiobookshelf_compat.enabled", false)
if err != nil {
return nil, err
}
if audiobooksEnabled {
// Default :13378 mirrors the real Audiobookshelf server convention.
cfg.AudiobookshelfCompat.Listen = stringOr(m, "audiobookshelf_compat.listen", ":13378")
cfg.AudiobookshelfCompat.PublicURL = stringOr(m, "audiobookshelf_compat.public_url", "http://127.0.0.1:13378")
if absCompatEnabled {
cfg.AudiobookshelfCompat.Listen = ":13378"
}
// JellyfinCompat
+3 -3
View File
@@ -34,8 +34,8 @@ func TestLoadFromDBMetadataPresignExpiryRejectsInvalidDuration(t *testing.T) {
}
}
func TestLoadFromDBAudiobooksFeatureFlagGatesCompatListener(t *testing.T) {
cfg, err := LoadFromDB(map[string]string{"audiobooks.enabled": "false"})
func TestLoadFromDBAudiobookshelfCompatFlagGatesCompatListener(t *testing.T) {
cfg, err := LoadFromDB(map[string]string{"audiobookshelf_compat.enabled": "false"})
if err != nil {
t.Fatalf("LoadFromDB() returned error: %v", err)
}
@@ -43,7 +43,7 @@ func TestLoadFromDBAudiobooksFeatureFlagGatesCompatListener(t *testing.T) {
t.Fatalf("disabled audiobooks listener = %q, want empty", cfg.AudiobookshelfCompat.Listen)
}
cfg, err = LoadFromDB(map[string]string{"audiobooks.enabled": "true"})
cfg, err = LoadFromDB(map[string]string{"audiobookshelf_compat.enabled": "true"})
if err != nil {
t.Fatalf("LoadFromDB() returned error: %v", err)
}
+331 -20
View File
@@ -6,11 +6,14 @@ import (
"errors"
"fmt"
"sort"
"strconv"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
var (
@@ -26,11 +29,135 @@ var (
)
type Repository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// AAD builders binding each history-import secret column to its row id.
func sourceAdminTokenAAD(sourceID int) string {
return secret.RowAAD("history_import_sources", "admin_token", strconv.Itoa(sourceID))
}
func connectServerAccessKeyAAD(sessionID string, index int, server ConnectServer) string {
return secret.RowAAD("history_import_connect_sessions", "servers_json.access_key", sessionID+":"+strconv.Itoa(index)+":"+server.ID)
}
func connectSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_connect_sessions", "connect_access_token", sessionID)
}
func plexServerAccessTokenAAD(sessionID string, index int, server PlexServer) string {
return secret.RowAAD("history_import_plex_sessions", "servers_json.access_token", sessionID+":"+strconv.Itoa(index)+":"+server.ClientIdentifier)
}
func plexSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_plex_sessions", "auth_token", sessionID)
}
func (r *Repository) encryptSourceAdminToken(sourceID int, token string) (string, error) {
return r.cipher.Encrypt(token, sourceAdminTokenAAD(sourceID))
}
func (r *Repository) encryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
if out[i].AccessKey == "" {
continue
}
accessKey, err := r.cipher.Encrypt(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) encryptPlaintextConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, bool, error) {
out := append([]ConnectServer(nil), servers...)
changed := false
for i := range out {
accessKey, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt connect server access key: %w", err)
}
if didChange {
out[i].AccessKey = accessKey
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
accessKey, err := r.cipher.DecryptIfEncrypted(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) marshalConnectServersForWrite(sessionID string, servers []ConnectServer) ([]byte, error) {
encrypted, err := r.encryptConnectServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
}
func (r *Repository) encryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
if out[i].AccessToken == "" {
continue
}
accessToken, err := r.cipher.Encrypt(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) encryptPlaintextPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, bool, error) {
out := append([]PlexServer(nil), servers...)
changed := false
for i := range out {
accessToken, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt plex server access token: %w", err)
}
if didChange {
out[i].AccessToken = accessToken
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
accessToken, err := r.cipher.DecryptIfEncrypted(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) marshalPlexServersForWrite(sessionID string, servers []PlexServer) ([]byte, error) {
encrypted, err := r.encryptPlexServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
}
func (r *Repository) ListEnabledSources(ctx context.Context) ([]Source, error) {
@@ -80,20 +207,44 @@ func (r *Repository) GetSourceByID(ctx context.Context, id int) (*Source, error)
}
func (r *Repository) CreateSource(ctx context.Context, input CreateSourceInput) (*Source, error) {
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order,
admin_token)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6, NULLIF($7, ''))
tx, err := r.pool.Begin(ctx)
if err != nil {
return nil, fmt.Errorf("begin creating history import source: %w", err)
}
defer tx.Rollback(ctx)
row := tx.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6)
RETURNING id, name, source_type, base_url, COALESCE(system_id, ''), enabled, sort_order,
(admin_token IS NOT NULL) AS has_admin_token,
created_at, updated_at`,
input.Name, input.SourceType, input.BaseURL, input.SystemID, input.Enabled, input.SortOrder,
input.AdminToken,
)
source, err := scanSource(row)
if err != nil {
return nil, fmt.Errorf("creating history import source: %w", err)
}
if input.AdminToken != "" {
encryptedToken, err := r.encryptSourceAdminToken(source.ID, input.AdminToken)
if err != nil {
return nil, fmt.Errorf("encrypt admin token for source %d: %w", source.ID, err)
}
result, err := tx.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, source.ID, encryptedToken)
if err != nil {
return nil, fmt.Errorf("setting admin token for source %d: %w", source.ID, err)
}
if result.RowsAffected() == 0 {
return nil, ErrSourceNotFound
}
source.HasAdminToken = true
}
if err := tx.Commit(ctx); err != nil {
return nil, fmt.Errorf("commit creating history import source: %w", err)
}
return source, nil
}
@@ -135,18 +286,22 @@ func (r *Repository) DeleteSource(ctx context.Context, id int) error {
}
func (r *Repository) CreateConnectSession(ctx context.Context, session ConnectSession) (*ConnectSession, error) {
serversJSON, err := json.Marshal(session.Servers)
serversJSON, err := r.marshalConnectServersForWrite(session.ID, session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling connect servers: %w", err)
}
accessToken, err := r.cipher.Encrypt(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt connect access token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_connect_sessions (
id, user_id, connect_user_id, connect_access_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.ConnectUserID, session.ConnectAccessToken, serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.ConnectUserID, accessToken, serversJSON, session.ExpiresAt,
)
stored, err := scanConnectSession(row)
stored, err := r.scanConnectSession(row)
if err != nil {
return nil, fmt.Errorf("creating connect session: %w", err)
}
@@ -158,7 +313,7 @@ func (r *Repository) GetConnectSession(ctx context.Context, userID int, sessionI
SELECT id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_connect_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := scanConnectSession(row)
session, err := r.scanConnectSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrConnectSessionNotFound
}
@@ -199,18 +354,22 @@ func (r *Repository) DeleteExpiredConnectSessions(ctx context.Context) error {
}
func (r *Repository) CreatePlexSession(ctx context.Context, session PlexSession) (*PlexSession, error) {
serversJSON, err := json.Marshal(session.Servers)
serversJSON, err := r.marshalPlexServersForWrite(session.ID, session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling plex servers: %w", err)
}
authToken, err := r.cipher.Encrypt(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt plex auth token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_plex_sessions (
id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(session.AuthToken), serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(authToken), serversJSON, session.ExpiresAt,
)
stored, err := scanPlexSession(row)
stored, err := r.scanPlexSession(row)
if err != nil {
return nil, fmt.Errorf("creating plex session: %w", err)
}
@@ -222,7 +381,7 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
SELECT id, user_id, pin_id, pin_code, COALESCE(auth_token, ''), servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_plex_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := scanPlexSession(row)
session, err := r.scanPlexSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrPlexSessionNotFound
}
@@ -239,14 +398,18 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
}
func (r *Repository) UpdatePlexSessionAuth(ctx context.Context, sessionID, authToken string, servers []PlexServer) error {
serversJSON, err := json.Marshal(servers)
serversJSON, err := r.marshalPlexServersForWrite(sessionID, servers)
if err != nil {
return fmt.Errorf("marshaling plex servers: %w", err)
}
encryptedToken, err := r.cipher.Encrypt(authToken, plexSessionTokenAAD(sessionID))
if err != nil {
return fmt.Errorf("encrypt plex auth token: %w", err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET auth_token = $2, servers_json = $3, updated_at = NOW()
WHERE id = $1`, sessionID, authToken, serversJSON)
WHERE id = $1`, sessionID, encryptedToken, serversJSON)
if err != nil {
return fmt.Errorf("updating plex session auth %s: %w", sessionID, err)
}
@@ -280,6 +443,136 @@ func (r *Repository) DeleteExpiredPlexSessions(ctx context.Context) error {
return nil
}
// BackfillSessionServerSecrets encrypts credentials nested inside history-import
// session server JSON. The top-level session token columns are handled by the
// generic column backfill, but Emby Connect access keys and Plex per-server
// access tokens live inside servers_json and need a typed JSON pass.
func (r *Repository) BackfillSessionServerSecrets(ctx context.Context) (int, error) {
connectN, connectErr := r.backfillConnectSessionServerSecrets(ctx)
plexN, plexErr := r.backfillPlexSessionServerSecrets(ctx)
return connectN + plexN, errors.Join(connectErr, plexErr)
}
func (r *Repository) backfillConnectSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_connect_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select connect sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan connect session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate connect sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []ConnectServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode connect session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextConnectServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt connect session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode connect session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_connect_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update connect session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) backfillPlexSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_plex_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select plex sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan plex session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate plex sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []PlexServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode plex session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextPlexServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt plex session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode plex session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update plex session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) ProfileExistsForUser(ctx context.Context, userID int, profileID string) (bool, error) {
var exists bool
if err := r.pool.QueryRow(ctx,
@@ -770,7 +1063,7 @@ func scanSources(rows pgx.Rows) ([]Source, error) {
return sources, rows.Err()
}
func scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
func (r *Repository) scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
var session ConnectSession
var serversJSON []byte
if err := scanner.Scan(
@@ -779,10 +1072,19 @@ func scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectS
); err != nil {
return nil, err
}
token, err := r.cipher.DecryptIfEncrypted(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt connect access token: %w", err)
}
session.ConnectAccessToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling connect session servers: %w", err)
}
session.Servers, err = r.decryptConnectServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
@@ -921,7 +1223,7 @@ func nonNilUnmatchedSamples(values []UnmatchedSample) []UnmatchedSample {
return values
}
func scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
func (r *Repository) scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
var session PlexSession
var authToken *string
var serversJSON []byte
@@ -935,10 +1237,19 @@ func scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession
if authToken != nil {
session.AuthToken = *authToken
}
token, err := r.cipher.DecryptIfEncrypted(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt plex auth token: %w", err)
}
session.AuthToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling plex session servers: %w", err)
}
session.Servers, err = r.decryptPlexServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
+9 -1
View File
@@ -21,10 +21,14 @@ var (
// SetSourceAdminToken stores an admin token for the given source.
func (r *Repository) SetSourceAdminToken(ctx context.Context, sourceID int, token string) error {
encryptedToken, err := r.encryptSourceAdminToken(sourceID, token)
if err != nil {
return fmt.Errorf("encrypt admin token for source %d: %w", sourceID, err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, sourceID, token)
WHERE id = $1`, sourceID, encryptedToken)
if err != nil {
return fmt.Errorf("setting admin token for source %d: %w", sourceID, err)
}
@@ -73,6 +77,10 @@ func (r *Repository) GetSourceWithAdminToken(ctx context.Context, sourceID int)
if err != nil {
return nil, "", fmt.Errorf("getting source %d with admin token: %w", sourceID, err)
}
adminToken, err = r.cipher.DecryptIfEncrypted(adminToken, sourceAdminTokenAAD(sourceID))
if err != nil {
return nil, "", fmt.Errorf("decrypt admin token for source %d: %w", sourceID, err)
}
return &s, adminToken, nil
}
+194 -6
View File
@@ -1,15 +1,34 @@
package historyimport
import (
"bytes"
"crypto/rand"
"encoding/json"
"fmt"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/secret"
)
// newTestRepo builds a Repository with a real cipher for scan-method tests.
func newTestRepo(t *testing.T) *Repository {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return &Repository{cipher: c}
}
type plexSessionScannerStub struct {
authToken *string
servers []PlexServer
authToken *string
servers []PlexServer
serversRaw []byte
}
func (s plexSessionScannerStub) Scan(dest ...any) error {
@@ -52,9 +71,13 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
return fmt.Errorf("dest[4] type = %T, want *string or **string", dest[4])
}
serversJSON, err := json.Marshal(s.servers)
if err != nil {
return err
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
}
serversDest, ok := dest[5].(*[]byte)
if !ok {
@@ -92,7 +115,7 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
t.Parallel()
session, err := scanPlexSession(plexSessionScannerStub{
session, err := newTestRepo(t).scanPlexSession(plexSessionScannerStub{
authToken: nil,
servers: []PlexServer{{
Name: "Plex Server",
@@ -118,4 +141,169 @@ func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
if session.Servers[0].ClientIdentifier != "server-1" {
t.Fatalf("session.Servers[0].ClientIdentifier = %q, want server-1", session.Servers[0].ClientIdentifier)
}
if session.Servers[0].AccessToken != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", session.Servers[0].AccessToken)
}
}
type connectSessionScannerStub struct {
accessToken string
servers []ConnectServer
serversRaw []byte
}
func (s connectSessionScannerStub) Scan(dest ...any) error {
if len(dest) != 9 {
return fmt.Errorf("unexpected scan dest count: %d", len(dest))
}
id, ok := dest[0].(*string)
if !ok {
return fmt.Errorf("dest[0] type = %T, want *string", dest[0])
}
*id = "connect-session-1"
userID, ok := dest[1].(*int)
if !ok {
return fmt.Errorf("dest[1] type = %T, want *int", dest[1])
}
*userID = 42
connectUserID, ok := dest[2].(*string)
if !ok {
return fmt.Errorf("dest[2] type = %T, want *string", dest[2])
}
*connectUserID = "connect-user-1"
accessToken, ok := dest[3].(*string)
if !ok {
return fmt.Errorf("dest[3] type = %T, want *string", dest[3])
}
*accessToken = s.accessToken
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
}
serversDest, ok := dest[4].(*[]byte)
if !ok {
return fmt.Errorf("dest[4] type = %T, want *[]byte", dest[4])
}
*serversDest = serversJSON
expiresAt, ok := dest[5].(*time.Time)
if !ok {
return fmt.Errorf("dest[5] type = %T, want *time.Time", dest[5])
}
*expiresAt = time.Date(2026, time.April, 1, 13, 0, 0, 0, time.UTC)
consumedAt, ok := dest[6].(**time.Time)
if !ok {
return fmt.Errorf("dest[6] type = %T, want **time.Time", dest[6])
}
*consumedAt = nil
createdAt, ok := dest[7].(*time.Time)
if !ok {
return fmt.Errorf("dest[7] type = %T, want *time.Time", dest[7])
}
*createdAt = time.Date(2026, time.April, 1, 12, 0, 0, 0, time.UTC)
updatedAt, ok := dest[8].(*time.Time)
if !ok {
return fmt.Errorf("dest[8] type = %T, want *time.Time", dest[8])
}
*updatedAt = time.Date(2026, time.April, 1, 12, 30, 0, 0, time.UTC)
return nil
}
func TestConnectSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []ConnectServer{{
ID: "emby-server-1",
Name: "Emby Server",
AccessKey: "server-access-key",
URL: "https://emby.example",
HasRemoteURL: true,
}}
raw, err := repo.marshalConnectServersForWrite("connect-session-1", servers)
if err != nil {
t.Fatalf("marshalConnectServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-access-key")) {
t.Fatalf("stored connect servers JSON leaked access key: %s", raw)
}
session, err := repo.scanConnectSession(connectSessionScannerStub{
accessToken: "connect-token",
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanConnectSession: %v", err)
}
if got := session.Servers[0].AccessKey; got != "server-access-key" {
t.Fatalf("session.Servers[0].AccessKey = %q, want server-access-key", got)
}
}
func TestPlexSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []PlexServer{{
Name: "Plex Server",
ClientIdentifier: "server-1",
AccessToken: "server-token",
RemoteURL: "https://plex.example",
Owned: true,
HasRemoteURL: true,
}}
raw, err := repo.marshalPlexServersForWrite("plex-session-1", servers)
if err != nil {
t.Fatalf("marshalPlexServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-token")) {
t.Fatalf("stored plex servers JSON leaked access token: %s", raw)
}
session, err := repo.scanPlexSession(plexSessionScannerStub{
authToken: nil,
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanPlexSession: %v", err)
}
if got := session.Servers[0].AccessToken; got != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", got)
}
}
func TestSourceAdminTokenEncryptionUsesSourceAAD(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
ciphertext, err := repo.encryptSourceAdminToken(42, "admin-token")
if err != nil {
t.Fatalf("encryptSourceAdminToken: %v", err)
}
if ciphertext == "admin-token" || bytes.Contains([]byte(ciphertext), []byte("admin-token")) {
t.Fatalf("encrypted admin token leaked plaintext: %q", ciphertext)
}
plaintext, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(42))
if err != nil {
t.Fatalf("decrypt with matching source AAD: %v", err)
}
if plaintext != "admin-token" {
t.Fatalf("decrypted admin token = %q, want admin-token", plaintext)
}
if _, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(43)); err == nil {
t.Fatalf("decrypt with different source AAD succeeded, want authentication failure")
}
}
+54
View File
@@ -0,0 +1,54 @@
package jellycompat
import (
"context"
"log/slog"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/catalog"
)
// ScopeResolver resolves a viewer request into an effective access scope.
// Implemented by *access.Resolver.
type ScopeResolver interface {
Resolve(ctx context.Context, input access.ResolveInput) (access.Scope, error)
}
// NewScopeAccessFilter returns an AccessFilterResolver backed by the shared
// access scope resolver, so account-level library restrictions
// (users.library_ids), profile library restrictions, user-disabled
// libraries, content-rating ceilings, and playback-quality ceilings all
// apply to the Jellyfin-compat API exactly as they do to the native API.
//
// PIN verification is skipped because compat login already verified the
// profile PIN (password#pin convention). Resolution failures fail closed:
// the viewer gets an empty library allowlist rather than full access.
func NewScopeAccessFilter(resolver ScopeResolver) AccessFilterResolver {
return func(ctx context.Context, userID int, profileID string) catalog.AccessFilter {
scope, err := resolver.Resolve(ctx, access.ResolveInput{
UserID: userID,
ProfileID: profileID,
SkipPINVerification: true,
})
if err != nil {
slog.Warn("jellycompat: access scope resolution failed; denying library access",
"user_id", userID,
"profile_id", profileID,
"error", err,
)
return catalog.AccessFilter{
AllowedLibraryIDs: []int{},
UserID: userID,
ProfileID: profileID,
}
}
return catalog.AccessFilter{
AllowedLibraryIDs: scope.AllowedLibraryIDs,
DisabledLibraryIDs: scope.DisabledLibraryIDs,
MaxContentRating: scope.MaxContentRating,
MaxPlaybackQuality: scope.MaxPlaybackQuality,
UserID: userID,
ProfileID: profileID,
}
}
}
+177
View File
@@ -0,0 +1,177 @@
package jellycompat
import (
"context"
"errors"
"reflect"
"testing"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/models"
)
type stubScopeResolver struct {
scope access.Scope
err error
input access.ResolveInput
}
func (s *stubScopeResolver) Resolve(_ context.Context, input access.ResolveInput) (access.Scope, error) {
s.input = input
return s.scope, s.err
}
func TestScopeAccessFilterMapsScope(t *testing.T) {
resolver := &stubScopeResolver{
scope: access.Scope{
UserID: 7,
ProfileID: "profile-1",
AllowedLibraryIDs: []int{2, 19},
LibrariesRestricted: true,
MaxContentRating: "PG-13",
MaxPlaybackQuality: "1080p",
},
}
filter := NewScopeAccessFilter(resolver)(context.Background(), 7, "profile-1")
if !resolver.input.SkipPINVerification {
t.Fatal("expected SkipPINVerification=true (PIN is verified at compat login)")
}
if resolver.input.UserID != 7 || resolver.input.ProfileID != "profile-1" {
t.Fatalf("unexpected resolve input: %+v", resolver.input)
}
if !reflect.DeepEqual(filter.AllowedLibraryIDs, []int{2, 19}) {
t.Fatalf("AllowedLibraryIDs = %v, want [2 19]", filter.AllowedLibraryIDs)
}
if filter.MaxContentRating != "PG-13" {
t.Fatalf("MaxContentRating = %q, want PG-13", filter.MaxContentRating)
}
if filter.MaxPlaybackQuality != "1080p" {
t.Fatalf("MaxPlaybackQuality = %q, want 1080p", filter.MaxPlaybackQuality)
}
if filter.UserID != 7 || filter.ProfileID != "profile-1" {
t.Fatalf("identity not propagated: %+v", filter)
}
}
func TestScopeAccessFilterUnrestrictedScope(t *testing.T) {
resolver := &stubScopeResolver{
scope: access.Scope{
UserID: 1,
DisabledLibraryIDs: []int{7},
},
}
filter := NewScopeAccessFilter(resolver)(context.Background(), 1, "profile-1")
if filter.AllowedLibraryIDs != nil {
t.Fatalf("AllowedLibraryIDs = %v, want nil (unrestricted)", filter.AllowedLibraryIDs)
}
if !reflect.DeepEqual(filter.DisabledLibraryIDs, []int{7}) {
t.Fatalf("DisabledLibraryIDs = %v, want [7]", filter.DisabledLibraryIDs)
}
}
func TestScopeAccessFilterFailsClosed(t *testing.T) {
resolver := &stubScopeResolver{err: errors.New("boom")}
filter := NewScopeAccessFilter(resolver)(context.Background(), 7, "profile-1")
if filter.AllowedLibraryIDs == nil || len(filter.AllowedLibraryIDs) != 0 {
t.Fatalf("AllowedLibraryIDs = %v, want empty non-nil allowlist (deny all)", filter.AllowedLibraryIDs)
}
}
type stubFolderSource struct {
enabled []*models.MediaFolder
byIDs []*models.MediaFolder
listByIDsArgs []int
}
func (s *stubFolderSource) GetEnabled(context.Context) ([]*models.MediaFolder, error) {
return s.enabled, nil
}
func (s *stubFolderSource) ListByIDs(_ context.Context, ids []int) ([]*models.MediaFolder, error) {
s.listByIDsArgs = ids
return s.byIDs, nil
}
func TestListUserLibrariesRestrictedAllowlist(t *testing.T) {
folders := &stubFolderSource{
byIDs: []*models.MediaFolder{
{ID: 2, Name: "TV Shows", Type: "series"},
{ID: 19, Name: "Movies", Type: "movies"},
},
}
svc := &directContentService{
folderRepo: folders,
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
AllowedLibraryIDs: []int{2, 19},
LibrariesRestricted: true,
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 7, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if !reflect.DeepEqual(folders.listByIDsArgs, []int{2, 19}) {
t.Fatalf("ListByIDs called with %v, want [2 19]", folders.listByIDsArgs)
}
if len(libraries) != 2 {
t.Fatalf("got %d libraries, want 2", len(libraries))
}
}
func TestListUserLibrariesFiltersDisabled(t *testing.T) {
svc := &directContentService{
folderRepo: &stubFolderSource{
enabled: []*models.MediaFolder{
{ID: 2, Name: "TV Shows", Type: "series"},
{ID: 7, Name: "XXX", Type: "movies"},
{ID: 19, Name: "Movies", Type: "movies"},
},
},
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
DisabledLibraryIDs: []int{7},
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 1, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if len(libraries) != 2 {
t.Fatalf("got %d libraries, want 2 (disabled library not filtered)", len(libraries))
}
for _, lib := range libraries {
if lib.ID == 7 {
t.Fatal("user-disabled library 7 still present in views list")
}
}
}
func TestListUserLibrariesEmptyAllowlistDeniesAll(t *testing.T) {
svc := &directContentService{
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
AllowedLibraryIDs: []int{},
LibrariesRestricted: true,
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 7, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if len(libraries) != 0 {
t.Fatalf("got %d libraries, want 0 for empty allowlist", len(libraries))
}
}
@@ -142,6 +142,7 @@ func TestBuildPlaybackSource_SeedsRequestedAudioStreamIndex(t *testing.T) {
version,
DeviceProfile{},
playbackInfoRequest{AudioStreamIndex: compatIntValuePtr(requestedAudioStreamIndex)},
true,
)
if source.SelectedAudioStreamIndex == nil {
+38 -12
View File
@@ -128,8 +128,7 @@ func (a *Authenticator) RequireSession(next http.Handler) http.Handler {
}
}
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
serveWithSession(next, w, r, session)
})
}
@@ -141,28 +140,55 @@ func safeTokenPrefix(token string) string {
return token[:8] + "..."
}
// serveWithSession injects the resolved compat session into the request context
// and continues the handler chain.
func serveWithSession(next http.Handler, w http.ResponseWriter, r *http.Request, session *Session) {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
}
// resolveCompatToken resolves a token to a compat session: a session-store token
// (normal login) or, matching Jellyfin, an sa_ admin API key (synthesized
// session bound to the key user's primary profile). Returns false when the token
// matches neither. keyAuth may be nil (resolveSession handles a nil receiver).
func resolveCompatToken(ctx context.Context, sessions *SessionStore, keyAuth *AdminAPIKeyAuthenticator, token string) (*Session, bool) {
if token == "" {
return nil, false
}
if session, ok := sessions.Get(token); ok {
return session, true
}
if strings.HasPrefix(token, "sa_") {
if session, _, _ := keyAuth.resolveSession(ctx, token); session != nil {
return session, true
}
}
return nil, false
}
// PlaybackSessionAuth creates middleware that falls back to playback session
// authentication for media stream endpoints where external players (e.g. libmpv)
// don't forward auth headers or query parameters.
func PlaybackSessionAuth(sessions *SessionStore, playbackStore *PlaybackSessionStore) func(next http.Handler) http.Handler {
func PlaybackSessionAuth(sessions *SessionStore, playbackStore *PlaybackSessionStore, keyAuth *AdminAPIKeyAuthenticator) func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Try standard token auth first.
// Try standard token auth first — a compat session token or an sa_
// admin key (synthesized session).
if token, ok := ExtractToken(r); ok {
if session, sessionOK := sessions.Get(token); sessionOK {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
if session, ok := resolveCompatToken(r.Context(), sessions, keyAuth, token); ok {
serveWithSession(next, w, r, session)
return
}
}
// Prefer the exact negotiated playback session when the client
// includes PlaySessionId on follow-up stream requests.
// Follow-up HLS requests (master/segment) carry only PlaySessionId,
// no auth header or api_key. Resolve the negotiated session's
// CompatToken — which for an API-key stream is itself the sa_ key,
// so it must go through the same session-or-API-key resolution.
if playSessionID := firstNonEmpty(r.URL.Query().Get("PlaySessionId"), r.URL.Query().Get("PlaySessionID")); playSessionID != "" {
if playSession, found := playbackStore.Get(playSessionID); found {
if session, ok := sessions.Get(playSession.CompatToken); ok {
ctx := context.WithValue(r.Context(), compatSessionKey, session)
next.ServeHTTP(w, r.WithContext(ctx))
if session, ok := resolveCompatToken(r.Context(), sessions, keyAuth, playSession.CompatToken); ok {
serveWithSession(next, w, r, session)
return
}
writeError(w, http.StatusUnauthorized, "Unauthorized", "Session expired")
+254 -39
View File
@@ -2,12 +2,15 @@ package jellycompat
import (
"context"
"fmt"
"log/slog"
"net/http"
"strings"
"sync"
"time"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type adminAPIKeyContextKey string
@@ -24,8 +27,17 @@ type apiKeyUserLoader interface {
}
type AdminAPIKeyAuthenticator struct {
keys apiKeyValidator
users apiKeyUserLoader
keys apiKeyValidator
users apiKeyUserLoader
provider userstore.UserStoreProvider
now func() time.Time
// profiles caches only the primary-profile lookup per user — never an
// authorization decision, which is re-checked on every request.
profiles *apiKeyProfileCache
lastUsedMu sync.Mutex
lastUsedAt map[int64]time.Time
}
type adminAPIKeyAuthResult struct {
@@ -36,11 +48,26 @@ type adminAPIKeyAuthResult struct {
ok bool
}
func NewAdminAPIKeyAuthenticator(keys apiKeyValidator, users apiKeyUserLoader) *AdminAPIKeyAuthenticator {
// NewAdminAPIKeyAuthenticator builds the authenticator. keys and users are
// required (a nil return disables API-key auth entirely). provider is optional:
// when present, an admin key can synthesize a compat session (see
// resolveSession); when nil, only the admin-bool path (RequireAdminAPIKey /
// RequireSessionOrAdminAPIKey) is available.
func NewAdminAPIKeyAuthenticator(keys apiKeyValidator, users apiKeyUserLoader, provider userstore.UserStoreProvider, now func() time.Time) *AdminAPIKeyAuthenticator {
if keys == nil || users == nil {
return nil
}
return &AdminAPIKeyAuthenticator{keys: keys, users: users}
if now == nil {
now = time.Now
}
return &AdminAPIKeyAuthenticator{
keys: keys,
users: users,
provider: provider,
now: now,
profiles: newAPIKeyProfileCache(),
lastUsedAt: make(map[int64]time.Time),
}
}
func AdminAPIKeyFromContext(ctx context.Context) bool {
@@ -77,46 +104,234 @@ func RequireSessionOrAdminAPIKey(sessionAuth *Authenticator, keyAuth *AdminAPIKe
}
}
// RequireSessionOrAPIKeySession authenticates either a compat session token or
// an sa_ admin API key. For an API key it injects a session synthesized for the
// key user's primary profile, so ordinary handlers (which read
// SessionFromContext) work unchanged for both auth modes — matching Jellyfin,
// where an API key authorizes the same endpoints a user token does. When
// API-key auth is unavailable (nil authenticator or no UserStoreProvider) an
// sa_ token falls through to session-token auth and is rejected there.
func RequireSessionOrAPIKeySession(sessionAuth *Authenticator, keyAuth *AdminAPIKeyAuthenticator) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if token, ok := ExtractToken(r); ok && strings.HasPrefix(token, "sa_") {
if session, result, handled := keyAuth.resolveSession(r.Context(), token); handled {
if !result.ok {
writeError(w, result.status, result.code, result.message)
return
}
serveWithSession(next, w, r, session)
return
}
}
sessionAuth.RequireSession(next).ServeHTTP(w, r)
})
}
}
// authenticate validates an admin API key and, on success, returns a result
// carrying the admin-API-key marker in context. Used by the server-to-server
// admin routes that act on the admin identity rather than a compat session.
func (a *AdminAPIKeyAuthenticator) authenticate(r *http.Request) adminAPIKeyAuthResult {
unauthorized := adminAPIKeyAuthResult{
ctx: r.Context(),
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Invalid API key",
token, _ := ExtractToken(r)
apiKey, _, res := a.validate(r.Context(), token)
if !res.ok {
return res
}
if a == nil || a.keys == nil || a.users == nil {
return unauthorized
}
token, ok := ExtractToken(r)
if !ok || !strings.HasPrefix(token, "sa_") {
return unauthorized
}
apiKey, err := a.keys.GetByKey(r.Context(), token)
if err != nil || apiKey == nil {
return unauthorized
}
user, err := a.users.GetByID(r.Context(), apiKey.UserID)
if err != nil || user == nil || !user.Enabled {
return unauthorized
}
if user.Role != "admin" {
return adminAPIKeyAuthResult{
ctx: r.Context(),
status: http.StatusForbidden,
code: "Forbidden",
message: "Admin access required",
}
}
go func(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := a.keys.UpdateLastUsed(ctx, id); err != nil {
slog.Debug("jellycompat api key last-used update failed", "id", id, "error", err)
}
}(apiKey.ID)
a.touchLastUsed(apiKey.ID)
return adminAPIKeyAuthResult{
ctx: context.WithValue(r.Context(), adminAPIKeyKey, true),
status: http.StatusOK,
ok: true,
}
}
// validate authenticates an admin API key by token and returns the key plus its
// owning admin user. The full authorization check — key lookup, user load,
// enabled, admin role — runs on EVERY call, so revocation (key deletion, user
// disable, role change) takes effect on the very next request. On failure the
// result carries the HTTP status/code/message: 401 for a missing/unknown/
// disabled key, 403 for a non-admin key.
func (a *AdminAPIKeyAuthenticator) validate(ctx context.Context, token string) (*models.APIKey, *models.User, adminAPIKeyAuthResult) {
unauthorized := adminAPIKeyAuthResult{
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Invalid API key",
}
if a == nil || a.keys == nil || a.users == nil {
return nil, nil, unauthorized
}
if !strings.HasPrefix(token, "sa_") {
return nil, nil, unauthorized
}
apiKey, err := a.keys.GetByKey(ctx, token)
if err != nil || apiKey == nil {
return nil, nil, unauthorized
}
user, err := a.users.GetByID(ctx, apiKey.UserID)
if err != nil || user == nil || !user.Enabled {
return nil, nil, unauthorized
}
if user.Role != "admin" {
return nil, nil, adminAPIKeyAuthResult{
status: http.StatusForbidden,
code: "Forbidden",
message: "Admin access required",
}
}
return apiKey, user, adminAPIKeyAuthResult{ok: true}
}
// touchLastUsed records key usage without blocking the request, throttled to at
// most once per apiKeyLastUsedInterval per key so per-request validation on the
// hot path (e.g. HLS segments) does not issue a DB write each time.
func (a *AdminAPIKeyAuthenticator) touchLastUsed(id int64) {
now := a.now()
a.lastUsedMu.Lock()
if last, ok := a.lastUsedAt[id]; ok && now.Sub(last) < apiKeyLastUsedInterval {
a.lastUsedMu.Unlock()
return
}
a.lastUsedAt[id] = now
a.lastUsedMu.Unlock()
go func(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := a.keys.UpdateLastUsed(ctx, id); err != nil {
slog.Debug("jellycompat api key last-used update failed", "id", id, "error", err)
}
}(id)
}
// resolveSession authenticates an sa_ API key and returns a compat session
// synthesized for the key user's primary profile, so ordinary browse/stream
// handlers (which read SessionFromContext) work unchanged for an API key.
//
// handled reports whether this authenticator can serve API-key session auth at
// all: false when the authenticator is nil or has no UserStoreProvider wired, in
// which case the caller must fall through to session-token auth. When handled is
// true, result.ok reports success; on failure result carries what to return.
//
// The key and owning user are re-validated on EVERY call (no cached
// authorization), so revocation is immediate. Only the primary-profile lookup is
// cached (see primaryProfile), sparing the hot path the ListProfiles round-trip.
func (a *AdminAPIKeyAuthenticator) resolveSession(ctx context.Context, token string) (*Session, adminAPIKeyAuthResult, bool) {
if a == nil || a.provider == nil {
return nil, adminAPIKeyAuthResult{}, false
}
apiKey, user, res := a.validate(ctx, token)
if !res.ok {
return nil, res, true
}
a.touchLastUsed(apiKey.ID)
profile, err := a.primaryProfile(ctx, user.ID)
if err != nil {
slog.Warn("jellycompat api key session synthesis failed", "user_id", user.ID, "error", err)
return nil, adminAPIKeyAuthResult{
status: http.StatusUnauthorized,
code: "Unauthorized",
message: "Unable to resolve a profile for this API key",
}, true
}
// Upstream Silo token fields are left empty and the expiry zero: an API key
// has no refreshable Silo token, and a zero expiry makes RequireSession's
// refresh path a no-op. Downstream services scope by StreamAppUserID/ProfileID.
return &Session{
Token: token,
Username: user.Username,
AccountUsername: user.Username,
ProfileID: profile.ID,
ProfileName: profile.Name,
PseudoUserID: PseudoUserID(user.ID, profile.ID),
StreamAppUserID: user.ID,
CreatedAt: a.now(),
}, adminAPIKeyAuthResult{ok: true}, true
}
// primaryProfile returns the account's primary profile, caching the result per
// user for apiKeyProfileCacheTTL. This caches profile DATA only — never an
// authorization decision — so it is safe against revocation (validated
// separately on every request).
func (a *AdminAPIKeyAuthenticator) primaryProfile(ctx context.Context, userID int) (userstore.Profile, error) {
if profile, ok := a.profiles.get(userID, a.now()); ok {
return profile, nil
}
store, err := a.provider.ForUser(ctx, userID)
if err != nil {
return userstore.Profile{}, fmt.Errorf("open user store for user %d: %w", userID, err)
}
profiles, err := store.ListProfiles(ctx)
if err != nil {
return userstore.Profile{}, fmt.Errorf("list profiles for user %d: %w", userID, err)
}
profile, err := selectPrimaryProfile(userID, profiles)
if err != nil {
return userstore.Profile{}, err
}
a.profiles.put(userID, profile, a.now().Add(apiKeyProfileCacheTTL))
return profile, nil
}
// selectPrimaryProfile returns the account's primary profile, falling back to
// the first profile (with a warning) when none is flagged primary, and erroring
// when the account has no profiles.
func selectPrimaryProfile(userID int, profiles []userstore.Profile) (userstore.Profile, error) {
if len(profiles) == 0 {
return userstore.Profile{}, fmt.Errorf("user %d has no profiles", userID)
}
for _, p := range profiles {
if p.IsPrimary {
return p, nil
}
}
slog.Warn("jellycompat api key: no primary profile found, using first",
"user_id", userID, "profile_id", profiles[0].ID)
return profiles[0], nil
}
const (
// apiKeyProfileCacheTTL bounds how long a primary-profile lookup is reused.
// Profiles are stable, so this is generous; it only affects how quickly a
// reassigned/renamed primary profile is picked up, never authorization
// (the key and user are re-validated on every request).
apiKeyProfileCacheTTL = 5 * time.Minute
// apiKeyLastUsedInterval throttles api_keys.last_used_at writes so that
// per-request validation on the hot path does not write each time.
apiKeyLastUsedInterval = time.Minute
)
type apiKeyProfileCacheEntry struct {
profile userstore.Profile
expiresAt time.Time
}
// apiKeyProfileCache caches the primary-profile lookup per user so the hot path
// (e.g. HLS segment requests) skips the ListProfiles round-trip. It caches only
// profile data, never an authorization decision.
type apiKeyProfileCache struct {
mu sync.RWMutex
entries map[int]apiKeyProfileCacheEntry
}
func newAPIKeyProfileCache() *apiKeyProfileCache {
return &apiKeyProfileCache{entries: make(map[int]apiKeyProfileCacheEntry)}
}
func (c *apiKeyProfileCache) get(userID int, now time.Time) (userstore.Profile, bool) {
c.mu.RLock()
entry, ok := c.entries[userID]
c.mu.RUnlock()
if !ok || !entry.expiresAt.After(now) {
return userstore.Profile{}, false
}
return entry.profile, true
}
func (c *apiKeyProfileCache) put(userID int, profile userstore.Profile, expiresAt time.Time) {
c.mu.Lock()
c.entries[userID] = apiKeyProfileCacheEntry{profile: profile, expiresAt: expiresAt}
c.mu.Unlock()
}
@@ -0,0 +1,365 @@
package jellycompat
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// fakeUserStore embeds userstore.UserStore so only ListProfiles needs an
// implementation; any other method call panics (the synthesis path must not
// touch them).
type fakeUserStore struct {
userstore.UserStore
profiles []userstore.Profile
listErr error
}
func (s *fakeUserStore) ListProfiles(context.Context) ([]userstore.Profile, error) {
return s.profiles, s.listErr
}
type fakeUserStoreProvider struct {
store *fakeUserStore
forErr error
forCalls int
}
func (p *fakeUserStoreProvider) ForUser(context.Context, int) (userstore.UserStore, error) {
p.forCalls++
if p.forErr != nil {
return nil, p.forErr
}
return p.store, nil
}
func (p *fakeUserStoreProvider) Close() error { return nil }
// adminKeyAuthWithPrimary builds an authenticator backed by an admin key whose
// account has a non-primary and a primary profile (in that order, so the test
// proves IsPrimary selection is order-independent).
func adminKeyAuthWithPrimary(t *testing.T, clock func() time.Time) (*AdminAPIKeyAuthenticator, *fakeUserStoreProvider, *fakeAPIKeyValidator) {
t.Helper()
validator := &fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}}
users := &fakeAPIKeyUserLoader{user: &models.User{ID: 2, Username: "admin", Role: "admin", Enabled: true}}
provider := &fakeUserStoreProvider{store: &fakeUserStore{profiles: []userstore.Profile{
{ID: "kids", Name: "Kids", IsPrimary: false},
{ID: "p1", Name: "Parent", IsPrimary: true},
}}}
keyAuth := NewAdminAPIKeyAuthenticator(validator, users, provider, clock)
if keyAuth == nil {
t.Fatal("expected non-nil authenticator")
}
return keyAuth, provider, validator
}
func apiKeyRequest() *http.Request {
req := httptest.NewRequest(http.MethodGet, "/Items", nil)
req.Header.Set("X-Emby-Token", "sa_test")
return req
}
func TestRequireSessionOrAPIKeySession_SynthesizesPrimaryProfile(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, provider, _ := adminKeyAuthWithPrimary(t, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
var got *Session
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = SessionFromContext(r.Context())
w.WriteHeader(http.StatusOK)
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest())
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if got == nil {
t.Fatal("expected synthesized session in context")
}
if got.StreamAppUserID != 2 {
t.Errorf("StreamAppUserID = %d, want 2", got.StreamAppUserID)
}
if got.ProfileID != "p1" || got.ProfileName != "Parent" {
t.Errorf("profile = %q/%q, want p1/Parent", got.ProfileID, got.ProfileName)
}
if want := PseudoUserID(2, "p1"); got.PseudoUserID != want {
t.Errorf("PseudoUserID = %s, want %s", got.PseudoUserID, want)
}
if got.Username != "admin" {
t.Errorf("Username = %q, want admin", got.Username)
}
// Refresh-skip invariants: no upstream Silo token, zero expiry.
if got.StreamAppAccessToken != "" || got.StreamAppRefreshToken != "" {
t.Errorf("expected empty upstream tokens, got access=%q refresh=%q", got.StreamAppAccessToken, got.StreamAppRefreshToken)
}
if !got.StreamAppTokenExpiry.IsZero() {
t.Error("expected zero StreamAppTokenExpiry so RequireSession skips refresh")
}
if provider.forCalls != 1 {
t.Errorf("ForUser calls = %d, want 1", provider.forCalls)
}
}
func TestRequireSessionOrAPIKeySession_RejectsNonAdminKey(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
validator := &fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}}
users := &fakeAPIKeyUserLoader{user: &models.User{ID: 2, Username: "bob", Role: "user", Enabled: true}}
provider := &fakeUserStoreProvider{store: &fakeUserStore{profiles: []userstore.Profile{{ID: "p1", IsPrimary: true}}}}
keyAuth := NewAdminAPIKeyAuthenticator(validator, users, provider, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for a non-admin key")
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest())
if rec.Code != http.StatusForbidden {
t.Fatalf("status = %d, want 403", rec.Code)
}
// The non-admin account's profiles must never be consulted.
if provider.forCalls != 0 {
t.Errorf("ForUser calls = %d, want 0 (rejected before synthesis)", provider.forCalls)
}
}
func TestRequireSessionOrAPIKeySession_RejectsUnknownKey(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, _ := adminKeyAuthWithPrimary(t, clock)
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for an unknown key")
}))
req := httptest.NewRequest(http.MethodGet, "/Items", nil)
req.Header.Set("X-Emby-Token", "sa_doesnotmatch")
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
// TestRequireSessionOrAPIKeySession_FallsThroughWhenSynthesisUnavailable guards
// the regression where the middleware fronts the whole browse group: a nil
// authenticator or one without a UserStoreProvider must fall through to session
// auth (clean 401) rather than panic.
func TestRequireSessionOrAPIKeySession_FallsThroughWhenSynthesisUnavailable(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
sessionAuth := &Authenticator{sessions: NewSessionStore(time.Hour, clock)}
cases := map[string]*AdminAPIKeyAuthenticator{
"nil authenticator": nil,
"nil provider": NewAdminAPIKeyAuthenticator(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
nil, clock,
),
}
for name, keyAuth := range cases {
t.Run(name, func(t *testing.T) {
h := RequireSessionOrAPIKeySession(sessionAuth, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run without a valid session")
}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, apiKeyRequest()) // must not panic
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
})
}
}
// TestResolveSession_RevalidatesEachCallCachesProfile: the key + owning user are
// validated on every call (so revocation is immediate), while the primary-profile
// lookup is cached until apiKeyProfileCacheTTL.
func TestResolveSession_RevalidatesEachCallCachesProfile(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, provider, validator := adminKeyAuthWithPrimary(t, clock)
ctx := context.Background()
for i := 1; i <= 3; i++ {
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatalf("resolve %d should succeed", i)
}
}
if validator.getCalls != 3 {
t.Errorf("GetByKey calls = %d, want 3 (re-validated every request)", validator.getCalls)
}
if provider.forCalls != 1 {
t.Errorf("ForUser calls = %d, want 1 (profile cached)", provider.forCalls)
}
// After the profile TTL, the profile is re-fetched (auth still every call).
now = now.Add(apiKeyProfileCacheTTL + time.Second)
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatal("post-TTL resolve should succeed")
}
if validator.getCalls != 4 {
t.Errorf("GetByKey calls = %d, want 4", validator.getCalls)
}
if provider.forCalls != 2 {
t.Errorf("ForUser calls = %d, want 2 after profile TTL", provider.forCalls)
}
}
// TestResolveSession_RevocationIsImmediate: once a key is revoked, the very next
// request fails — no cached authorization decision survives (Finding 2).
func TestResolveSession_RevocationIsImmediate(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, validator := adminKeyAuthWithPrimary(t, clock)
ctx := context.Background()
if s, _, handled := keyAuth.resolveSession(ctx, "sa_test"); !handled || s == nil {
t.Fatal("initial resolve should succeed")
}
validator.key = nil // revoke: GetByKey now returns not-found
s, res, handled := keyAuth.resolveSession(ctx, "sa_test")
if !handled {
t.Fatal("expected handled=true for an sa_ token")
}
if s != nil || res.ok {
t.Fatalf("revoked key must not resolve a session; got session=%v ok=%v", s != nil, res.ok)
}
if res.status != http.StatusUnauthorized {
t.Errorf("status = %d, want 401", res.status)
}
}
// TestPlaybackSessionAuth_APIKeyViaPlaySessionId: HLS follow-up requests carry
// only PlaySessionId. When the negotiated session's CompatToken is an sa_ key,
// PlaybackSessionAuth must still resolve it (Finding 1).
func TestPlaybackSessionAuth_APIKeyViaPlaySessionId(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, _ := adminKeyAuthWithPrimary(t, clock)
sessions := NewSessionStore(time.Hour, clock)
playback := NewPlaybackSessionStore(time.Hour, clock)
playback.Put(PlaybackSession{ID: "ps1", CompatToken: "sa_test"})
var got *Session
h := PlaybackSessionAuth(sessions, playback, keyAuth)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = SessionFromContext(r.Context())
w.WriteHeader(http.StatusOK)
}))
// No token header / no api_key — only PlaySessionId, like an HLS segment URL.
req := httptest.NewRequest(http.MethodGet, "/Videos/x/hls/p/seg0.ts?PlaySessionId=ps1", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if got == nil || got.Token != "sa_test" || got.StreamAppUserID != 2 {
t.Fatalf("expected synthesized api-key session (token sa_test, user 2); got %+v", got)
}
}
// TestPlaybackSessionAuth_RevokedAPIKeyViaPlaySessionId: a revoked key fails even
// through the PlaySessionId fallback.
func TestPlaybackSessionAuth_RevokedAPIKeyViaPlaySessionId(t *testing.T) {
now := fixedNow()
clock := func() time.Time { return now }
keyAuth, _, validator := adminKeyAuthWithPrimary(t, clock)
validator.key = nil // revoked
sessions := NewSessionStore(time.Hour, clock)
playback := NewPlaybackSessionStore(time.Hour, clock)
playback.Put(PlaybackSession{ID: "ps1", CompatToken: "sa_test"})
h := PlaybackSessionAuth(sessions, playback, keyAuth)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
t.Fatal("handler should not run for a revoked key")
}))
req := httptest.NewRequest(http.MethodGet, "/Videos/x/hls/p/seg0.ts?PlaySessionId=ps1", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
// TestValidatePseudoUser confirms the path-userId guard a synthesized session
// relies on: empty or matching ids pass; a mismatched id 404s.
func TestValidatePseudoUser(t *testing.T) {
session := &Session{PseudoUserID: PseudoUserID(2, "p1")}
if !validatePseudoUser(httptest.NewRecorder(), "", session) {
t.Error("empty userID should pass")
}
if !validatePseudoUser(httptest.NewRecorder(), session.PseudoUserID.String(), session) {
t.Error("matching userID should pass")
}
rec := httptest.NewRecorder()
if validatePseudoUser(rec, PseudoUserID(2, "other").String(), session) {
t.Error("mismatched userID should fail")
}
if rec.Code != http.StatusNotFound {
t.Errorf("mismatch status = %d, want 404", rec.Code)
}
}
func TestHandleUsers_ReturnsCallerAsSingleElement(t *testing.T) {
h := NewAuthHandler(&config.Config{}, nil, nil)
session := &Session{PseudoUserID: PseudoUserID(2, "p1"), Username: "admin"}
req := httptest.NewRequest(http.MethodGet, "/Users", nil)
req = req.WithContext(context.WithValue(req.Context(), compatSessionKey, session))
rec := httptest.NewRecorder()
h.HandleUsers(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
var users []map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &users); err != nil {
t.Fatalf("unmarshal: %v; body=%s", err, rec.Body.String())
}
if len(users) != 1 {
t.Fatalf("len = %d, want 1 (current profile only)", len(users))
}
if users[0]["Id"] != session.PseudoUserID.String() {
t.Errorf("Id = %v, want %s", users[0]["Id"], session.PseudoUserID.String())
}
if users[0]["Name"] != "admin" {
t.Errorf("Name = %v, want admin", users[0]["Name"])
}
}
func TestHandleUsers_RequiresSession(t *testing.T) {
h := NewAuthHandler(&config.Config{}, nil, nil)
rec := httptest.NewRecorder()
h.HandleUsers(rec, httptest.NewRequest(http.MethodGet, "/Users", nil))
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
func TestRouter_UsersRequiresAuth(t *testing.T) {
cfg, err := config.LoadFromDB(map[string]string{})
if err != nil {
t.Fatalf("LoadFromDB: %v", err)
}
router := NewRouter(Dependencies{Config: cfg})
rec := httptest.NewRecorder()
router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/Users", nil))
if rec.Code != http.StatusUnauthorized {
t.Fatalf("GET /Users status = %d, want 401", rec.Code)
}
}
+12 -4
View File
@@ -120,7 +120,7 @@ func TestRequireSession_NoAuthService_PassesThroughExpiredStreamAppToken(t *test
}
func TestRequireAdminAPIKey_AcceptsAdminKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
)
@@ -141,7 +141,7 @@ func TestRequireAdminAPIKey_AcceptsAdminKey(t *testing.T) {
}
func TestRequireAdminAPIKey_RejectsNonAdminKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"}},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "user", Enabled: true}},
)
@@ -159,7 +159,7 @@ func TestRequireAdminAPIKey_RejectsNonAdminKey(t *testing.T) {
}
func TestRequireAdminAPIKey_RejectsNilAPIKey(t *testing.T) {
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{returnNilWithoutError: true},
&fakeAPIKeyUserLoader{user: &models.User{ID: 2, Role: "admin", Enabled: true}},
)
@@ -178,7 +178,7 @@ func TestRequireAdminAPIKey_RejectsNilAPIKey(t *testing.T) {
func TestRequireAdminAPIKey_LastUsedUpdateHasDeadline(t *testing.T) {
called := make(chan bool, 1)
authn := NewAdminAPIKeyAuthenticator(
authn := newAdminAPIKeyAuthForTest(
&fakeAPIKeyValidator{
key: &models.APIKey{ID: 1, UserID: 2, Key: "sa_test"},
update: func(ctx context.Context, _ int64) error {
@@ -210,13 +210,21 @@ func TestRequireAdminAPIKey_LastUsedUpdateHasDeadline(t *testing.T) {
}
}
// newAdminAPIKeyAuthForTest builds an authenticator without a UserStoreProvider,
// exercising the admin-bool path only (no session synthesis).
func newAdminAPIKeyAuthForTest(keys apiKeyValidator, users apiKeyUserLoader) *AdminAPIKeyAuthenticator {
return NewAdminAPIKeyAuthenticator(keys, users, nil, nil)
}
type fakeAPIKeyValidator struct {
key *models.APIKey
returnNilWithoutError bool
getCalls int
update func(context.Context, int64) error
}
func (f *fakeAPIKeyValidator) GetByKey(_ context.Context, key string) (*models.APIKey, error) {
f.getCalls++
if f.returnNilWithoutError {
return nil, nil
}
+170 -3
View File
@@ -46,6 +46,14 @@ type itemAccessSource interface {
GetByIDs(ctx context.Context, contentIDs []string) ([]*models.MediaItem, error)
}
// folderListSource is the subset of *catalog.FolderRepository that
// directContentService relies on. Defined as an interface so tests can
// substitute a stub without standing up a Postgres pool.
type folderListSource interface {
GetEnabled(ctx context.Context) ([]*models.MediaFolder, error)
ListByIDs(ctx context.Context, ids []int) ([]*models.MediaFolder, error)
}
// seasonListSource is the subset of *catalog.SeasonRepository that
// directContentService relies on.
type seasonListSource interface {
@@ -61,6 +69,7 @@ type seasonListSource interface {
type episodeListSource interface {
ListBySeason(ctx context.Context, seriesID string, seasonNum int) ([]*models.Episode, error)
ListBySeriesGroupedBySeason(ctx context.Context, seriesID string) (map[int][]*models.Episode, error)
ListBySeriesIDs(ctx context.Context, seriesIDs []string) (map[string][]*models.Episode, error)
}
// directContentService implements ContentService by calling catalog repos directly.
@@ -70,7 +79,7 @@ type directContentService struct {
seasonRepo seasonListSource
episodeRepo episodeListSource
detailSvc *catalog.DetailService
folderRepo *catalog.FolderRepository
folderRepo folderListSource
storeProvider userstore.UserStoreProvider
accessFilter AccessFilterResolver
posterPresigner LibraryPosterPresigner
@@ -83,7 +92,7 @@ func newDirectContentService(
seasonRepo *catalog.SeasonRepository,
episodeRepo *catalog.EpisodeRepository,
detailSvc *catalog.DetailService,
folderRepo *catalog.FolderRepository,
folderRepo folderListSource,
storeProvider userstore.UserStoreProvider,
accessFilter AccessFilterResolver,
) *directContentService {
@@ -135,7 +144,12 @@ func (s *directContentService) ListUserLibraries(ctx context.Context, session *S
var folders []*models.MediaFolder
var err error
if len(filter.AllowedLibraryIDs) > 0 {
if filter.AllowedLibraryIDs != nil {
// A non-nil allowlist means the viewer is restricted; an empty
// allowlist grants access to no libraries at all.
if len(filter.AllowedLibraryIDs) == 0 {
return []upstreamUserLibrary{}, nil
}
folders, err = s.folderRepo.ListByIDs(ctx, filter.AllowedLibraryIDs)
} else {
folders, err = s.folderRepo.GetEnabled(ctx)
@@ -144,8 +158,16 @@ func (s *directContentService) ListUserLibraries(ctx context.Context, session *S
return nil, fmt.Errorf("list libraries: %w", err)
}
disabled := make(map[int]struct{}, len(filter.DisabledLibraryIDs))
for _, id := range filter.DisabledLibraryIDs {
disabled[id] = struct{}{}
}
libraries := make([]upstreamUserLibrary, 0, len(folders))
for _, f := range folders {
if _, ok := disabled[f.ID]; ok {
continue
}
lib := upstreamUserLibrary{
ID: f.ID,
Name: f.Name,
@@ -265,6 +287,18 @@ func (s *directContentService) BrowseItems(ctx context.Context, session *Session
if len(collected) > requestedLimit {
collected = collected[:requestedLimit]
}
// The handler's resolveUserStateForContentIDs overlay only covers items
// with their own progress rows, which a series never has — aggregate
// series watch state here so library views carry Played/UnplayedItemCount.
// The is_played path already did this via enrichListItemsUserData.
// That same absence is what keeps this safe from the handler overlay:
// userDataDTO only overrides item-level UserData when a direct progress
// row exists, and one never does for a series id.
if isPlayedFilter == "" {
s.enrichSeriesUserData(ctx, session, collected)
}
if totalKnown {
hasMore = totalFromCatalog > requestedOffset+requestedLimit
}
@@ -347,6 +381,17 @@ func (s *directContentService) GetItemDetail(ctx context.Context, session *Sessi
IsInProgress: progress.PositionSeconds > 0 && !progress.Completed,
}
}
// A series never has a progress row of its own, so roll watch
// state up from its episodes (mirrors applySeasonUserData) to
// give clients Played/UnplayedItemCount at the series level.
if result.UserData == nil && strings.EqualFold(result.Type, "series") && s.episodeRepo != nil {
if episodesBySeries, epErr := s.episodeRepo.ListBySeriesIDs(ctx, []string{contentID}); epErr == nil {
episodes := episodesBySeries[contentID]
progressMap := chunkedProgressByMediaItems(ctx, store, session.ProfileID, modelEpisodeContentIDs(episodes))
result.UserData = seriesUserDataFromEpisodes(episodes, progressMap)
}
}
}
}
@@ -545,6 +590,128 @@ func (s *directContentService) enrichListItemsUserData(ctx context.Context, sess
items[i].UserData = seasonUserDataFromProgress(progress)
}
}
s.enrichSeriesListUserData(ctx, session, store, items)
}
// enrichSeriesUserData is enrichSeriesListUserData with store acquisition,
// for callers that haven't already resolved the user store.
func (s *directContentService) enrichSeriesUserData(ctx context.Context, session *Session, items []upstreamListItem) {
if s.storeProvider == nil || len(items) == 0 {
return
}
store, err := s.userStore(ctx, session)
if err != nil {
return
}
s.enrichSeriesListUserData(ctx, session, store, items)
}
// enrichSeriesListUserData fills aggregated user data for series rows in a
// list response. A series never has a progress row of its own, so Played and
// UnplayedItemCount are rolled up from episode progress, mirroring
// applySeasonUserData semantics.
func (s *directContentService) enrichSeriesListUserData(ctx context.Context, session *Session, store userstore.UserStore, items []upstreamListItem) {
if s.episodeRepo == nil {
return
}
seriesIDs := make([]string, 0, len(items))
for i := range items {
if items[i].UserData == nil && items[i].ContentID != "" && strings.EqualFold(items[i].Type, "series") {
seriesIDs = append(seriesIDs, items[i].ContentID)
}
}
if len(seriesIDs) == 0 {
return
}
// Cap the rollup so a max-limit browse (compatBrowseMaxLimit) of a
// series-only library can't materialize hundreds of thousands of episode
// rows in one request. Series past the cap keep a nil UserData — the same
// shape they had before series aggregation existed. Real clients page at
// 20-100 items, well under the cap.
const maxSeriesUserDataRollups = 250
if len(seriesIDs) > maxSeriesUserDataRollups {
seriesIDs = seriesIDs[:maxSeriesUserDataRollups]
}
episodesBySeries, err := s.episodeRepo.ListBySeriesIDs(ctx, seriesIDs)
if err != nil {
return
}
episodeIDs := make([]string, 0, 256)
for _, episodes := range episodesBySeries {
episodeIDs = append(episodeIDs, modelEpisodeContentIDs(episodes)...)
}
progressMap := chunkedProgressByMediaItems(ctx, store, session.ProfileID, episodeIDs)
for i := range items {
if items[i].UserData != nil || !strings.EqualFold(items[i].Type, "series") {
continue
}
if episodes, ok := episodesBySeries[items[i].ContentID]; ok {
items[i].UserData = seriesUserDataFromEpisodes(episodes, progressMap)
}
}
}
// seriesUserDataFromEpisodes computes WatchedCount/UnplayedCount/
// InProgressCount/Played for a whole series from a pre-fetched progressMap.
// Pure function — no I/O. Counting semantics match the native API's series
// rollup (all episodes including specials; in-progress = started but not
// completed).
func seriesUserDataFromEpisodes(episodes []*models.Episode, progressMap map[string]userstore.WatchProgress) *catalog.SeasonUserData {
watched := 0
unplayed := 0
inProgress := 0
for _, ep := range episodes {
if ep == nil {
continue
}
progress, ok := progressMap[ep.ContentID]
if ok && progress.Completed {
watched++
continue
}
if ok && progress.PositionSeconds > 0 {
inProgress++
}
unplayed++
}
return &catalog.SeasonUserData{
WatchedCount: watched,
UnplayedCount: unplayed,
InProgressCount: inProgress,
Played: unplayed == 0 && len(episodes) > 0,
}
}
// modelEpisodeContentIDs returns the non-empty content ids of the given episodes.
func modelEpisodeContentIDs(episodes []*models.Episode) []string {
ids := make([]string, 0, len(episodes))
for _, ep := range episodes {
if ep != nil && ep.ContentID != "" {
ids = append(ids, ep.ContentID)
}
}
return ids
}
// chunkedProgressByMediaItems batches ListProgressByMediaItems calls when a
// series list expands to thousands of episode ids: per-user stores may be
// SQLite-backed (999 bind-variable default), and chunking also keeps
// Postgres IN-lists at a sane size. Returns an empty map (never nil); chunks
// that error are skipped.
func chunkedProgressByMediaItems(ctx context.Context, store userstore.UserStore, profileID string, mediaItemIDs []string) map[string]userstore.WatchProgress {
const chunkSize = 500
result := make(map[string]userstore.WatchProgress, len(mediaItemIDs))
for start := 0; start < len(mediaItemIDs); start += chunkSize {
chunk, err := store.ListProgressByMediaItems(ctx, profileID, mediaItemIDs[start:min(start+chunkSize, len(mediaItemIDs))])
if err != nil {
continue
}
for id, progress := range chunk {
result[id] = progress
}
}
return result
}
// enrichSeasonUserData adds aggregated user data for a season. Used by
+17
View File
@@ -176,6 +176,23 @@ func (h *AuthHandler) HandleCurrentUser(w http.ResponseWriter, r *http.Request)
writeJSON(w, http.StatusOK, h.userDTO(session))
}
// HandleUsers serves GET /Users.
//
// Jellyfin's user-list endpoint accepts any authenticated caller (including an
// API key) and is what Tunarr probes to verify an API-key connection (the key
// has no "me"). Silo separates login accounts from household profiles and is
// multi-account, so we return only the caller's own user as a single-element
// list rather than enumerating every account's profiles — listing all users
// would leak identities across households.
func (h *AuthHandler) HandleUsers(w http.ResponseWriter, r *http.Request) {
session := SessionFromContext(r.Context())
if session == nil {
writeError(w, http.StatusUnauthorized, "Unauthorized", "Missing authentication token")
return
}
writeJSON(w, http.StatusOK, []userDTOResponse{h.userDTO(session)})
}
// HandleUserByID serves GET /Users/{id}.
func (h *AuthHandler) HandleUserByID(w http.ResponseWriter, r *http.Request) {
session := SessionFromContext(r.Context())
+112 -19
View File
@@ -1183,27 +1183,45 @@ func (h *ItemsHandler) writeNextUpResponse(w http.ResponseWriter, r *http.Reques
return
}
// The scan stays on list-level data (per-item GetItemDetail fanout over
// the FULL next-up list was 5N DB roundtrips for users with long lists);
// only the sliced page below is upgraded to detail when the client asked
// for detail-level Fields — see upgradeProgressPageToDetail for why the
// listing's MediaSources are load-bearing for Infuse/SenPlayer.
pageIDs := make([]string, 0, len(results))
items := make([]baseItemDTO, 0, len(results))
for _, res := range results {
target, ok := episodeTargets[res.ContentID]
if !ok {
continue
}
// NextUp deliberately serves list-level data even when the client
// requests Fields=People|Chapters|MediaStreams|MediaSources, mirroring
// the Resume hydrator (handlers_items.go ~L1721). Per-item
// GetItemDetail fanout was 5N DB roundtrips for users with long
// next-up lists, and the detail data was never load-bearing for the
// row UX — clients refetch MediaSources via /Items/{id}/PlaybackInfo
// on play.
dto := h.mapper.itemFromList(target.Item, favorites[res.ContentID], progress[res.ContentID], query.requestedFields)
h.applyCompatEpisodeTarget(&dto, target)
stubDetailListFields(&dto, query.requestedFields)
pageIDs = append(pageIDs, res.ContentID)
items = append(items, dto)
}
total := len(items)
items = sliceBaseItems(items, query.startIndex, query.limit)
pageIDs = slicePageIDs(pageIDs, query.startIndex, query.limit)
if query.needsDetailFields {
for i := range items {
if i >= maxDetailUpgrades || i >= len(pageIDs) {
break
}
detail, detailErr := h.content.GetItemDetail(r.Context(), session, pageIDs[i], nil)
if detailErr != nil {
continue
}
h.rememberDetailImages(*detail)
dto := h.mapper.itemFromDetailWithFields(*detail, favorites[pageIDs[i]], progress[pageIDs[i]], query.requestedFields)
if target, ok := episodeTargets[pageIDs[i]]; ok {
h.applyCompatEpisodeTarget(&dto, target)
}
items[i] = dto
}
}
applyImageTypeLimit(items, query.imageTypeLimit)
writeJSON(w, http.StatusOK, queryResultDTO{
Items: items,
@@ -1212,6 +1230,22 @@ func (h *ItemsHandler) writeNextUpResponse(w http.ResponseWriter, r *http.Reques
})
}
// slicePageIDs applies sliceBaseItems' bounds to the parallel content-id
// slice so page positions stay aligned after StartIndex/Limit slicing.
func slicePageIDs(ids []string, startIndex, limit int) []string {
if startIndex < 0 {
startIndex = 0
}
if startIndex >= len(ids) {
return []string{}
}
if limit <= 0 {
limit = len(ids)
}
end := min(startIndex+limit, len(ids))
return ids[startIndex:end]
}
// HandleUpcoming serves GET /Shows/Upcoming.
//
// Android TV's show-detail page calls this endpoint to populate a single
@@ -1722,10 +1756,20 @@ func (h *ItemsHandler) handleResumeResponse(w http.ResponseWriter, r *http.Reque
}
type progressHydratedItem struct {
itemType string
dto baseItemDTO
itemType string
contentID string
isFavorite bool
entry upstreamProgress
target *compatEpisodeTarget
dto baseItemDTO
}
// maxDetailUpgrades caps how many returned-page items get re-mapped through
// the per-item detail path when the client requested detail-level Fields.
// Entries past the cap keep their stubbed list-level DTO. Guardrail against
// absurd client limits — normal Resume/NextUp requests are 20-40 items.
const maxDetailUpgrades = 100
func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, status string, query itemsQuery, typeSet map[string]bool, libraryID *int) ([]baseItemDTO, int, error) {
if len(typeSet) == 0 && libraryID == nil && !query.enableTotalRecordCount {
batchSize := min(max(query.limit*2, 48), 200)
@@ -1733,7 +1777,7 @@ func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, s
batchSize = 48
}
result := make([]baseItemDTO, 0, query.limit)
result := make([]progressHydratedItem, 0, query.limit)
offset := query.startIndex
for {
progressEntries, err := h.userData.ListProgress(ctx, session, status, batchSize, offset)
@@ -1752,14 +1796,14 @@ func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, s
if len(result) >= query.limit {
break
}
result = append(result, item.dto)
result = append(result, item)
}
if len(result) >= query.limit || len(progressEntries) < batchSize {
break
}
offset += len(progressEntries)
}
return result, 0, nil
return h.finishProgressPage(ctx, session, result, query, libraryID), 0, nil
}
batchSize := min(max(query.limit*3, 48), 200)
@@ -1767,7 +1811,7 @@ func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, s
batchSize = 48
}
items := make([]baseItemDTO, 0, query.limit)
items := make([]progressHydratedItem, 0, query.limit)
matchedCount := 0
offset := 0
@@ -1789,7 +1833,7 @@ func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, s
continue
}
if matchedCount >= query.startIndex && len(items) < query.limit {
items = append(items, item.dto)
items = append(items, item)
}
matchedCount++
}
@@ -1807,7 +1851,49 @@ func (h *ItemsHandler) loadProgressPage(ctx context.Context, session *Session, s
if query.enableTotalRecordCount {
total = matchedCount
}
return items, total, nil
return h.finishProgressPage(ctx, session, items, query, libraryID), total, nil
}
// finishProgressPage upgrades the returned page to detail-mapped DTOs when
// the client asked for detail-level Fields, then unwraps to the DTO slice.
func (h *ItemsHandler) finishProgressPage(ctx context.Context, session *Session, page []progressHydratedItem, query itemsQuery, libraryID *int) []baseItemDTO {
h.upgradeProgressPageToDetail(ctx, session, page, query, libraryID)
dtos := make([]baseItemDTO, 0, len(page))
for _, item := range page {
dtos = append(dtos, item.dto)
}
return dtos
}
// upgradeProgressPageToDetail re-maps the returned page of progress rows
// through the detail path when the client requested detail-level Fields
// (e.g. MediaSources). The scan loop in loadProgressPage stays on list-level
// data so type filtering over long in-progress lists never fans out — the
// per-ENTRY fanout was the 38s timeout in error-report-2026-05-08.md §6 —
// but the returned page is bounded by the request limit, so this costs the
// same as HandleLatest's existing per-item detail path. The listing's
// MediaSources ARE load-bearing for some clients: Infuse and SenPlayer build
// their Continue Watching rows from them and discard entries served only the
// list-level stub.
func (h *ItemsHandler) upgradeProgressPageToDetail(ctx context.Context, session *Session, page []progressHydratedItem, query itemsQuery, libraryID *int) {
if !query.needsDetailFields {
return
}
for i := range page {
if i >= maxDetailUpgrades {
break
}
detail, err := h.content.GetItemDetail(ctx, session, page[i].contentID, libraryID)
if err != nil {
continue
}
h.rememberDetailImages(*detail)
dto := h.mapper.itemFromDetailWithFields(*detail, page[i].isFavorite, &page[i].entry, query.requestedFields)
if page[i].target != nil {
h.applyCompatEpisodeTarget(&dto, *page[i].target)
}
page[i].dto = dto
}
}
func (h *ItemsHandler) hydrateProgressItems(ctx context.Context, session *Session, entries []upstreamProgress, fields map[string]bool, libraryID *int) ([]progressHydratedItem, error) {
@@ -1843,8 +1929,11 @@ func (h *ItemsHandler) hydrateProgressItems(ctx context.Context, session *Sessio
dto := h.mapper.itemFromList(item, favorites[entry.MediaItemID], &entry, fields)
stubDetailListFields(&dto, fields)
result = append(result, progressHydratedItem{
itemType: strings.ToLower(item.Type),
dto: dto,
itemType: strings.ToLower(item.Type),
contentID: entry.MediaItemID,
isFavorite: favorites[entry.MediaItemID],
entry: entry,
dto: dto,
})
continue
}
@@ -1853,8 +1942,12 @@ func (h *ItemsHandler) hydrateProgressItems(ctx context.Context, session *Sessio
h.applyCompatEpisodeTarget(&dto, target)
stubDetailListFields(&dto, fields)
result = append(result, progressHydratedItem{
itemType: "episode",
dto: dto,
itemType: "episode",
contentID: entry.MediaItemID,
isFavorite: favorites[entry.MediaItemID],
entry: entry,
target: &target,
dto: dto,
})
}
}
+29 -1
View File
@@ -20,6 +20,7 @@ import (
"github.com/go-chi/chi/v5"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
@@ -154,6 +155,22 @@ func (h *PlaybackHandler) playbackThresholds(ctx context.Context) userstore.Prog
return t
}
var errTranscode4KDisallowed = errors.New("4k video transcode disallowed by server settings")
// allow4KVideoTranscode reads the allow_4k_transcode server setting,
// defaulting to deny like the native playback handler.
func (h *PlaybackHandler) allow4KVideoTranscode(ctx context.Context) bool {
if h.SettingsRepo == nil {
return false
}
v, _ := h.SettingsRepo.Get(ctx, "allow_4k_transcode")
return v == "true"
}
func is4KResolution(res string) bool {
return access.CompareQuality(res, "2160p") >= 0
}
// NewPlaybackHandler creates a playback handler.
func NewPlaybackHandler(
cfg *config.Config,
@@ -300,6 +317,9 @@ func (h *PlaybackHandler) startRemoteTranscode(
initialSeekSeconds float64,
transcodeNodeURL string,
) error {
if !source.TranscodeAudio && is4KResolution(source.Version.Resolution) && !h.allow4KVideoTranscode(ctx) {
return errTranscode4KDisallowed
}
if d := float64(source.Version.Duration); d > 0 && initialSeekSeconds > d {
initialSeekSeconds = d
}
@@ -416,8 +436,9 @@ func (h *PlaybackHandler) HandlePlaybackInfo(w http.ResponseWriter, r *http.Requ
sources := make([]PlaybackMediaSource, 0, len(detail.Versions))
sourceDTOs := make([]mediaSourceDTO, 0, len(detail.Versions))
allow4KTranscode := h.allow4KVideoTranscode(r.Context())
for _, version := range detail.Versions {
source := h.buildPlaybackSource(routeItemID, playSessionID, version, profile, req)
source := h.buildPlaybackSource(routeItemID, playSessionID, version, profile, req, allow4KTranscode)
if req.MediaSourceID != "" && source.ID != req.MediaSourceID {
continue
}
@@ -513,6 +534,7 @@ func (h *PlaybackHandler) buildPlaybackSource(
version catalog.FileVersion,
profile DeviceProfile,
req playbackInfoRequest,
allow4KTranscode bool,
) PlaybackMediaSource {
sourceID := h.codec.EncodeIntID(EncodedIDMediaSource, int64(version.FileID))
enableDirectPlay := boolDefault(req.EnableDirectPlay, true)
@@ -531,6 +553,12 @@ func (h *PlaybackHandler) buildPlaybackSource(
allowAudioCopy &&
audioSupported
supportsTranscoding := boolDefault(req.EnableTranscoding, true) && profile.SupportsTranscoding(version)
// Don't offer full video encodes of 4K sources when allow_4k_transcode is
// off. Audio-only transcodes (transcodeAudio) stream-copy the video and
// stay available.
if supportsTranscoding && !transcodeAudio && !allow4KTranscode && is4KResolution(version.Resolution) {
supportsTranscoding = false
}
audioIndex := defaultAudioStreamIndex(version)
subtitleIndex := defaultSubtitleStreamIndex(version)
@@ -25,11 +25,11 @@ func (s *panicOnGetItemDetailContent) GetItemDetail(_ context.Context, _ *Sessio
// Fields=People|Chapters|MediaStreams|MediaSources, producing the 38s timeout
// observed in production for users with large in-progress lists.
//
// The function now serves list-level data unconditionally and stubs the four
// detail-only fields with empty (but non-nil) collections via
// stubDetailListFields. Standard Jellyfin clients refetch MediaSources via
// /Items/{id}/PlaybackInfo on play, so the detail data was never load-bearing
// for the Resume row UX.
// hydrateProgressItems is the SCAN phase and must stay detail-free: it runs
// over up to 200 entries per loop iteration purely for type filtering. The
// detail data clients need (real MediaSources — load-bearing for Infuse and
// SenPlayer Continue Watching rows) is served by upgradeProgressPageToDetail,
// which runs only on the returned page, bounded by the request limit.
func TestHydrateProgressItems_DoesNotCallGetItemDetail(t *testing.T) {
codec := NewResourceIDCodec()
@@ -65,6 +65,10 @@ func (c *countingEpisodeListSource) ListBySeriesGroupedBySeason(_ context.Contex
return out, nil
}
func (c *countingEpisodeListSource) ListBySeriesIDs(_ context.Context, _ []string) (map[string][]*models.Episode, error) {
return map[string][]*models.Episode{}, nil
}
// TestListSeasons_UsesBatchEpisodeFetch verifies ListSeasons makes exactly
// one ListBySeriesGroupedBySeason call (replacing N per-season ListBySeason
// calls) and exactly one ListProgressByMediaItems call (replacing N
+35 -7
View File
@@ -204,8 +204,11 @@ func (m *mapper) itemFromList(item upstreamListItem, isFavorite bool, progress *
// The values are minimal: each carries the required (non-omitempty) fields
// only. Clients that consume this data downstream will see a single source
// with a stub ID, a single Video stream at index 0, and a single placeholder
// person; standard Jellyfin clients refetch real MediaSources via
// /Items/{id}/PlaybackInfo on play, so this stub is never load-bearing.
// person. The stub is a FALLBACK, not a no-op: Infuse and SenPlayer build
// their Continue Watching rows from the listing's MediaSources, so pages
// within the detail-upgrade cap get real sources (upgradeProgressPageToDetail)
// and only overflow/error entries see the stub — which must therefore still
// look playable and serialize collections as empty, never null.
var (
stubDetailPerson = personDTO{ID: "0", Name: ""}
stubDetailMediaSource = mediaSourceDTO{ID: "0"}
@@ -214,10 +217,11 @@ var (
// stubDetailListFields sets the four detail-only fields on a list-mapped DTO
// to single-element placeholder slices when the client requested them via
// Fields=People|Chapters|MediaStreams|MediaSources. Used by the Resume
// hydrator, which deliberately skips per-item GetItemDetail fanout; clients
// that need real MediaSources refetch them via /Items/{id}/PlaybackInfo on
// play.
// Fields=People|Chapters|MediaStreams|MediaSources. Used by the Resume/NextUp
// scan phase, which deliberately skips per-item GetItemDetail fanout; the
// returned page is then re-mapped through the real detail path
// (upgradeProgressPageToDetail), so these stubs reach clients only for
// entries past the detail-upgrade cap or whose detail fetch failed.
func stubDetailListFields(dto *baseItemDTO, fields map[string]bool) {
if len(fields) == 0 {
return
@@ -234,7 +238,31 @@ func stubDetailListFields(dto *baseItemDTO, fields map[string]bool) {
dto.MediaStreams = []mediaStreamDTO{stubDetailMediaStream}
}
if fields["mediasources"] && dto.MediaSources == nil {
dto.MediaSources = []mediaSourceDTO{stubDetailMediaSource}
// The stub must still look PLAYABLE: some clients (SenPlayer) decide
// whether to render a Resume row entry from the listing's
// MediaSources, and an all-false playability stub makes them drop
// every item. Keep the stub ID ("0" — never a registered media source
// owner) but mirror the shape detailMediaSourceDTO produces; the real
// source set is still refetched via /Items/{id}/PlaybackInfo on play.
src := stubDetailMediaSource
src.Protocol = "File"
src.Type = "Default"
src.VideoType = "VideoFile"
src.Name = dto.Name
src.RunTimeTicks = dto.RunTimeTicks
src.SupportsTranscoding = true
src.SupportsDirectStream = true
src.SupportsDirectPlay = true
src.SupportsProbing = true
src.TranscodingSubProtocol = "hls"
src.MediaStreams = []mediaStreamDTO{stubDetailMediaStream}
// Real servers never serialize these as JSON null (always []/{});
// strict client deserializers fail the whole response on a null
// array, which empties the Resume/NextUp rows entirely.
src.Formats = []string{}
src.RequiredHTTPHeaders = map[string]string{}
src.MediaAttachments = []map[string]any{}
dto.MediaSources = []mediaSourceDTO{src}
}
}
@@ -157,3 +157,59 @@ func TestStubDetailListFields_FreshChapterMapPerCall(t *testing.T) {
t.Errorf("mutating dto a's chapter map leaked into dto b: b.Chapters[0][Name] = %v", got)
}
}
// TestStubDetailListFields_MediaSourceStubLooksPlayable locks in that the
// MediaSources stub advertises playability. Some clients (SenPlayer) decide
// whether to render a Resume row entry from the listing's MediaSources; an
// all-false playability stub made them drop every entry (the Continue
// Watching row showed empty while item-page resume worked fine).
func TestStubDetailListFields_MediaSourceStubLooksPlayable(t *testing.T) {
dto := baseItemDTO{Name: "Pilot", RunTimeTicks: 34800000000}
stubDetailListFields(&dto, map[string]bool{"mediasources": true})
if len(dto.MediaSources) != 1 {
t.Fatalf("MediaSources = %v, want single-element slice", dto.MediaSources)
}
src := dto.MediaSources[0]
if !src.SupportsDirectPlay || !src.SupportsDirectStream || !src.SupportsTranscoding || !src.SupportsProbing {
t.Errorf("stub source must advertise playability; got %+v", src)
}
if src.ID != "0" {
t.Errorf("stub source ID = %q, want \"0\" (never a registered media source owner)", src.ID)
}
if src.Name != "Pilot" || src.RunTimeTicks != 34800000000 {
t.Errorf("stub source should mirror item Name/RunTimeTicks; got Name=%q ticks=%d", src.Name, src.RunTimeTicks)
}
if len(src.MediaStreams) != 1 {
t.Errorf("stub source should nest a stream stub; got %v", src.MediaStreams)
}
// Null arrays (vs empty) break strict client deserializers — real servers
// always serialize these as []/{}.
if src.Formats == nil || src.RequiredHTTPHeaders == nil || src.MediaAttachments == nil {
t.Errorf("stub source must not serialize null collections; got Formats=%v RequiredHTTPHeaders=%v MediaAttachments=%v",
src.Formats, src.RequiredHTTPHeaders, src.MediaAttachments)
}
}
// TestStubDetailListFields_FreshMediaSourcePerCall guards against reference
// sharing across responses: the stub is built by copying the package-level
// stubDetailMediaSource value, which only stays safe while that var's
// reference-typed fields (slices/maps) remain nil. Mirrors the existing
// chapter-map guard.
func TestStubDetailListFields_FreshMediaSourcePerCall(t *testing.T) {
a := baseItemDTO{}
b := baseItemDTO{}
fields := map[string]bool{"mediasources": true}
stubDetailListFields(&a, fields)
stubDetailListFields(&b, fields)
a.MediaSources[0].RequiredHTTPHeaders["X-Mutated"] = "by-a"
a.MediaSources[0].MediaStreams[0].Index = 99
if len(b.MediaSources[0].RequiredHTTPHeaders) != 0 {
t.Error("mutating dto a's RequiredHTTPHeaders leaked into dto b")
}
if b.MediaSources[0].MediaStreams[0].Index == 99 {
t.Error("mutating dto a's nested stream leaked into dto b")
}
}
+165
View File
@@ -0,0 +1,165 @@
package jellycompat
import (
"context"
"errors"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
)
type stubSettingsReader struct {
values map[string]string
err error
}
func (s stubSettingsReader) Get(_ context.Context, key string) (string, error) {
if s.err != nil {
return "", s.err
}
return s.values[key], nil
}
func TestAllow4KVideoTranscode(t *testing.T) {
tests := []struct {
name string
repo SettingsReader
want bool
}{
{name: "nil repo defaults to deny", repo: nil, want: false},
{name: "unset defaults to deny", repo: stubSettingsReader{}, want: false},
{name: "read error defaults to deny", repo: stubSettingsReader{err: errors.New("read failed")}, want: false},
{name: "explicit false denies", repo: stubSettingsReader{values: map[string]string{"allow_4k_transcode": "false"}}, want: false},
{name: "explicit true allows", repo: stubSettingsReader{values: map[string]string{"allow_4k_transcode": "true"}}, want: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
h := &PlaybackHandler{SettingsRepo: tt.repo}
if got := h.allow4KVideoTranscode(context.Background()); got != tt.want {
t.Errorf("allow4KVideoTranscode() = %v, want %v", got, tt.want)
}
})
}
}
func TestIs4KResolution(t *testing.T) {
for res, want := range map[string]bool{
"2160p": true,
"4320p": true,
"1080p": false,
"720p": false,
"": false,
} {
if got := is4KResolution(res); got != want {
t.Errorf("is4KResolution(%q) = %v, want %v", res, got, want)
}
}
}
func TestBuildPlaybackSource4KVideoTranscodeGate(t *testing.T) {
version4K := catalog.FileVersion{
FileID: 1,
Resolution: "2160p",
CodecVideo: "hevc",
CodecAudio: "eac3",
Container: "mkv",
}
version1080 := version4K
version1080.Resolution = "1080p"
// Can only decode h264: HEVC versions need a full video transcode.
h264Only := DeviceProfile{
DirectPlayProfiles: []DirectPlayProfile{
{Type: "Video", Container: "mp4", VideoCodec: "h264", AudioCodec: "aac"},
},
}
// Decodes HEVC but not EAC3: transcoding path stream-copies the video.
hevcNoEac3 := DeviceProfile{
DirectPlayProfiles: []DirectPlayProfile{
{Type: "Video", Container: "mp4", VideoCodec: "h264,hevc", AudioCodec: "aac"},
},
}
tests := []struct {
name string
version catalog.FileVersion
profile DeviceProfile
allow4K bool
wantTranscoding bool
wantTranscodeAudio bool
}{
{
name: "4K video transcode blocked when disallowed",
version: version4K,
profile: h264Only,
allow4K: false,
wantTranscoding: false,
},
{
name: "4K video transcode offered when allowed",
version: version4K,
profile: h264Only,
allow4K: true,
wantTranscoding: true,
},
{
name: "4K audio-only transcode (video copy) stays allowed",
version: version4K,
profile: hevcNoEac3,
allow4K: false,
wantTranscoding: true,
wantTranscodeAudio: true,
},
{
name: "non-4K video transcode unaffected",
version: version1080,
profile: h264Only,
allow4K: false,
wantTranscoding: true,
},
}
h := &PlaybackHandler{codec: NewResourceIDCodec()}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
source := h.buildPlaybackSource("item", "ps", tt.version, tt.profile, playbackInfoRequest{}, tt.allow4K)
if source.SupportsTranscoding != tt.wantTranscoding {
t.Errorf("SupportsTranscoding = %v, want %v", source.SupportsTranscoding, tt.wantTranscoding)
}
if source.TranscodeAudio != tt.wantTranscodeAudio {
t.Errorf("TranscodeAudio = %v, want %v", source.TranscodeAudio, tt.wantTranscodeAudio)
}
})
}
}
func TestEnsureTranscodeSession4KGuard(t *testing.T) {
h := &PlaybackHandler{} // nil SettingsRepo: 4K video transcodes denied
source := PlaybackMediaSource{
FileID: 1,
Version: catalog.FileVersion{FileID: 1, Resolution: "2160p", CodecVideo: "hevc"},
}
if _, err := h.ensureTranscodeSession(context.Background(), "ps", "session", source); !errors.Is(err, errTranscode4KDisallowed) {
t.Errorf("ensureTranscodeSession() error = %v, want errTranscode4KDisallowed", err)
}
// Video-copy sessions pass the guard (and fail later on the missing file
// resolver, which is fine for this test).
source.TranscodeAudio = true
if _, err := h.ensureTranscodeSession(context.Background(), "ps", "session", source); errors.Is(err, errTranscode4KDisallowed) {
t.Error("ensureTranscodeSession() blocked a video-copy session")
}
}
func TestStartRemoteTranscode4KGuard(t *testing.T) {
h := &PlaybackHandler{} // nil SettingsRepo: 4K video transcodes denied
source := PlaybackMediaSource{
FileID: 1,
Version: catalog.FileVersion{FileID: 1, Resolution: "2160p", CodecVideo: "hevc"},
}
if err := h.startRemoteTranscode(context.Background(), "session", source, nil, 0, "http://node"); !errors.Is(err, errTranscode4KDisallowed) {
t.Errorf("startRemoteTranscode() error = %v, want errTranscode4KDisallowed", err)
}
}
+6 -5
View File
@@ -70,12 +70,12 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.SubtitleRepo != nil {
subtitleRepo = deps.SubtitleRepo
} else if deps.DB != nil {
subtitleRepo = subtitles.NewPgRepository(deps.DB)
subtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
}
itemsHandler := NewItemsHandler(deps.ContentService, deps.UserDataService, deps.IDCodec, deps.Config, deps.ImageCache, nextUpRepo, deps.BrowseRepo, deps.PersonRepo, deps.DetailSvc, deps.ItemRepo, deps.EpisodeRepo, deps.AccessFilterFn, subtitleRepo)
itemsHandler.recommender = deps.Recommender
autoscanHandler := NewAutoscanHandler(deps.FolderRepo, deps.ScanQueue, deps.IDCodec, itemsHandler)
adminAPIKeyAuth := NewAdminAPIKeyAuthenticator(deps.APIKeyValidator, deps.APIKeyUserLoader)
adminAPIKeyAuth := NewAdminAPIKeyAuthenticator(deps.APIKeyValidator, deps.APIKeyUserLoader, deps.UserStoreProvider, deps.Now)
autoscanVirtualFoldersRegistered := false
if deps.Authenticator != nil && adminAPIKeyAuth != nil && autoscanHandler != nil {
r.With(RequireSessionOrAdminAPIKey(deps.Authenticator, adminAPIKeyAuth)).
@@ -126,8 +126,9 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.Authenticator != nil {
r.Group(func(r chi.Router) {
r.Use(deps.Authenticator.RequireSession)
r.Use(RequireSessionOrAPIKeySession(deps.Authenticator, adminAPIKeyAuth))
r.Get("/Users/Me", authHandler.HandleCurrentUser)
r.Get("/Users", authHandler.HandleUsers)
r.Get("/Users/{id}", authHandler.HandleUserByID)
r.Get("/UserViews", itemsHandler.HandleViews)
r.Get("/UserViews/GroupingOptions", itemsHandler.HandleGroupingOptionsStub)
@@ -205,7 +206,7 @@ func NewRouter(deps Dependencies) chi.Router {
// Stream routes: use playback-session auth fallback for media players
// (e.g. libmpv) that don't forward auth headers or query parameters.
r.Group(func(r chi.Router) {
r.Use(PlaybackSessionAuth(deps.SessionStore, deps.PlaybackStore))
r.Use(PlaybackSessionAuth(deps.SessionStore, deps.PlaybackStore, adminAPIKeyAuth))
r.Method(http.MethodHead, "/Videos/{id}/stream", http.HandlerFunc(playbackHandler.HandleVideoStream))
r.Get("/Videos/{id}/stream", playbackHandler.HandleVideoStream)
r.Method(http.MethodHead, "/Videos/{id}/stream.{container}", http.HandlerFunc(playbackHandler.HandleVideoStream))
@@ -240,7 +241,7 @@ func withDefaults(deps Dependencies) Dependencies {
deps.SessionStore = NewPersistentSessionStore(
deps.Config.JellyfinCompat.SessionTTL,
deps.Now,
NewSessionRepository(deps.DB),
NewSessionRepository(deps.DB, deps.SecretCipher),
)
} else {
deps.SessionStore = NewSessionStore(deps.Config.JellyfinCompat.SessionTTL, deps.Now)
@@ -0,0 +1,111 @@
package jellycompat
import (
"testing"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
func ep(id string) *models.Episode {
return &models.Episode{ContentID: id}
}
// TestSeriesUserDataFromEpisodes covers the series watch-state rollup: a
// series has no progress row of its own, so Played/UnplayedCount/
// InProgressCount are aggregated from per-episode progress.
func TestSeriesUserDataFromEpisodes(t *testing.T) {
cases := []struct {
name string
episodes []*models.Episode
progress map[string]userstore.WatchProgress
wantWatched int
wantUnplayed int
wantInProgress int
wantPlayed bool
}{
{
name: "no episodes",
episodes: nil,
progress: map[string]userstore.WatchProgress{},
// Played must be false for an empty series — unplayed==0 alone
// must not flip it.
wantPlayed: false,
},
{
name: "all watched",
episodes: []*models.Episode{ep("a"), ep("b")},
progress: map[string]userstore.WatchProgress{
"a": {Completed: true},
"b": {Completed: true},
},
wantWatched: 2,
wantPlayed: true,
},
{
name: "mixed watched, in-progress, untouched",
episodes: []*models.Episode{ep("a"), ep("b"), ep("c")},
progress: map[string]userstore.WatchProgress{
"a": {Completed: true},
"b": {PositionSeconds: 120},
},
wantWatched: 1,
wantUnplayed: 2,
wantInProgress: 1,
wantPlayed: false,
},
{
name: "no progress at all",
episodes: []*models.Episode{ep("a")},
progress: map[string]userstore.WatchProgress{},
wantUnplayed: 1,
wantPlayed: false,
},
{
name: "nil episodes skipped",
episodes: []*models.Episode{nil, ep("a"), nil},
progress: map[string]userstore.WatchProgress{
"a": {Completed: true},
},
wantWatched: 1,
wantPlayed: true,
},
{
name: "zero-position progress row is not in-progress",
episodes: []*models.Episode{ep("a")},
progress: map[string]userstore.WatchProgress{
"a": {PositionSeconds: 0},
},
wantUnplayed: 1,
wantInProgress: 0,
wantPlayed: false,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := seriesUserDataFromEpisodes(tc.episodes, tc.progress)
if got.WatchedCount != tc.wantWatched {
t.Errorf("WatchedCount = %d, want %d", got.WatchedCount, tc.wantWatched)
}
if got.UnplayedCount != tc.wantUnplayed {
t.Errorf("UnplayedCount = %d, want %d", got.UnplayedCount, tc.wantUnplayed)
}
if got.InProgressCount != tc.wantInProgress {
t.Errorf("InProgressCount = %d, want %d", got.InProgressCount, tc.wantInProgress)
}
if got.Played != tc.wantPlayed {
t.Errorf("Played = %v, want %v", got.Played, tc.wantPlayed)
}
})
}
}
// TestModelEpisodeContentIDs verifies nil episodes and empty content ids are
// dropped — these ids feed SQL IN-lists.
func TestModelEpisodeContentIDs(t *testing.T) {
got := modelEpisodeContentIDs([]*models.Episode{nil, ep(""), ep("a"), ep("b")})
if len(got) != 2 || got[0] != "a" || got[1] != "b" {
t.Errorf("modelEpisodeContentIDs = %v, want [a b]", got)
}
}
+3 -1
View File
@@ -16,6 +16,7 @@ import (
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/recommendations"
"github.com/Silo-Server/silo-server/internal/scantrigger"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/subtitles"
"github.com/Silo-Server/silo-server/internal/userstore"
)
@@ -24,6 +25,7 @@ import (
type Dependencies struct {
Config *config.Config
DB *pgxpool.Pool
SecretCipher *secret.Cipher // at-rest credential cipher (required when DB is set)
ClientIPResolver *clientip.Resolver
HTTPClient *http.Client
Now func() time.Time
@@ -132,7 +134,7 @@ func (s *Server) SessionStore() *SessionStore {
// Call this once after constructing the server; goroutines stop when ctx is cancelled.
func (s *Server) StartBackgroundTasks(ctx context.Context) {
if s.deps.DB != nil {
repo := NewSessionRepository(s.deps.DB)
repo := NewSessionRepository(s.deps.DB, s.deps.SecretCipher)
StartSessionCleanup(ctx, repo, 1*time.Hour)
}
}
+35 -8
View File
@@ -9,21 +9,32 @@ import (
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
const jellycompatSessionColumns = `token, username, account_username, profile_id, profile_name, pseudo_user_id, streamapp_user_id, streamapp_access_token, streamapp_refresh_token, streamapp_token_expiry, created_at, expires_at`
// SessionRepository persists compat sessions in PostgreSQL.
type SessionRepository struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cipher *secret.Cipher
}
// NewSessionRepository creates a new compat session repository.
func NewSessionRepository(pool *pgxpool.Pool) *SessionRepository {
return &SessionRepository{pool: pool}
func NewSessionRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *SessionRepository {
return &SessionRepository{pool: pool, cipher: cipher}
}
func scanCompatSession(row pgx.Row) (*Session, error) {
// jellycompatTokenAAD binds a streamapp_* token ciphertext to its session row,
// keyed by the session token (the table's primary key). The session token
// itself is not encrypted (it is matched by value on lookup), so it is a stable,
// known identifier on both the read and write paths.
func jellycompatTokenAAD(column, token string) string {
return secret.RowAAD("jellycompat_sessions", column, token)
}
func (r *SessionRepository) scanCompatSession(row pgx.Row) (*Session, error) {
var session Session
err := row.Scan(
&session.Token,
@@ -45,6 +56,13 @@ func scanCompatSession(row pgx.Row) (*Session, error) {
}
return nil, fmt.Errorf("scan compat session: %w", err)
}
// Decrypt the bridged Silo access/refresh tokens (read-path contract).
if session.StreamAppAccessToken, err = r.cipher.DecryptIfEncrypted(session.StreamAppAccessToken, jellycompatTokenAAD("streamapp_access_token", session.Token)); err != nil {
return nil, fmt.Errorf("decrypt streamapp access token: %w", err)
}
if session.StreamAppRefreshToken, err = r.cipher.DecryptIfEncrypted(session.StreamAppRefreshToken, jellycompatTokenAAD("streamapp_refresh_token", session.Token)); err != nil {
return nil, fmt.Errorf("decrypt streamapp refresh token: %w", err)
}
return &session, nil
}
@@ -54,7 +72,16 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
session.Token = uuid.NewString()
}
_, err := r.pool.Exec(ctx, `
accessToken, err := r.cipher.Encrypt(session.StreamAppAccessToken, jellycompatTokenAAD("streamapp_access_token", session.Token))
if err != nil {
return fmt.Errorf("encrypt streamapp access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(session.StreamAppRefreshToken, jellycompatTokenAAD("streamapp_refresh_token", session.Token))
if err != nil {
return fmt.Errorf("encrypt streamapp refresh token: %w", err)
}
_, err = r.pool.Exec(ctx, `
INSERT INTO jellycompat_sessions (
token, username, account_username, profile_id, profile_name, pseudo_user_id,
streamapp_user_id, streamapp_access_token, streamapp_refresh_token,
@@ -84,8 +111,8 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
session.ProfileName,
session.PseudoUserID,
session.StreamAppUserID,
session.StreamAppAccessToken,
session.StreamAppRefreshToken,
accessToken,
refreshToken,
session.StreamAppTokenExpiry,
session.CreatedAt,
session.ExpiresAt,
@@ -98,7 +125,7 @@ func (r *SessionRepository) Upsert(ctx context.Context, session Session) error {
// GetByToken loads an active compat session by token.
func (r *SessionRepository) GetByToken(ctx context.Context, token string, now time.Time) (*Session, error) {
return scanCompatSession(r.pool.QueryRow(ctx,
return r.scanCompatSession(r.pool.QueryRow(ctx,
`SELECT `+jellycompatSessionColumns+`
FROM jellycompat_sessions
WHERE token = $1 AND expires_at > $2`,
@@ -0,0 +1,51 @@
package jellycompat
import "testing"
// TestSlicePageIDs locks slicePageIDs to sliceBaseItems' exact bounds
// semantics — the two slice the same page in parallel in
// writeNextUpResponse, and any divergence would attach detail upgrades to
// the wrong DTOs.
func TestSlicePageIDs(t *testing.T) {
ids := []string{"a", "b", "c", "d"}
cases := []struct {
name string
startIndex int
limit int
want []string
}{
{"plain page", 0, 2, []string{"a", "b"}},
{"offset page", 1, 2, []string{"b", "c"}},
{"limit past end clamps", 2, 10, []string{"c", "d"}},
{"start past end is empty", 9, 2, []string{}},
{"zero limit returns rest", 1, 0, []string{"b", "c", "d"}},
{"negative start clamps to zero", -3, 2, []string{"a", "b"}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := slicePageIDs(ids, tc.startIndex, tc.limit)
if len(got) != len(tc.want) {
t.Fatalf("len = %d, want %d (%v)", len(got), len(tc.want), got)
}
for i := range got {
if got[i] != tc.want[i] {
t.Errorf("got[%d] = %q, want %q", i, got[i], tc.want[i])
}
}
// Mirror against sliceBaseItems on a same-length DTO slice.
dtos := make([]baseItemDTO, len(ids))
for i := range ids {
dtos[i] = baseItemDTO{ID: ids[i]}
}
sliced := sliceBaseItems(dtos, tc.startIndex, tc.limit)
if len(sliced) != len(got) {
t.Fatalf("sliceBaseItems len %d diverges from slicePageIDs len %d", len(sliced), len(got))
}
for i := range sliced {
if sliced[i].ID != got[i] {
t.Errorf("alignment broken at %d: dto %q vs id %q", i, sliced[i].ID, got[i])
}
}
})
}
}
+17 -1
View File
@@ -165,6 +165,10 @@ func (h *PlaybackHandler) HandleMasterManifest(w http.ResponseWriter, r *http.Re
return
}
if err := h.startRemoteTranscode(r.Context(), playSession.UpstreamSessionID, *source, file, playSession.InitialSeekSeconds, tcNode.URL); err != nil {
if errors.Is(err, errTranscode4KDisallowed) {
writeError(w, http.StatusForbidden, "Forbidden", "4K video transcoding is disabled on this server")
return
}
writeError(w, http.StatusBadGateway, "TranscodeStartFailed", "Transcode node rejected the request")
return
}
@@ -182,6 +186,10 @@ func (h *PlaybackHandler) HandleMasterManifest(w http.ResponseWriter, r *http.Re
// Ensure the transcode process is running.
_, err = h.ensureTranscodeManifest(r.Context(), session, playSession.ID, *source)
if err != nil {
if errors.Is(err, errTranscode4KDisallowed) {
writeError(w, http.StatusForbidden, "Forbidden", "4K video transcoding is disabled on this server")
return
}
if errors.Is(err, playback.ErrManifestNotReady) {
writeError(w, http.StatusServiceUnavailable, "NotReady", "Transcode playlist not ready")
return
@@ -228,6 +236,10 @@ func (h *PlaybackHandler) HandleHLSManifest(w http.ResponseWriter, r *http.Reque
// Ensure the transcode process is running.
_, err := h.ensureTranscodeManifest(r.Context(), session, playSession.ID, *source)
if err != nil {
if errors.Is(err, errTranscode4KDisallowed) {
writeError(w, http.StatusForbidden, "Forbidden", "4K video transcoding is disabled on this server")
return
}
if errors.Is(err, playback.ErrManifestNotReady) {
writeError(w, http.StatusServiceUnavailable, "NotReady", "Transcode playlist not ready")
return
@@ -810,6 +822,9 @@ func (h *PlaybackHandler) ensureTranscodeSession(ctx context.Context, playSessio
if existing := h.getTranscodeSession(upstreamSessionID); existing != nil {
return existing, nil
}
if !source.TranscodeAudio && is4KResolution(source.Version.Resolution) && !h.allow4KVideoTranscode(ctx) {
return nil, errTranscode4KDisallowed
}
if h.fileResolver == nil {
return nil, fmt.Errorf("file resolver not available")
}
@@ -1019,8 +1034,9 @@ func (h *PlaybackHandler) createStaticPlaySession(ctx context.Context, session *
playSessionID := h.codec.EncodeStringID(EncodedIDPlaySession, uuidNewString())
sources := make([]PlaybackMediaSource, 0, len(detail.Versions))
allow4KTranscode := h.allow4KVideoTranscode(ctx)
for _, version := range detail.Versions {
source := h.buildPlaybackSource(routeID, playSessionID, version, DeviceProfile{}, playbackInfoRequest{})
source := h.buildPlaybackSource(routeID, playSessionID, version, DeviceProfile{}, playbackInfoRequest{}, allow4KTranscode)
sources = append(sources, source)
}
+98 -14
View File
@@ -70,6 +70,10 @@ type metadataItemRepo interface {
ListUnmatchedByFolderAndPathPrefix(ctx context.Context, folderID int, pathPrefix string, limit int) ([]string, error)
}
type metadataItemDeleteRepo interface {
Delete(ctx context.Context, contentID string) ([]string, error)
}
type metadataProviderIDRepo interface {
GetByContentID(ctx context.Context, contentID string) ([]*models.MediaItemProviderID, error)
ReplaceByContentID(ctx context.Context, contentID string, providerIDs map[string]string) error
@@ -3741,18 +3745,9 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
return s.hooks.createOrFindSkeleton(ctx, file, folderID)
}
libraryType := ""
if s != nil && s.folderRepo != nil && folderID > 0 {
folder, err := s.folderRepo.GetByID(ctx, folderID)
if err != nil {
slog.Warn("metadata: failed to load folder context for path classification",
"folder_id", folderID,
"file_path", file.FilePath,
"error", err,
)
} else if folder != nil {
libraryType = folder.Type
}
libraryType, err := s.folderTypeForSkeleton(ctx, folderID)
if err != nil {
return nil, err
}
contentRootPath := filepath.Dir(file.FilePath)
@@ -4027,7 +4022,12 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
// pre-confirmation ownership claims. Matching is driven by explicit IDs or
// later provider confirmation to avoid cross-root false merges.
// No existing item — create skeleton.
// No existing item — create skeleton. Re-check the folder state while still
// under the dedup lock so a library disabled for deletion cannot receive new
// skeleton items from a matcher that passed an older outer enabled check.
if _, err := s.folderTypeForSkeleton(ctx, folderID); err != nil {
return nil, err
}
contentID, err := generateContentID()
if err != nil {
return nil, fmt.Errorf("generate content id: %w", err)
@@ -4054,10 +4054,53 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
// Link file, claim root, and create library membership.
if err := s.fileRepo.UpdateContentID(ctx, file.ID, contentID); err != nil {
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up unlinked skeleton",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("linking file to skeleton: %w", err)
}
if _, err := s.folderTypeForSkeleton(ctx, folderID); err != nil {
if clearErr := s.fileRepo.UpdateContentID(ctx, file.ID, ""); clearErr != nil {
slog.Warn("metadata: failed to clear file link after disabled skeleton folder",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", clearErr,
)
}
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up skeleton for disabled folder",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("validating skeleton folder before membership: %w", err)
}
if err := s.upsertLibraryMembership(ctx, contentID, folderID); err != nil {
s.logLibraryMembershipError("upserting skeleton membership", contentID, folderID, err)
if clearErr := s.fileRepo.UpdateContentID(ctx, file.ID, ""); clearErr != nil {
slog.Warn("metadata: failed to clear file link after skeleton membership failure",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", clearErr,
)
}
if cleanupErr := s.deleteCreatedSkeleton(ctx, contentID); cleanupErr != nil {
slog.Warn("metadata: failed to clean up membershipless skeleton",
"content_id", contentID,
"file_id", file.ID,
"folder_id", folderID,
"error", cleanupErr,
)
}
return nil, fmt.Errorf("upserting skeleton membership: %w", err)
}
res.ContentID = contentID
@@ -4065,6 +4108,47 @@ func (s *MetadataService) createOrFindSkeleton(ctx context.Context, file *models
return res, nil
}
func (s *MetadataService) folderTypeForSkeleton(ctx context.Context, folderID int) (string, error) {
if s == nil || s.folderRepo == nil || folderID <= 0 {
return "", nil
}
folder, err := s.folderRepo.GetByID(ctx, folderID)
if err != nil {
return "", fmt.Errorf("loading folder context for skeleton: %w", err)
}
if folder == nil {
return "", fmt.Errorf("folder %d is unavailable for skeleton creation", folderID)
}
if !folder.Enabled {
return "", fmt.Errorf("folder %d is disabled for skeleton creation", folderID)
}
return folder.Type, nil
}
func (s *MetadataService) deleteCreatedSkeleton(ctx context.Context, contentID string) error {
if s == nil || strings.TrimSpace(contentID) == "" {
return nil
}
if repo, ok := s.itemRepo.(metadataItemDeleteRepo); ok {
if _, err := repo.Delete(ctx, contentID); err != nil && !errors.Is(err, catalog.ErrItemNotFound) {
return err
}
return nil
}
if s.dbPool == nil {
return nil
}
_, err := s.dbPool.Exec(ctx, `
DELETE FROM media_items
WHERE content_id = $1
AND NOT EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = media_items.content_id
)`, contentID)
return err
}
// claimRoot records a canonical root path claim for dedup. Failures are logged
// but not fatal — the root claim is an optimization, not a hard requirement.
func (s *MetadataService) claimRoot(ctx context.Context, folderID int, rootPath, contentID string) {
+126 -1
View File
@@ -11,6 +11,7 @@ import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
@@ -81,6 +82,16 @@ func (r *fakeItemRepo) Upsert(_ context.Context, item *models.MediaItem) error {
return nil
}
func (r *fakeItemRepo) Delete(_ context.Context, contentID string) ([]string, error) {
r.mu.Lock()
defer r.mu.Unlock()
if _, ok := r.items[contentID]; !ok {
return nil, catalog.ErrItemNotFound
}
delete(r.items, contentID)
return nil, nil
}
func (r *fakeItemRepo) IncrementRefreshFailure(_ context.Context, contentID string) error {
r.mu.Lock()
defer r.mu.Unlock()
@@ -259,6 +270,8 @@ type fakeFileRepo struct {
groupContent map[string]string // "folderID:version:key" -> contentID
groupFiles map[string][]*models.MediaFile
matchStamps map[int]time.Time // fileID -> match_attempted_at
updateErrors map[int]error
afterUpdate func(fileID int, contentID string)
claimUnmatchedCalls int
claimNonSeriesCalls int
claimMixedCalls int
@@ -273,13 +286,22 @@ func newFakeFileRepo() *fakeFileRepo {
groupContent: make(map[string]string),
groupFiles: make(map[string][]*models.MediaFile),
matchStamps: make(map[int]time.Time),
updateErrors: make(map[int]error),
}
}
func (r *fakeFileRepo) UpdateContentID(_ context.Context, fileID int, contentID string) error {
r.mu.Lock()
defer r.mu.Unlock()
if err := r.updateErrors[fileID]; err != nil {
r.mu.Unlock()
return err
}
r.contentIDs[fileID] = contentID
afterUpdate := r.afterUpdate
r.mu.Unlock()
if afterUpdate != nil {
afterUpdate(fileID, contentID)
}
return nil
}
@@ -1168,6 +1190,109 @@ func TestCreateOrFindSkeleton_NoFolderIDs(t *testing.T) {
}
}
func TestCreateOrFindSkeleton_DisabledFolderDoesNotCreateItem(t *testing.T) {
h := newTestHarness()
h.service.folderRepo = &fakeMetadataFolderRepo{
folders: map[int]*models.MediaFolder{
10: {ID: 10, Type: "movies", Enabled: false},
},
}
ctx := context.Background()
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected disabled folder error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !strings.Contains(err.Error(), "disabled") {
t.Fatalf("expected disabled error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected no items to be created, got %d", len(h.itemRepo.items))
}
if cid := h.fileRepo.contentIDs[file.ID]; cid != "" {
t.Fatalf("expected file to remain unlinked, got content_id %q", cid)
}
}
func TestCreateOrFindSkeleton_LinkFailureDeletesSkeleton(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
linkErr := errors.New("media file not found")
h.fileRepo.updateErrors[1] = linkErr
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected link error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !errors.Is(err, linkErr) {
t.Fatalf("expected wrapped link error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected skeleton item to be deleted, got %d items", len(h.itemRepo.items))
}
if len(h.libraryRepo.memberships) != 0 {
t.Fatalf("expected no library memberships, got %d", len(h.libraryRepo.memberships))
}
}
func TestCreateOrFindSkeleton_DisabledBeforeMembershipDeletesSkeleton(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
folder := &models.MediaFolder{ID: 10, Type: "movies", Enabled: true}
h.service.folderRepo = &fakeMetadataFolderRepo{
folders: map[int]*models.MediaFolder{10: folder},
}
h.fileRepo.afterUpdate = func(_ int, contentID string) {
if contentID != "" {
folder.Enabled = false
}
}
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/Inception (2010)/Inception.mkv",
}
result, err := h.service.createOrFindSkeleton(ctx, file, 10)
if err == nil {
t.Fatal("expected disabled folder error")
}
if result != nil {
t.Fatalf("expected no result, got %+v", result)
}
if !strings.Contains(err.Error(), "disabled") {
t.Fatalf("expected disabled error, got %v", err)
}
if len(h.itemRepo.items) != 0 {
t.Fatalf("expected skeleton item to be deleted, got %d items", len(h.itemRepo.items))
}
if len(h.libraryRepo.memberships) != 0 {
t.Fatalf("expected no library memberships, got %d", len(h.libraryRepo.memberships))
}
if cid := h.fileRepo.contentIDs[file.ID]; cid != "" {
t.Fatalf("expected file link to be cleared, got content_id %q", cid)
}
}
// TestCreateOrFindSkeleton_PendingItemGetsLibraryMembership verifies that
// library membership is created immediately for pending items, not just for
// matched items.
+16 -3
View File
@@ -11,6 +11,7 @@ import (
"github.com/Silo-Server/silo-server/internal/cache"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/secret"
)
// BootstrapOverrides holds values from env/CLI that must survive config
@@ -29,16 +30,21 @@ type Watcher struct {
mu sync.RWMutex
cfg *config.Config
pool *pgxpool.Pool
cipher *secret.Cipher
eventBus cache.EventBus
bootstrap BootstrapOverrides
onChange []func(old, updated *config.Config)
reloadCh chan struct{} // buffered(1), event bus writes here
}
// NewWatcher creates a new config watcher. Call Start to begin watching.
func NewWatcher(pool *pgxpool.Pool, eventBus cache.EventBus, bootstrap BootstrapOverrides) *Watcher {
// NewWatcher creates a new config watcher. Call Start to begin watching. The
// cipher decrypts sensitive server_settings values (read here via raw SQL)
// before they reach config.LoadFromDB, so a hot reload never feeds ciphertext
// into the live config (which would, e.g., break JWT validation).
func NewWatcher(pool *pgxpool.Pool, cipher *secret.Cipher, eventBus cache.EventBus, bootstrap BootstrapOverrides) *Watcher {
return &Watcher{
pool: pool,
cipher: cipher,
eventBus: eventBus,
bootstrap: bootstrap,
reloadCh: make(chan struct{}, 1),
@@ -114,7 +120,14 @@ func (w *Watcher) reload(ctx context.Context) error {
if err := rows.Scan(&k, &v); err != nil {
return fmt.Errorf("scan server_settings row: %w", err)
}
m[k] = v
// Decrypt sensitive keys (read-path contract: legacy plaintext passes
// through, enc:v1: values decrypt, corrupt ciphertext errors) so
// LoadFromDB always sees plaintext.
decrypted, derr := w.cipher.DecryptIfEncrypted(v, secret.SettingsAAD(k))
if derr != nil {
return fmt.Errorf("decrypt server_settings %q: %w", k, derr)
}
m[k] = decrypted
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate server_settings: %w", err)

Some files were not shown because too many files have changed in this diff Show More