Compare commits

..
Author SHA1 Message Date
RXWatcher 7fb9bef7ed fix(audiobooks): align player seek bar props 2026-06-07 09:52:16 +02:00
RXWatcher 843bfdf3eb chore(audiobooks): adopt goose layout for ABS migrations 2026-06-07 09:51:05 +02:00
RXWatcher dd04f5b354 fix(audiobooks): clean up ABS branch review fixes 2026-06-07 09:50:15 +02:00
RXWatcher 86a77beae2 fix(audiobooks): stabilize ABS playback progress reporting 2026-06-07 09:50:15 +02:00
RXWatcher 0327336ed9 feat(audiobooks): add ABS compatibility and polish 2026-06-07 09:50:15 +02:00
RXWatcher 5f317da348 fix(audiobooks): address native API review comments 2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 3460678b6b feat(audiobooks): wire navigation and routes
Adds an Audiobooks entry to the sidebar and registers the two new
routes (/audiobooks for the library grid, /audiobooks/book/:id for
detail). The player renders inline inside the detail page; no
dedicated player route is required for MVP.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 bde7fc80b3 feat(audiobooks): HTML5 audio player with chapter navigation
Single-file audiobook playback for MVP. Multi-file queuing arrives in
a follow-up. Streams via the existing /api/v1/direct-download GET
endpoint. Position is reported to /api/v1/audiobooks/{id}/progress
every 10s while playing plus on pause/seek/end. Skip-30s, playback
rate select, chapter list panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 ce6e85ddb9 feat(audiobooks): detail page with chapter list
Renders cover, title, author, narrator, year, and overview alongside a
chapter list. Clicking a chapter opens an inline sticky
AudiobookPlayer at that chapter's start. A "Resume" button restarts
playback at the saved progress position if present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 165b0205ae feat(audiobooks): library grid page at /audiobooks
Renders a paginated grid of audiobook cards using the
useAudiobookLibrary hook. Each card links to /audiobooks/book/{id}.
Cards show poster, title, and year; falls back to a "No cover"
placeholder when the audiobook has no poster_url. Empty state hints
to operators that they need to set a library's type to 'audiobooks'.

Routes themselves are wired in Task 8 (navigation integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 bbc421a809 feat(audiobooks): frontend types and TanStack Query hooks
TypeScript types match the JSON shapes from the new
/api/v1/audiobooks endpoints (list, detail, progress). Three hooks:
useAudiobookLibrary (list), useAudiobook (detail), and
useReportAudiobookProgress (mutation that invalidates the detail
query on success so progress updates reflect immediately).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 399807a6fe feat(audiobooks): progress endpoint at POST /api/v1/audiobooks/{id}/progress
UPSERTs user_watch_progress for the caller's (user_id, profile_id,
content_id). Body carries position_seconds; clients are expected to
post every 5-10s during playback plus on pause/seek (matching silo's
existing video progress cadence).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 b16e014dc6 feat(audiobooks): detail endpoint at GET /api/v1/audiobooks/{id}
Returns the media_items row, its media_files (with chapters JSONB),
author/narrator extracted from item_people (kinds 7/8), and the
caller's per-profile listening progress from user_watch_progress.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 5e43be2278 feat(audiobooks): list endpoint at GET /api/v1/audiobooks
Paginated list of media_items with type='audiobook' scoped to the
caller's accessible libraries via the existing access filter.
Mirrors silo's existing list-style handlers for movies and series.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcherandClaude Opus 4.7 edc01ef733 docs(audiobooks): implementation plan sub-plan 3 (API + frontend MVP)
Third of six sub-plans. 9 tasks: three REST endpoints (list/detail/
progress), TanStack Query hooks + types, three React pages
(Library/Detail/Player), and navigation integration. Scoped to MVP —
author/series indices, smart collections, share links, and other
nice-to-haves from the spec are deferred. Streaming reuses silo's
existing /api/v1/stream/{session_id}; no new transcode code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:49:19 +02:00
RXWatcher 69c5e6d1a1 chore(audiobooks): adopt goose migration layout 2026-06-07 09:48:35 +02:00
RXWatcher 0cd37aa618 fix(audiobooks): propagate scanner cancellation 2026-06-07 09:44:34 +02:00
RXWatcher 4d894379df fix(audiobooks): tighten scanner identity handling 2026-06-07 09:44:34 +02:00
RXWatcher 44fc0090c7 fix(audiobooks): address foundation review comments 2026-06-07 09:44:34 +02:00
RXWatcherandClaude Opus 4.7 38c7fb44e1 chore(migrations): renumber 142_audiobooks_feature_flag to 160 for origin/main merge
Companion to 3c6f062's 141 renumber — origin/main also added
142_episode_catalog_entries (alongside 141_episode_title_sort_index)
at a version this branch had used for the audiobooks feature flag.
Renumber ours to 160 so 142 is open for the upstream migration;
schema_versions on silo-prod is updated in lockstep so the migrator
sees audiobooks_feature_flag as already applied at its new version.

This was the only remaining collision (verified by checking for
duplicate version prefixes across migrations/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:44:21 +02:00
RXWatcherandClaude Opus 4.7 c0222a55e1 chore(migrations): renumber 141_media_folders_kind_noop to 159 for origin/main merge
Same shape as eb8f67d (the 140→157 renumber from the previous main
merge). origin/main added 141_episode_title_sort_index at the same
version this branch had used for media_folders_kind_noop. Renumber
ours to 159 (next free above the audiobook_series truncate at 158) so
141 is open for the upstream migration. schema_versions on the
running database is updated in lockstep so the migrator sees
media_folders_kind_noop as already applied at its new version.

Pending migrations on silo-prod after this rename: 141
(episode_title_sort_index, main) and any other newer ones from main
that the branch hasn't picked up yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:44:21 +02:00
RXWatcherandClaude Opus 4.7 ea6636f9d9 chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge
origin/main added 140_user_permissions at the same version this branch
had used for podcast_feeds. Renumber ours to 157 (next free above the
collections-unify migration at 156) so 140 is free for the upstream
migration. schema_versions on the running database is updated in lockstep
so the migrator sees podcast_feeds as already applied at its new version.

Same pattern as d59c1cb (renumber 139_abs_sessions to 147 for the prior
main merge). Pending migrations after this rename: 132 (downloaded
subtitles admin index, main), 140 (user_permissions, main), and 156
(unify_user_collections, this branch).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:44:21 +02:00
RXWatcherandClaude Opus 4.7 8c172ec9dd chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge
origin/main adds 139_media_requests at the same number our local
audiobook branch had used for abs_sessions. Renumber ours to 147 to
free up 139 for the upstream migration. The schema_versions row is
updated in lockstep on the running database so the migrator sees the
abs_sessions migration as already applied at its new version.

Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook
feature flag, abs playback sessions, podcast episode guid, audiobook
series, audiobook title cleanup) stay where they are — they don't
collide with anything on origin/main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:44:21 +02:00
RXWatcherandClaude Opus 4.7 514b72cddb feat(audiobooks): expose audiobooks/podcasts library types in admin UI
Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown
in the admin libraries page so operators can flag a folder as an
audiobook or podcast library. Extends contentLevelsForType() so the
admin UI's downstream filtering treats those types correctly
(audiobook -> ['audiobook'], podcasts -> ['podcast',
'podcast_episode']).

Backend scanner branches for these types were already wired in
sub-plan 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:44:21 +02:00
RXWatcherandClaude Opus 4.7 6983925b2e docs(audiobooks): implementation plan sub-plan 5 (podcasts)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 d897124e94 feat(audiobooks): filesystem podcast scanner
ScanPodcastFolder walks a podcasts-typed media folder, treating each
subdirectory as a podcast show and each audio file inside as an
episode. Writes media_items.type='podcast' + episodes rows + media_files
rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5;
this task covers filesystem-only ingestion.

ScanFolder dispatches to this path when folder.Type='podcasts'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 543534a058 feat(audiobooks): scanner write path produces audiobook media_items
ScanAudiobookFolder walks an audiobooks-typed media folder and treats
each immediate subdirectory as one audiobook. For each parsed audiobook
it upserts:
  - one media_items row with type='audiobook'
  - one media_files row per audio file (with chapters JSONB)
  - author/narrator links in item_people (kind=7, kind=8)

Adds itemRepo and personRepo to the Scanner struct, wired from
fileRepo.Pool() in NewScanner — no constructor signature change needed.

ScanFolder dispatches to this path when folder.Type='audiobooks',
bypassing the per-file movie/TV pipeline because audiobooks are
folder-scoped entities.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 234491f1cc feat(audiobooks): multi-file audiobook folder support
Folders containing N audio files (one per chapter/part) get one
parsedAudiobookFile per file; each file's chapter list is synthesized
as a single chapter with title = filename stem. Title/author/series/
year come from the first file's tags.

Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in
favor of the existing firstNonEmpty already in probe.go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 cb0cb04534 feat(audiobooks): parser for single-file audiobook folders
parseAudiobookFolder reads tags + chapters via the existing ProbeFile
(now that Task 5 exposes FormatTags on ProbeData) and produces a
parsedAudiobook struct. Title falls back from "title" tag to "album";
author from "artist" -> "album_artist" -> "composer"; series from
"album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools).
Year parsed from "date" or "year" tags, tolerating ISO dates and
parenthesized forms.

Single-file case only; multi-file folders (one audio file per chapter)
return a placeholder error and arrive in Task 7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 d4dc695e1d feat(audiobooks): expose ffprobe format tags on ProbeData
The audiobook scanner needs format-level tags (title, artist, album,
date) for media_items metadata; ffprobe already parses them in
ffprobeFormat.Tags but ProbeData previously discarded them. Add
FormatTags map[string]string to ProbeData, populate it in
convertProbeData via a new normalizeFormatTags helper that lowercases
keys and trims values.

Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and
format tags, and a test that verifies ProbeFile() returns both
correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 f4b9d4cf38 refactor(scanner): replace movieLibrary bool with typed walkMode
Lets walkLogicalTree dispatch on multiple library shapes (video, movie,
audiobook, podcast) without proliferating boolean flags. Behavior for
existing video and movie libraries is unchanged; audiobook and podcast
modes will be consumed by the upcoming audiobook.go and podcast.go
parsers in later tasks of this sub-plan.

walkModeFor() derives the mode from a media_folders.type string;
unknown types default to walkModeVideo to preserve prior behavior for
any caller still passing a raw type.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 fd41f56381 feat(audiobooks): library-type recognizers for scanner dispatch
isAudiobookLibraryType and isPodcastLibraryType match singular and
plural forms case-insensitively, mirroring isMovieLibraryType. Used by
upcoming scanner walk branches (Task 4) that filter audio files into
audiobook and podcast libraries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:35 +02:00
RXWatcherandClaude Opus 4.7 aef670c3bc feat(audiobooks): add audio-extension recognizer for scanner
Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by
upcoming audiobook and podcast scanner branches to filter directory
walks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 3a43537e67 feat(audiobooks): add Author and Narrator PersonKind constants
Discovery audit confirmed item_people.kind is unconstrained smallint
with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook
people-links written by the upcoming scanner branches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 bc2b815391 docs(audiobooks): implementation plan sub-plan 2 (scanner)
Second of six sub-plans. 10 tasks: PersonKind constants for Author and
Narrator, audio-extension recognizer, library-type helpers, a
walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter
extraction via ffprobe, single-file and multi-file audiobook parsers,
scanner write path producing media_items.type='audiobook', and a
filesystem podcast parser (RSS deferred to sub-plan 5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 7af0cf241e style(audiobooks): cosmetic cleanups in scaffolded package
Two pre-emptive cleanups flagged by code review before sub-plan 2
copies the patterns:

  1. Sort the internal/audiobooks import after internal/adminjob in
     cmd/silo/main.go (alphabetical).
  2. Drop the redundant "audiobooks: " prefix from the Enabled() error
     wrap; matches how every other top-level service package
     (watchstate, scanqueue, metadata, etc.) formats errors.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 209fb50f16 feat(audiobooks): scaffold internal/audiobooks package
Empty-but-compiling Service that reads the audiobooks.enabled feature
flag from server_settings. Wired into cmd/silo so the package is
referenced from the binary; no routes mounted, no scheduled tasks
registered, no DB writes. Subsequent sub-plans hang scanner branches,
ABS handlers, Socket.io, podcast refresher, and SPA pages off this
Service.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 a5a0fad86e feat(audiobooks): migration 142 add audiobooks.enabled flag
Server-settings row that gates the absorbed audiobooks feature.
Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent
sub-plans branch on this flag and operators flip it to 'true' at
cutover.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 9c6964e1a1 chore(audiobooks): migration 141 no-op for media_folders.type
Sub-plan 1 originally reserved migration 141 to add a 'kind' column to
media_libraries discriminating audiobook/podcast libraries. Discovery
audit (sub-plan 1 Task 1) found that the actual table is media_folders
and it already has a type text NOT NULL column with no CHECK constraint
or enum, so 'audiobooks' and 'podcasts' can be added as future values
without DDL.

Landing this migration as a documented no-op preserves the version
numbering audit trail and pins the decision in git history. The
matching down migration is also a no-op.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 901285b225 style(audiobooks): uppercase PRIMARY KEY in migration 140
Aligns with the codebase convention (type keywords lowercase,
constraint keywords uppercase) established in migration 139's
post-style-fix form. Cosmetic only — running schema is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 620cd2746b feat(audiobooks): migration 140 add podcast_feeds table
Side table on media_items for RSS-subscribed podcasts. Holds feed URL,
ETag/Last-Modified for conditional fetches, last-refresh timestamp, and
the per-feed refresh interval consumed by the upcoming
podcastfeed.Refresher scheduled task.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 355da1a4fd style(audiobooks): match codebase conventions in migration 139
Lowercases type keywords in the abs_sessions CREATE TABLE body to
match neighboring migrations, fixes the client_version column
alignment, and replaces the misleading "parallel to
jellycompat_sessions" header comment with a more accurate
description of the table's role.

Cosmetic only — the running schema is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 8deb18273d feat(audiobooks): migration 139 add abs_sessions table
Parallel of jellycompat_sessions for Audiobookshelf-compatible clients.
Lets ABS mobile/desktop apps maintain a device-bound session that
silo's audiobooks/abs handlers will validate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 a8227723bf docs(audiobooks): discovery findings for absorption sub-plan 1
Locks schema/code decisions for migrations 139-142 and downstream
sub-plans. Resolves open Risk questions from the absorption design spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 28ef3ac3c8 docs(audiobooks): implementation plan sub-plan 1 (discovery + schema)
First of six sub-plans for the absorption. Six tasks: a discovery
audit that resolves the spec's Risk questions, four idempotent SQL
migrations (abs_sessions, podcast_feeds, media_libraries.kind,
audiobooks.enabled feature flag), and an empty-but-compiling
internal/audiobooks package scaffolded into cmd/silo. Lands as a
strict no-op for users (feature flag defaults to false).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
RXWatcherandClaude Opus 4.7 6d77e5a135 docs(audiobooks): design spec for plugin absorption
Plan to absorb silo-plugin-audiobooks into silo-server as a first-party
feature. Audiobooks land in silo's existing SPA; ABS clients connect
directly. Hard constraints: reuse existing tables (media_items,
media_files, user_watch_progress, user_playback_sessions, people,
item_people, library_collections); only two new tables (abs_sessions,
podcast_feeds) and at most one column add (media_libraries.kind);
silo's main :8080 listener handles ABS Socket.io natively. Out of
scope: audiobook requests flow, smart collections, share links,
external recommender, custom metadata providers, separate audiobook
SPA.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 09:43:34 +02:00
353 changed files with 8553 additions and 22655 deletions
-19
View File
@@ -1,12 +1,6 @@
# Silo Docker deployment
# For the default docker compose setup, MEDIA_ROOT is the main value you need to set.
# Optional: layer compose files through env. Leave unset for CPU-only defaults.
# Linux/macOS example for enabling NVIDIA override:
# COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
# Windows example:
# COMPOSE_FILE=docker-compose.yml;docker-compose.nvidia.yml
# Required for docker compose: host path to your media library.
# Example: MEDIA_ROOT=/srv/media
MEDIA_ROOT=/srv/media
@@ -46,19 +40,6 @@ POSTGRES_SHM_SIZE=8gb
# PROXY_PORT=8083
# TRANSCODE_PORT=8082
# Optional NVIDIA override controls (used by docker-compose.nvidia.yml).
# NVIDIA_GPU_COUNT=1
# REQUIRED: master key for at-rest credential encryption. Silo refuses to start
# without it. Integration API keys, S3 keys, and other server-owned secrets are
# encrypted under a key derived from this value, so it must be at least 32
# characters and kept secret. Generate one with:
# openssl rand -base64 48
# Back it up SEPARATELY from your database dumps (treat it like a CA private
# key): losing it makes every encrypted secret unrecoverable. See
# docs/architecture/secret-encryption.md.
# SECRET_KEY=replace-with-output-of-openssl-rand-base64-48
# Run from source / advanced overrides
# Only DATABASE_URL is required when running Silo outside the default docker compose stack.
# DATABASE_URL=postgres://silo:password@localhost:5432/silo
-23
View File
@@ -34,29 +34,6 @@ The easiest way to run Silo is with Docker Compose. The default stack assumes yo
If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing `DATABASE_URL` and `REDIS_URL` instead.
### Optional NVIDIA/NVENC
GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.
Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.
Use the optional override file when you want NVENC:
```sh
docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
```
If you want this controlled from `.env`, set `COMPOSE_FILE`:
```dotenv
COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
NVIDIA_GPU_COUNT=1
```
Windows uses `;` instead of `:` between compose files.
Then `docker compose up -d` will include the NVIDIA override automatically.
4. **Configure through the admin UI**
Add libraries, users, metadata providers, and playback settings from the web interface.
+66 -104
View File
@@ -4,6 +4,7 @@ import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"flag"
"fmt"
"io/fs"
@@ -17,7 +18,6 @@ import (
"sort"
"strconv"
"strings"
"sync/atomic"
"syscall"
"time"
@@ -72,10 +72,11 @@ import (
"github.com/Silo-Server/silo-server/internal/ratelimit"
"github.com/Silo-Server/silo-server/internal/recommendations"
mediarequests "github.com/Silo-Server/silo-server/internal/requests"
"github.com/Silo-Server/silo-server/internal/requests/radarr"
"github.com/Silo-Server/silo-server/internal/requests/sonarr"
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/server"
"github.com/Silo-Server/silo-server/internal/subtitles"
@@ -140,7 +141,7 @@ func mustGetSetting(store interface {
func configureOperationalLogging(
ctx context.Context,
pool *pgxpool.Pool,
settingsRepo catalog.SettingsStore,
settingsRepo *catalog.ServerSettingsRepo,
redisCfg config.RedisConfig,
logStreamHub *logstream.Hub,
baseHandler slog.Handler,
@@ -251,38 +252,6 @@ func maybeApplyPostgresTuning(ctx context.Context, pool *pgxpool.Pool, appMaxCon
}
}
// runCredentialBackfills sweeps any plaintext server-owned credential to
// ciphertext on the primary (migration-running) node. All passes are
// best-effort: a failed row leaves the prior plaintext (no new exposure) and
// still reads via the read-path pass-through, so a backfill error must never
// block boot. The sensitive-settings pass runs first so the arr
// resolve-then-encrypt pass sees consistent referenced settings.
func runCredentialBackfills(ctx context.Context, pool *pgxpool.Pool, cipher *secret.Cipher, settings *catalog.EncryptedSettingsRepo) {
settingsN, err := settings.BackfillSensitiveSettings(ctx)
if err != nil {
slog.Error("secret backfill: sensitive settings", "error", err)
}
columnsN, err := secret.BackfillColumns(ctx, pool, cipher, secret.ColumnBackfillTargets())
if err != nil {
slog.Error("secret backfill: credential columns", "error", err)
}
historyServersN, err := historyimport.NewRepository(pool, cipher).BackfillSessionServerSecrets(ctx)
if err != nil {
slog.Error("secret backfill: history import session server credentials", "error", err)
}
// The arr resolver is the encrypting settings decorator: it decrypts a
// sensitive target (e.g. requests.radarr.api_key) or passes through a
// plaintext custom key, exactly replicating the deleted resolveAPIKey.
arrN, err := secret.BackfillReferencedColumns(ctx, pool, cipher, settings.Get, secret.ArrKeyBackfillTargets())
if err != nil {
slog.Error("secret backfill: arr api keys", "error", err)
}
if total := settingsN + columnsN + historyServersN + arrN; total > 0 {
slog.Info("secret backfill: encrypted plaintext credentials at rest",
"settings", settingsN, "columns", columnsN, "history_session_servers", historyServersN, "arr_keys", arrN, "total", total)
}
}
func main() {
envFile := flag.String("env", ".env", "path to .env bootstrap file")
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
@@ -297,15 +266,6 @@ func main() {
log.Fatalf("bootstrap: %v", err)
}
// Construct the at-rest credential cipher from SECRET_KEY immediately after
// bootstrap, before any settings repo is built. It is threaded explicitly as
// a dependency into every repo that stores a server-owned secret — never a
// package-level global.
dataCipher, err := secret.New(bc.SecretKey)
if err != nil {
log.Fatalf("secret cipher: %v", err)
}
// Step 2: Connect to PostgreSQL (bootstrap pool with default max connections)
bootstrapDBCfg := config.DatabaseConfig{URL: bc.DatabaseURL, MaxConnections: 20}
pool, err := database.NewPool(ctx, bootstrapDBCfg)
@@ -354,11 +314,7 @@ func main() {
// Run migrations only for integrated/api modes. Proxy and transcode nodes
// should never alter the schema — they may scale independently and would
// race or apply migrations before the primary node is deliberately upgraded.
// The same gate decides whether this node runs the credential-encryption
// backfills: only the primary (migration-running) node sweeps plaintext to
// ciphertext; secondary nodes read whatever the primary encrypted.
isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == ""
if isPrimaryNode {
if bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == "" {
migCtx, migCancel := context.WithTimeout(ctx, 5*time.Minute)
if migErr := database.RunMigrations(migCtx, pool, migrations.FS, "sql"); migErr != nil {
migCancel()
@@ -368,14 +324,8 @@ func main() {
slog.Info("database migrations applied")
}
// Step 3: Load settings from DB. settingsRepo is the encrypting decorator so
// every consumer (config.LoadFromDB, admin, ABS, watchers) transparently sees
// plaintext while sensitive keys rest as ciphertext. The settings backfill
// (run after migrations, before this GetAll) is wired further below.
settingsRepo := catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
if isPrimaryNode {
runCredentialBackfills(ctx, pool, dataCipher, settingsRepo)
}
// Step 3: Load settings from DB
settingsRepo := catalog.NewServerSettingsRepo(pool)
settings, err := settingsRepo.GetAll(ctx)
if err != nil {
log.Fatalf("loading settings: %v", err)
@@ -446,9 +396,7 @@ func main() {
log.Fatalf("recreating pool with configured max_connections: %v", err)
}
}
// Re-wrap with the encrypting decorator so the recreated pool's settings repo
// still encrypts/decrypts — no raw settings repo may escape into later wiring.
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
settingsRepo = catalog.NewServerSettingsRepo(pool)
nodeID := resolveNodeIdentity()
// Step 9: Validate
@@ -477,8 +425,6 @@ func main() {
appCtx, appCancel := context.WithCancel(ctx)
defer appCancel()
restartReqCh := make(chan struct{}, 1)
var restartRequested atomic.Bool
eventBus := cache.NewEventBus(cfg.Redis.URL)
logStreamHub := logstream.NewHub(nodeID, eventBus)
@@ -512,7 +458,7 @@ func main() {
DatabaseURL: cfg.Database.URL,
JFListen: cfg.JellyfinCompat.Listen,
}
watcher := nodeconfig.NewWatcher(pool, dataCipher, eventBus, bootstrap)
watcher := nodeconfig.NewWatcher(pool, eventBus, bootstrap)
if err := watcher.Start(appCtx); err != nil {
slog.Error("config watcher start failed", "error", err)
os.Exit(1)
@@ -571,7 +517,6 @@ func main() {
BootstrapSensitiveValues: bootstrapSensitiveValues,
AppContext: appCtx,
DB: pool,
SecretCipher: dataCipher,
EventBus: eventBus,
LogStreamHub: logStreamHub,
RealtimeHub: realtimeHub,
@@ -580,20 +525,14 @@ func main() {
OpsLogRepo: opsRepo,
FFmpegLogSink: playback.NewSlogFFmpegLogSink(slog.Default(), nodeID),
PublicURL: os.Getenv("SILO_PUBLIC_URL"),
RequestServerRestart: func(context.Context) error {
if !restartRequested.CompareAndSwap(false, true) {
return handlers.ErrServerRestartAlreadyRequested
}
restartReqCh <- struct{}{}
return nil
},
}
audiobooksService := audiobooks.New(&audiobooksSettingsAdapter{repo: settingsRepo})
absCompatEnabled, err := audiobooksService.ABSCompatEnabled(appCtx)
audiobooksEnabled, err := audiobooksService.Enabled(appCtx)
if err != nil {
slog.Warn("Audiobookshelf compatibility disabled; failed to read setting", "err", err)
absCompatEnabled = false
slog.Warn("audiobooks feature disabled; failed to read setting", "err", err)
audiobooksEnabled = false
}
deps.AudiobooksEnabled = audiobooksEnabled
adminJobCancelRegistry := adminjob.NewCancelRegistry()
deps.AdminJobCancelRegistry = adminJobCancelRegistry
if needsWorkers && deps.DB != nil {
@@ -648,7 +587,7 @@ func main() {
log.Fatalf("register watch provider: %v", err)
}
watchProviderService = watchsync.NewService(
watchsync.NewPostgresRepository(deps.DB, deps.SecretCipher),
watchsync.NewPostgresRepository(deps.DB),
watchProviderRegistry,
)
deps.WatchProviderService = watchProviderService
@@ -1233,7 +1172,7 @@ func main() {
deps.UserStoreProvider = userStoreProvider
}
if watchProviderService != nil {
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyRepo := historyimport.NewRepository(deps.DB)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, catalog.NewProviderIDRepository(deps.DB))
watchProviderService.
WithMatcher(historyimport.NewMatcher(historyRepo)).
@@ -1496,7 +1435,6 @@ func main() {
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil {
taskMgr.Register(tasks.NewScanLibrariesTask(deps.FolderRepo, deps.LibraryScanQueue, deps.EventBus))
}
taskMgr.Register(tasks.NewCleanupOrphanedMediaItemsTask(catalog.NewOrphanedProvisionalCleaner(deps.DB)))
if deps.IntroAnalyzer != nil {
taskMgr.Register(tasks.NewDetectIntroMarkersTask(deps.IntroAnalyzer, settingsRepo))
}
@@ -1532,15 +1470,15 @@ func main() {
taskMgr.Register(tasks.NewSyncWatchProvidersTask(watchProviderService))
}
requestReconcileSvc := mediarequests.NewService(
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
mediarequests.NewRepository(deps.DB),
nil,
mediarequests.NewCatalogPresence(
catalog.NewItemRepository(deps.DB),
catalog.NewProviderIDRepository(deps.DB),
),
)
requestReconcileSvc.SetRequesterIdentityResolver(plugins.RequesterIdentityFromLookup(plugins.NewPgUserIdentityLookup(deps.DB)))
api.AttachRequestRouter(requestReconcileSvc, pluginService)
requestReconcileSvc.SetSecretResolver(settingsRepo)
requestReconcileSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if userStoreProvider != nil {
reconcileResolver := access.NewResolver(
auth.NewUserRepository(deps.DB),
@@ -1551,15 +1489,13 @@ func main() {
}
taskMgr.Register(tasks.NewReconcileRequestsTask(requestReconcileSvc, 100))
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && pluginService != nil && pluginInstallationStore != nil {
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if err := autoscanRepo.MarkInterruptedEvents(appCtx); err != nil {
slog.Warn("autoscan: failed to mark interrupted polls", "err", err)
}
autoscanRepo := autoscan.NewRepository(deps.DB)
autoscanSvc := api.BuildAutoscanService(
autoscanRepo,
pluginService,
pluginInstallationStore,
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
mediarequests.NewRepository(deps.DB),
settingsRepo,
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -1583,7 +1519,7 @@ func main() {
taskMgr.Register(tasks.NewRepairProviderIDIntegrityTask(metadata.NewProviderIDIntegrityRepairer(deps.DB), historyReconciler))
taskMgr.Register(tasks.NewReconcileWatchHistoryTask(historyReconciler))
taskMgr.Register(tasks.NewSyncPodcastFeedsTask(podcastfeed.New(), podcastfeed.NewDBStore(deps.DB)))
if audiobookEnricher != nil {
if audiobooksEnabled && audiobookEnricher != nil {
taskMgr.Register(tasks.NewSyncAudiobookMetadataTask(audiobookEnricher))
}
if pluginInstallationStore != nil && pluginRuntimeConfigStore != nil && pluginService != nil {
@@ -1606,7 +1542,7 @@ func main() {
// available. Routes are mounted at the root level by NewRouter (not under
// /api/v1/) so ABS clients resolve /login, /api/*, /abs/api/*, and
// /abs/socket.io/* without path prefix hacks.
if absCompatEnabled && deps.DB != nil {
if audiobooksEnabled && deps.DB != nil {
absUserRepo := auth.NewUserRepository(deps.DB)
absSessionRepo := auth.NewSessionRepository(deps.DB)
absJWTService := auth.NewJWTService(
@@ -1879,7 +1815,6 @@ func main() {
compatDeps := jellycompat.Dependencies{
Config: cfg,
DB: deps.DB,
SecretCipher: dataCipher,
ClientIPResolver: ipResolver,
ProxyPool: deps.ProxyPool,
TranscodePool: deps.TranscodePool,
@@ -1933,7 +1868,7 @@ func main() {
compatDeps.FileResolver = deps.FileRepo
}
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB)
// Construct auth service for jellycompat login.
userRepo := auth.NewUserRepository(deps.DB)
@@ -1949,17 +1884,47 @@ func main() {
provider := auth.NewLocalProvider(userRepo, sessionRepo)
compatDeps.AuthService = auth.NewService(provider, jwtService, sessionRepo, userRepo, nil, nil, nil)
// Access filter resolver for viewer-scoped library access.
// Backed by the shared access.Resolver so account-level library
// restrictions (users.library_ids), profile restrictions,
// user-disabled libraries, and rating/quality ceilings apply to
// the compat API exactly as they do to the native API.
if userStoreProvider != nil {
compatDeps.AccessFilterFn = jellycompat.NewScopeAccessFilter(access.NewResolver(
userRepo,
userStoreProvider,
nil, // profile tokens unused: compat login already verifies PINs
))
// Access filter resolver for profile-scoped library access.
compatDeps.AccessFilterFn = func(ctx context.Context, userID int, profileID string) catalog.AccessFilter {
if userStoreProvider == nil {
return catalog.AccessFilter{}
}
store, err := userStoreProvider.ForUser(ctx, userID)
if err != nil {
return catalog.AccessFilter{}
}
profile, err := store.GetProfile(ctx, profileID)
if err != nil || profile == nil {
return catalog.AccessFilter{}
}
filter := catalog.AccessFilter{
MaxContentRating: profile.MaxContentRating,
}
if profile.LibraryRestrictionsEnabled && len(profile.AllowedLibraryIDs) > 0 {
filter.AllowedLibraryIDs = profile.AllowedLibraryIDs
}
// Apply user-disabled library IDs.
if raw, err := store.GetSetting(ctx, "disabled_library_ids"); err == nil && raw != "" {
var disabled []int
if json.Unmarshal([]byte(raw), &disabled) == nil && len(disabled) > 0 {
if filter.AllowedLibraryIDs != nil {
filtered := make([]int, 0, len(filter.AllowedLibraryIDs))
disSet := make(map[int]struct{}, len(disabled))
for _, id := range disabled {
disSet[id] = struct{}{}
}
for _, id := range filter.AllowedLibraryIDs {
if _, ok := disSet[id]; !ok {
filtered = append(filtered, id)
}
}
filter.AllowedLibraryIDs = filtered
} else {
filter.DisabledLibraryIDs = disabled
}
}
}
return filter
}
}
@@ -2051,9 +2016,6 @@ func main() {
case sig := <-sigCh:
appCancel()
slog.Info("received signal, shutting down", "signal", sig)
case <-restartReqCh:
appCancel()
slog.Info("server restart requested, shutting down")
case serverErr := <-errCh:
appCancel()
slog.Error("server error, shutting down", "error", serverErr)
@@ -2433,7 +2395,7 @@ func mapFolderTypeToMediaType(t string) string {
// exposes Get) to the audiobooks.SettingsReader interface (which
// requires GetString). The two signatures are identical modulo name.
type audiobooksSettingsAdapter struct {
repo catalog.SettingsStore
repo *catalog.ServerSettingsRepo
}
func (a *audiobooksSettingsAdapter) GetString(ctx context.Context, key string) (string, error) {
-20
View File
@@ -1,20 +0,0 @@
services:
silo:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: ${NVIDIA_GPU_COUNT:-1}
capabilities: [gpu]
# Optional distributed-mode example. Most single-host installs should
# just use the integrated service above.
# silo-transcode:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: ${NVIDIA_GPU_COUNT:-1}
# capabilities: [gpu]
+1 -10
View File
@@ -36,11 +36,6 @@ services:
restart: unless-stopped
environment:
MODE: integrated
# Master key for at-rest credential encryption. Required — the server
# refuses to start without it. Generate with: openssl rand -base64 48
# Back it up SEPARATELY from your database dumps; losing it makes encrypted
# secrets unrecoverable. See docs/architecture/secret-encryption.md.
SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
REDIS_URL: redis://redis:6379
SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
@@ -51,7 +46,7 @@ services:
- "${ABS_PORT:-13378}:13378"
volumes:
- ${MEDIA_ROOT:?Set MEDIA_ROOT in .env to the host media path}:${MEDIA_CONTAINER_ROOT:-/mnt/media}:ro
- ${MEDIA_BOOKS_ROOT:-${MEDIA_ROOT}}:${MEDIA_BOOKS_CONTAINER_ROOT:-${MEDIA_CONTAINER_ROOT:-/mnt/media}/books}:ro
- ${MEDIA_BOOKS_ROOT:-${MEDIA_ROOT}}:${MEDIA_BOOKS_CONTAINER_ROOT:-${MEDIA_CONTAINER_ROOT:-/mnt/media}}:ro
- ${SILO_DATA_ROOT:-/opt/silo}/plugins:/var/lib/silo/plugins
- ${SILO_DATA_ROOT:-/opt/silo}/transcode:/tmp/silo-transcode
- ${SILO_DATA_ROOT:-/opt/silo}/catalog-seeds:/catalog-seeds:ro
@@ -76,8 +71,6 @@ services:
# profiles: [proxy]
# environment:
# MODE: proxy
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# PORT: "8080"
@@ -102,8 +95,6 @@ services:
# profiles: [transcode]
# environment:
# MODE: transcode
# # Must be the SAME SECRET_KEY as the primary node (shared encrypted data).
# SECRET_KEY: ${SECRET_KEY:?Set SECRET_KEY in .env — generate one with openssl rand -base64 48}
# DATABASE_URL: postgres://${POSTGRES_USER:-silo}:${POSTGRES_PASSWORD:-silo}@postgres:5432/${POSTGRES_DB:-silo}?sslmode=disable
# REDIS_URL: redis://redis:6379
# SILO_PLUGIN_CACHE_DIR: /var/lib/silo/plugins
-142
View File
@@ -1,142 +0,0 @@
# At-rest credential encryption
Silo encrypts third-party integration credentials and other server-owned secrets
at rest. Sonarr/Radarr API keys, S3 keys, watch-sync OAuth tokens, the
Audiobookshelf-compat signing key, and the sensitive `server_settings` entries
are stored as ciphertext in PostgreSQL and decrypted only in memory. The key that
protects them lives **outside** the database, so a full DB dump or compromise
does not expose the secrets.
## How it works
- **Cipher:** AES-256-GCM with a random 12-byte nonce per value. Implemented in
`internal/secret`.
- **Key derivation:** the data key is HKDF-SHA256–derived from the `SECRET_KEY`
environment variable with a versioned domain label. `SECRET_KEY` is the
encryption root; it is never stored in the database.
- **Envelope:** every ciphertext is stored as `enc:v1:<base64url(nonce‖sealed)>`.
The `enc:v1:` prefix is the version marker (see *Key rotation* below).
- **Row binding (AAD):** each ciphertext is GCM-bound to its logical row via
additional-authenticated data — `table:column:<pk>` for per-row columns,
`server_settings:<key>` for settings. A DB-write attacker therefore cannot move
a credential blob from one row/column to another: decrypting under a different
binding fails authentication.
- **Read path:** an empty value stays empty; a non-`enc:v1:` value is treated as
legacy plaintext and passed through unchanged (so a not-yet-migrated row keeps
working); an `enc:v1:` value is decrypted and **any** failure (wrong key,
tampering, truncation) is surfaced as an error — it is never silently used as a
credential.
## SECRET_KEY
`SECRET_KEY` is **required**. The server refuses to start without it (it fatals
in `config.LoadBootstrap`), and it must be at least 32 characters.
Generate one with:
```bash
openssl rand -base64 48
```
Set it in the deployment environment (or in `.env` for source/dev runs, which
`godotenv` loads at startup). See `.env.example`.
### Back it up — separately from the database
Treat `SECRET_KEY` like a CA private key:
- **Store it outside your database backups.** A DB dump contains only ciphertext;
the value of at-rest encryption is entirely lost if the key travels with the
dump.
- Keep it in a secrets manager / sealed secret / password manager, not in the
repo or in the same bucket as your Postgres backups.
- Rotating the *deployment* (new host, restored DB) requires the **same**
`SECRET_KEY`. Restoring a database backup onto a node with a different
`SECRET_KEY` makes every encrypted secret unreadable.
### Key loss
If `SECRET_KEY` is lost, the encrypted secrets are **unrecoverable** — that is the
point of keeping the key out of the database. Recovery means re-entering the
affected credentials:
- Re-enter Sonarr/Radarr and Autoscan API keys, S3 keys, watch-sync connections,
history-import tokens, and subtitle provider credentials.
- `auth.jwt_secret` becomes unreadable, so all existing sessions are invalid and
users must log in again (a new secret is generated on next boot if the row is
cleared — see below).
## Rollout (automatic backfill)
On the first boot after deploying this change, the primary (migration-running)
node runs an idempotent, best-effort startup backfill that encrypts any
remaining plaintext in place:
1. sensitive `server_settings` keys,
2. the per-table credential columns (subtitles, watch-sync, webhook-sync,
history-import, including temporary server-list credentials stored in
session JSON),
3. the two arr `api_key_ref` columns — these are **resolved-then-encrypted**: a
legacy row that held a `server_settings` reference (e.g.
`requests.radarr.api_key`) is collapsed to the real credential before being
encrypted.
The backfill is safe to run repeatedly: already-encrypted values are skipped, and
a per-row guard makes concurrent multi-node boots converge without
double-encrypting. A failed row is logged and left as plaintext (no new exposure)
rather than blocking boot. Proxy/transcode nodes skip the backfill; they read
whatever the primary encrypted. No manual steps are required.
## Rollback / downgrade hazard
Downgrading to a binary that predates this change is **not** safe while secrets
are encrypted, because the old binary has no read path: it would read
`enc:v1:auth.jwt_secret` as a literal JWT secret (invalidating all sessions) and
read `enc:v1:`-prefixed integration keys as garbage credentials.
To downgrade safely you must first return the affected values to plaintext, for
example:
- Decrypt the sensitive `server_settings` back to plaintext (run a one-off that
reads each via the encrypting repo and writes the plaintext via the raw repo),
**or**
- At minimum, clear `auth.jwt_secret` so the older binary regenerates a fresh
plaintext one (this still logs everyone out), and re-enter any integration
credentials as plaintext.
There is no automatic "decrypt everything" downgrade path in this release; plan
downgrades accordingly.
## Key rotation (future)
The `enc:v1:` envelope and the HKDF domain label are versioned. A future rotation
would introduce `enc:v2:` with a new derived key (and `Decrypt` already dispatches
on the version, returning an explicit error for an unknown version). Rotation is
out of scope for this release; the versioning exists so it can be added without a
data migration of the envelope format.
## Scope and known gaps
Covered: arr (Requests + Autoscan) API keys, S3 keys, all sensitive
`server_settings`, watch-sync tokens, webhook-sync `access_token`,
history-import admin/session tokens and temporary server-list credentials,
subtitle provider `api_key`/`password`,
the Jellyfin-compat session's bridged Silo access/refresh tokens
(`jellycompat_sessions.streamapp_access_token` / `streamapp_refresh_token`), and
the ABS signing key.
Deliberately **not** encrypted (tracked as follow-ups):
- **Equality-looked-up secrets** — these are matched by exact value
(`WHERE … = $1`), and AES-GCM is randomized, so encrypting them would break the
lookup. They need a deterministic **blind-index hash** column instead:
`api_keys.api_key`, `webhook_sync_connections.webhook_secret`,
`jellycompat_sessions.token`, and `watch_together_rooms.join_token`.
- **Plugin runtime config** — `plugin_runtime_configs.config_value` is opaque
plugin-defined JSONB whose secret fields are manifest-declared and whose runtime
lives in a separate repo; encrypting it correctly needs a coordinated,
manifest-aware design.
Excluded (not a gap): `plex_sync_connections.*` is a dead table (zero Go
references); `oauth_completion.token_ciphertext` is already AES-GCM;
`users.password_hash` and the `*_hash` columns are already hashed.
+1 -1
View File
@@ -76,7 +76,7 @@ require (
)
require (
github.com/Silo-Server/silo-plugin-sdk v0.6.0
github.com/Silo-Server/silo-plugin-sdk v0.5.0
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 // indirect
+2 -2
View File
@@ -2,8 +2,8 @@ entgo.io/ent v0.14.3 h1:wokAV/kIlH9TeklJWGGS7AYJdVckr0DloWjIcO9iIIQ=
entgo.io/ent v0.14.3/go.mod h1:aDPE/OziPEu8+OWbzy4UlvWmD2/kbRuWfK2A40hcxJM=
github.com/PuerkitoBio/goquery v1.8.0 h1:PJTF7AmFCFKk1N6V6jmKfrNH9tV5pNE6lZMkG0gta/U=
github.com/PuerkitoBio/goquery v1.8.0/go.mod h1:ypIiRMtY7COPGk+I/YbZLbxsxn9g5ejnI2HSMtkjZvI=
github.com/Silo-Server/silo-plugin-sdk v0.6.0 h1:Gi9TdH9kt7b8X4xRXH493/nSYb9n0GO4VCWmlll0hKI=
github.com/Silo-Server/silo-plugin-sdk v0.6.0/go.mod h1:etqmxLTwjxpFH9goAjBDfNDoqHMv2/sqUXu8yx3hNfA=
github.com/Silo-Server/silo-plugin-sdk v0.5.0 h1:0yJpD0RP1AetGsWVCXoPih2ggLjN8hVp8blS+2RkIWU=
github.com/Silo-Server/silo-plugin-sdk v0.5.0/go.mod h1:etqmxLTwjxpFH9goAjBDfNDoqHMv2/sqUXu8yx3hNfA=
github.com/abadojack/whatlanggo v1.0.1 h1:19N6YogDnf71CTHm3Mp2qhYfkRdyvbgwWdd2EPxJRG4=
github.com/abadojack/whatlanggo v1.0.1/go.mod h1:66WiQbSbJBIlOZMsvbKe5m6pzQovxCH9B/K8tQB2uoc=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
+25 -11
View File
@@ -20,13 +20,12 @@ type autoscanQueuer = autoscan.Queuer
// RequestIntegrationLookup adapts the Requests repository to the autoscan
// connection resolver's RequestIntegrationLookup: it resolves a soft-linked
// Requests integration to its base URL and api key (the repo decrypts the key
// on read, so this returns plaintext).
// Requests integration to its base URL and (write-only) api-key ref.
type RequestIntegrationLookup struct {
Repo *mediarequests.Repository
}
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error) {
func (l RequestIntegrationLookup) Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error) {
integration, err := l.Repo.GetIntegration(ctx, integrationID)
if err != nil {
return "", "", err
@@ -65,8 +64,8 @@ type PluginScanSourceAdapter struct {
Svc *plugins.Service
}
func (a PluginScanSourceAdapter) ScanSourceClient(ctx context.Context, pluginID, capabilityID string) (autoscan.PollChangesClient, error) {
return a.Svc.ScanSourceClientByPluginID(ctx, pluginID, capabilityID)
func (a PluginScanSourceAdapter) ScanSourceClient(ctx context.Context, installationID int, capabilityID string) (autoscan.PollChangesClient, error) {
return a.Svc.ScanSourceClient(ctx, installationID, capabilityID)
}
// scanSourceCapabilityType is the plugin capability type autoscan discovery
@@ -76,6 +75,15 @@ const scanSourceCapabilityType = "scan_source.v1"
// PluginScanSourceLister adapts the plugin installation store to
// autoscan.ScanSourceLister: it enumerates every installed scan_source.v1
// capability across ALL installed plugins, regardless of enabled state.
//
// Using List (not ListEnabled) is deliberate for orphan detection: a temporarily
// DISABLED-but-installed plugin must NOT be treated as orphaned. If we dropped
// disabled plugins here, PollOnce would prune their sources and skip them with no
// last_error — they'd vanish silently. Keeping them present means PollOnce still
// attempts them; the plugin client fails to load (not running) and the source
// gets a visible RecordError instead. Only a fully UNINSTALLED plugin (gone from
// List entirely) is a true orphan. The Add-source picker shares this all-installed
// set, which is fine — operators can bind sources against an installed capability.
type PluginScanSourceLister struct {
Store *plugins.InstallationStore
}
@@ -96,9 +104,10 @@ func (l PluginScanSourceLister) ListScanSources(ctx context.Context) ([]autoscan
continue
}
out = append(out, autoscan.DiscoveredSource{
PluginID: inst.PluginID,
CapabilityID: c.ID,
DisplayName: scanSourceDisplayName(inst.PluginID, c),
InstallationID: c.InstallationID,
CapabilityID: c.ID,
PluginID: inst.PluginID,
DisplayName: scanSourceDisplayName(inst.PluginID, c),
})
}
}
@@ -126,22 +135,27 @@ func scanSourceDisplayName(pluginID string, c *plugins.Capability) string {
}
}
// AutoscanSecretResolver resolves an encrypted api-key reference to plaintext.
// It is satisfied by the server settings repo (Get(ctx, key) (string, error)).
type AutoscanSecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
}
// BuildAutoscanService wires the v2 autoscan engine from its concrete
// dependencies. Both the HTTP router (manual trigger) and the background poll
// task share this constructor so the adapter wiring lives in exactly one place.
// Credentials are now decrypted inline by the autoscan/requests repos, so there
// is no separate secret resolver to thread.
func BuildAutoscanService(
repo *autoscan.Repository,
pluginService *plugins.Service,
installationStore *plugins.InstallationStore,
requestsRepo *mediarequests.Repository,
secrets AutoscanSecretResolver,
folderRepo *catalog.FolderRepository,
queue autoscanQueuer,
redisClient *redis.Client,
) *autoscan.Service {
provider := autoscan.NewPluginProvider(PluginScanSourceAdapter{pluginService})
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo})
connRes := autoscan.NewConnectionResolver(RequestIntegrationLookup{requestsRepo}, secrets)
svc := autoscan.NewService(
repo,
provider,
+1 -33
View File
@@ -1,10 +1,6 @@
package api
import (
"testing"
mediarequests "github.com/Silo-Server/silo-server/internal/requests"
)
import "testing"
// TestCheckRequestIntegrationUsable verifies the Fix 4 gate: a reused Requests
// connection is rejected when the integration is disabled or has no base_url, so
@@ -26,31 +22,3 @@ func TestCheckRequestIntegrationUsable(t *testing.T) {
}
})
}
// TestCheckRequestIntegrationUsableGatesReuse is the Task 17 regression test. It
// locks the autoscan→requests credential-reuse behavior after the
// request_integrations table was generalized into the two-tier connection
// registry (Task 9/10): autoscan must keep reusing the Sonarr/Radarr connection
// rows via the same enabled + base_url gate, regardless of the added
// capability/installation columns.
func TestCheckRequestIntegrationUsableGatesReuse(t *testing.T) {
// Enabled + non-blank base_url -> usable (autoscan can reuse the connection).
if err := checkRequestIntegrationUsable("int-1", true, "http://sonarr:8989"); err != nil {
t.Fatalf("enabled integration with base_url should be usable, got: %v", err)
}
// Disabled -> error (autoscan must not poll a disabled connection).
if err := checkRequestIntegrationUsable("int-1", false, "http://sonarr:8989"); err == nil {
t.Fatal("disabled integration must be rejected")
}
// Blank base_url -> error.
if err := checkRequestIntegrationUsable("int-1", true, " "); err == nil {
t.Fatal("blank base_url must be rejected")
}
}
// Compile-time guard: autoscan's RequestIntegrationLookup reads these fields off
// mediarequests.Integration. If any is renamed/removed, autoscan reuse breaks —
// fail the build here rather than silently at runtime.
var _ = func(in mediarequests.Integration) (string, string, bool) {
return in.BaseURL, in.APIKeyRef, in.Enabled
}
+23 -4
View File
@@ -1137,10 +1137,29 @@ func (h *AdminHandler) HandleUpdateItemMetadata(w http.ResponseWriter, r *http.R
// --- Server Settings endpoints ---
// sensitiveSettingKeys is the audited allowlist of secret-bearing settings
// keys, shared with the at-rest encryption decorator so redaction and
// encryption can never drift apart. See catalog.SensitiveSettingKeys.
var sensitiveSettingKeys = catalog.SensitiveSettingKeys
var sensitiveSettingKeys = map[string]bool{
"auth.jwt_secret": true,
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"redis.url": true,
"recommendations.openai_api_key": true,
"recommendations.embedding_auth_token": true,
"tmdb.api_key": true,
"introdb.api_key": true,
"mdblist.api_key": true,
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
"subtitle_ai.api_key": true,
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
}
// HandleGetSettings handles GET /admin/settings.
func (h *AdminHandler) HandleGetSettings(w http.ResponseWriter, r *http.Request) {
+8 -44
View File
@@ -22,9 +22,7 @@ type AdminStats struct {
TotalFiles int `json:"total_files"`
TotalUsers int `json:"total_users"`
TotalMovies int `json:"total_movies"`
TotalMovieFiles int `json:"total_movie_files"`
TotalShows int `json:"total_shows"`
TotalShowFiles int `json:"total_show_files"`
ActiveStreams int `json:"active_streams"`
TotalStorageBytes int64 `json:"total_storage_bytes"`
WatchProviderActivity WatchProviderActivity `json:"watch_provider_activity"`
@@ -129,15 +127,13 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
}
var (
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalMovieFiles int64
totalShows int64
totalShowFiles int64
activeStreams int64
totalStorage int64
totalUsers int64
totalItems int64
totalFiles int64
totalMovies int64
totalShows int64
activeStreams int64
totalStorage int64
)
row := pool.QueryRow(ctx, `
@@ -155,34 +151,8 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
file_stats AS (
SELECT
COUNT(*)::bigint AS total_files,
COUNT(*) FILTER (WHERE file_kind = 'movie')::bigint AS total_movie_files,
COUNT(*) FILTER (WHERE file_kind = 'series')::bigint AS total_show_files,
COALESCE(SUM(file_size), 0)::bigint AS total_storage_bytes
FROM (
SELECT
media_files.file_size,
CASE
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'movie'
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_items.type, ''), ''))) = 'series'
THEN 'series'
WHEN episodes.content_id IS NOT NULL
THEN 'series'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(COALESCE(NULLIF(media_files.base_type, ''), ''))) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
WHEN lower(trim(media_folders.type)) IN ('movie', 'movies')
THEN 'movie'
WHEN lower(trim(media_folders.type)) IN ('series', 'tv', 'show', 'shows', 'tvshows')
THEN 'series'
ELSE ''
END AS file_kind
FROM media_files
JOIN media_folders ON media_folders.id = media_files.media_folder_id
LEFT JOIN media_items ON media_items.content_id = media_files.content_id
LEFT JOIN episodes ON episodes.content_id = media_files.episode_id
) classified_files
FROM media_files
),
session_stats AS (
SELECT COUNT(*)::bigint AS active_streams
@@ -193,9 +163,7 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
item_stats.total_items,
file_stats.total_files,
item_stats.total_movies,
file_stats.total_movie_files,
item_stats.total_shows,
file_stats.total_show_files,
session_stats.active_streams,
file_stats.total_storage_bytes
FROM user_stats
@@ -208,9 +176,7 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
&totalItems,
&totalFiles,
&totalMovies,
&totalMovieFiles,
&totalShows,
&totalShowFiles,
&activeStreams,
&totalStorage,
); err != nil {
@@ -228,9 +194,7 @@ func queryAdminStats(ctx context.Context, pool *pgxpool.Pool) (AdminStats, error
TotalItems: int(totalItems),
TotalFiles: int(totalFiles),
TotalMovies: int(totalMovies),
TotalMovieFiles: int(totalMovieFiles),
TotalShows: int(totalShows),
TotalShowFiles: int(totalShowFiles),
ActiveStreams: int(activeStreams),
TotalStorageBytes: totalStorage,
WatchProviderActivity: activity,
+531
View File
@@ -0,0 +1,531 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/models"
)
// HandleGetAudiobook serves GET /api/v1/audiobooks/{id}. Returns the
// audiobook media_item, its media_files (with embedded chapters),
// author/narrator from item_people (kinds 7/8), and the caller's
// per-profile listening progress.
func (h *AudiobookHandler) HandleGetAudiobook(w http.ResponseWriter, r *http.Request) {
contentID := chi.URLParam(r, "id")
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "id is required")
return
}
items, err := h.Items.GetByIDsWithAccess(r.Context(), []string{contentID}, requestAccessFilter(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load audiobook failed")
return
}
if len(items) == 0 {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
item := items[0]
if item == nil || item.Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
files, err := h.Files.GetByContentID(r.Context(), contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load files failed")
return
}
author, narrator, err := h.fetchAuthorNarrator(r, contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load people failed")
return
}
progress := h.fetchProgress(r, contentID)
alsoByAuthor := h.fetchAlsoByAuthor(r.Context(), contentID)
inSeries := h.fetchInSeries(r.Context(), contentID)
similar := h.fetchSimilar(r.Context(), contentID)
otherNarrations := h.fetchOtherNarrations(r.Context(), contentID)
resp := audiobookDetailResponse{
Audiobook: audiobookDetailItem{
ContentID: item.ContentID,
Title: item.Title,
Year: item.Year,
Overview: item.Overview,
PosterURL: h.presignAudiobookPoster(r.Context(), item.PosterPath),
Publisher: pickFirstString(item.Studios),
Genres: append([]string(nil), item.Genres...),
},
Author: author,
Narrator: narrator,
Files: audiobookDetailFiles(files),
Progress: progress,
AlsoByAuthor: alsoByAuthor,
InSeries: inSeries,
SimilarAudiobooks: similar,
OtherNarrations: otherNarrations,
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
// fetchAuthorNarrator retrieves the first author (kind=7) and narrator (kind=8)
// name from item_people for the given content ID.
func (h *AudiobookHandler) fetchAuthorNarrator(r *http.Request, contentID string) (author, narrator string, err error) {
people, err := h.Items.GetPeople(r.Context(), contentID)
if err != nil {
return "", "", err
}
for _, p := range people {
switch p.Kind {
case models.PersonKindAuthor:
if author == "" {
author = p.Name
}
case models.PersonKindNarrator:
if narrator == "" {
narrator = p.Name
}
}
if author != "" && narrator != "" {
break
}
}
return author, narrator, nil
}
// fetchAlsoByAuthor returns other audiobooks sharing any author (item_people
// kind=PersonKindAuthor) with the given book. Capped at 12 to keep the rail
// snappy on the client. Returns an empty slice when there is no author or
// no other matching titles.
func (h *AudiobookHandler) fetchAlsoByAuthor(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Files == nil {
return nil
}
const q = `
SELECT m2.content_id, m2.title, COALESCE(m2.year, 0), COALESCE(m2.poster_path, '')
FROM item_people ip1
JOIN item_people ip2
ON ip2.person_id = ip1.person_id
AND ip2.kind = ip1.kind
AND ip2.content_id <> ip1.content_id
JOIN media_items m2
ON m2.content_id = ip2.content_id
AND m2.type = 'audiobook'
WHERE ip1.content_id = $1
AND ip1.kind = $2
ORDER BY m2.year DESC NULLS LAST, LOWER(m2.sort_title)
LIMIT 12
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookRelatedItem, 0, 12)
seen := make(map[string]struct{})
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
if _, dup := seen[it.ContentID]; dup {
continue
}
seen[it.ContentID] = struct{}{}
it.PosterURL = h.presignPoster(ctx, poster)
out = append(out, it)
}
return out
}
// fetchSimilar returns the "You might also like" rail. Prefers embedding-based
// nearest-neighbor search when the source book has an embedding; falls back to
// shared-genre matching otherwise (book is brand new, embedding job hasn't run,
// or the embedding lookup errored). The genre fallback excludes same-author
// books so the rail complements rather than duplicates also_by_author.
func (h *AudiobookHandler) fetchSimilar(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Recs != nil {
if items := h.fetchSimilarByEmbedding(ctx, contentID); items != nil {
return items
}
}
return h.fetchSimilarByGenres(ctx, contentID)
}
// fetchSimilarByEmbedding returns nearest-neighbor audiobooks by cosine
// similarity on the gemini embedding vector. Returns nil (not an empty slice)
// when the source book has no embedding yet, so the caller can fall back to
// the genre-based recommender.
func (h *AudiobookHandler) fetchSimilarByEmbedding(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Recs == nil || h.Files == nil {
return nil
}
emb, err := h.Recs.GetEmbedding(ctx, contentID)
if err != nil || emb == nil {
return nil
}
// Pull extra candidates so we can drop the source book's own author from
// the rail (they belong in also_by_author).
scored, err := h.Recs.FindSimilar(ctx, emb, []string{contentID}, "audiobook", 36)
if err != nil || len(scored) == 0 {
return nil
}
ids := make([]string, 0, len(scored))
for _, s := range scored {
ids = append(ids, s.MediaItemID)
}
const q = `
WITH this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE m.content_id = ANY($3)
AND m.type = 'audiobook'
AND NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor, ids)
if err != nil {
return nil
}
defer rows.Close()
byID := make(map[string]audiobookRelatedItem, len(scored))
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
it.PosterURL = h.presignPoster(ctx, poster)
byID[it.ContentID] = it
}
out := make([]audiobookRelatedItem, 0, 12)
for _, s := range scored {
if it, ok := byID[s.MediaItemID]; ok {
out = append(out, it)
if len(out) >= 12 {
break
}
}
}
if len(out) == 0 {
return nil
}
return out
}
// fetchOtherNarrations returns sibling audiobook items that share the same
// "core" title (after stripping the common "read by X" / "(Read by X)" /
// "(UK Version: Read by X)" suffixes) and the same author. Powers the
// narrator picker dropdown on the detail page when a single book has
// multiple narration editions stored as separate items in the catalog.
func (h *AudiobookHandler) fetchOtherNarrations(ctx context.Context, contentID string) []audiobookNarration {
if h.Files == nil {
return nil
}
// The regex strips a trailing block of the form "[ separator] [( ]read by ...[) ]".
// Tested against patterns in the corpus: "Foo Read By X",
// "Foo - read by X", "Foo (Read by X)", "Foo (UK Version: Read by X)".
const stripRE = `\s*\(?\s*[-:,]?\s*(UK Version:?|US Version:?)?\s*[Rr]ead [Bb]y [^()]+\)?\s*$`
const q = `
WITH this_book AS (
SELECT trim(regexp_replace(title, $1, '', 'g')) AS norm_title
FROM media_items WHERE content_id = $2
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $2 AND kind = $3
)
SELECT
m.content_id,
m.title,
COALESCE(MAX(p.name), '') AS narrator,
COALESCE(m.year, 0)
FROM media_items m
LEFT JOIN item_people ipn ON ipn.content_id = m.content_id AND ipn.kind = $4
LEFT JOIN people p ON p.id = ipn.person_id
WHERE m.type = 'audiobook'
AND m.content_id <> $2
AND trim(regexp_replace(m.title, $1, '', 'g')) = (SELECT norm_title FROM this_book)
AND EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $3
AND ip.person_id IN (SELECT person_id FROM this_author)
)
GROUP BY m.content_id, m.title, m.year
ORDER BY m.year DESC NULLS LAST, m.title
LIMIT 8
`
rows, err := h.Files.Pool().Query(ctx, q,
stripRE, contentID, models.PersonKindAuthor, models.PersonKindNarrator)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookNarration, 0, 4)
for rows.Next() {
var n audiobookNarration
if err := rows.Scan(&n.ContentID, &n.Title, &n.Narrator, &n.Year); err != nil {
return nil
}
out = append(out, n)
}
if len(out) == 0 {
return nil
}
return out
}
// fetchSimilarByGenres returns up to 12 audiobooks sharing genres with the
// given book, excluding books by the same author so the rail complements
// rather than duplicates also_by_author. Used as a fallback when no
// embedding exists for the source yet.
func (h *AudiobookHandler) fetchSimilarByGenres(ctx context.Context, contentID string) []audiobookRelatedItem {
if h.Files == nil {
return nil
}
const q = `
WITH this_genres AS (
SELECT unnest(genres) AS g FROM media_items WHERE content_id = $1
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE m.type = 'audiobook'
AND m.content_id <> $1
AND m.genres && (SELECT array_agg(g) FROM this_genres)
AND NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)
ORDER BY
cardinality(ARRAY(SELECT unnest(m.genres) INTERSECT SELECT g FROM this_genres)) DESC,
COALESCE(m.year, 0) DESC,
LOWER(m.sort_title)
LIMIT 12
`
rows, err := h.Files.Pool().Query(ctx, q, contentID, models.PersonKindAuthor)
if err != nil {
return nil
}
defer rows.Close()
out := make([]audiobookRelatedItem, 0, 12)
for rows.Next() {
var it audiobookRelatedItem
var poster string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster); err != nil {
return nil
}
it.PosterURL = h.presignPoster(ctx, poster)
out = append(out, it)
}
return out
}
// fetchInSeries returns the series the audiobook belongs to (if any) along
// with the ordered list of sibling entries. Driven by the audiobook_series
// table populated by either the scanner (tag-derived) or migration 145's
// best-effort title-pattern backfill.
func (h *AudiobookHandler) fetchInSeries(ctx context.Context, contentID string) *audiobookSeriesGroup {
if h.Files == nil {
return nil
}
const q = `
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(m.poster_path, ''),
s.series_name,
s.series_index
FROM audiobook_series root
JOIN audiobook_series s ON LOWER(s.series_name) = LOWER(root.series_name)
JOIN media_items m ON m.content_id = s.content_id AND m.type = 'audiobook'
WHERE root.content_id = $1
ORDER BY s.series_index NULLS LAST, LOWER(m.sort_title)
LIMIT 30
`
rows, err := h.Files.Pool().Query(ctx, q, contentID)
if err != nil {
return nil
}
defer rows.Close()
var seriesName string
entries := make([]audiobookRelatedItem, 0, 16)
for rows.Next() {
var (
it audiobookRelatedItem
poster string
thisName string
idxNum *float64
)
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &poster, &thisName, &idxNum); err != nil {
return nil
}
if seriesName == "" {
seriesName = thisName
}
if idxNum != nil {
n := int(*idxNum)
if float64(n) == *idxNum {
it.SeriesIndex = &n
}
}
it.PosterURL = h.presignPoster(ctx, poster)
entries = append(entries, it)
}
// A series with only this book is not useful as a rail.
if len(entries) < 2 {
return nil
}
return &audiobookSeriesGroup{Name: seriesName, Entries: entries}
}
// fetchProgress returns the caller's listening progress for this audiobook, or
// nil when no progress exists or the caller is not authenticated with a profile.
func (h *AudiobookHandler) fetchProgress(r *http.Request, contentID string) *audiobookDetailProgress {
if h.StoreProvider == nil {
return nil
}
userID := apimw.GetUserID(r.Context())
if userID == 0 {
return nil
}
profileID := apimw.GetProfileID(r.Context())
if profileID == "" {
profileID = r.Header.Get("X-Profile-Id")
}
if profileID == "" {
return nil
}
store, err := h.StoreProvider.ForUser(r.Context(), userID)
if err != nil || store == nil {
return nil
}
wp, err := store.GetProgress(r.Context(), profileID, contentID)
if err != nil || wp == nil {
return nil
}
return &audiobookDetailProgress{
PositionSeconds: wp.PositionSeconds,
Completed: wp.Completed,
UpdatedAt: wp.UpdatedAt,
}
}
type audiobookDetailResponse struct {
Audiobook audiobookDetailItem `json:"audiobook"`
Author string `json:"author,omitempty"`
Narrator string `json:"narrator,omitempty"`
Files []audiobookDetailFile `json:"files"`
Progress *audiobookDetailProgress `json:"progress,omitempty"`
AlsoByAuthor []audiobookRelatedItem `json:"also_by_author,omitempty"`
InSeries *audiobookSeriesGroup `json:"in_series,omitempty"`
SimilarAudiobooks []audiobookRelatedItem `json:"similar_audiobooks,omitempty"`
OtherNarrations []audiobookNarration `json:"other_narrations,omitempty"`
}
// audiobookNarration is a sibling narration (same book, different
// narrator/edition). Returned alongside the primary item so the UI can
// surface a narrator picker.
type audiobookNarration struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Narrator string `json:"narrator,omitempty"`
Year int `json:"year,omitempty"`
}
type audiobookRelatedItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
SeriesIndex *int `json:"series_index,omitempty"`
}
type audiobookSeriesGroup struct {
Name string `json:"name,omitempty"`
Entries []audiobookRelatedItem `json:"entries"`
}
type audiobookDetailItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year"`
Overview string `json:"overview,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
Publisher string `json:"publisher,omitempty"`
Genres []string `json:"genres,omitempty"`
}
func (h *AudiobookHandler) presignPoster(ctx context.Context, path string) string {
if path == "" || h.Detail == nil {
return ""
}
return h.Detail.PresignURL(ctx, path, "featured")
}
func pickFirstString(values []string) string {
for _, v := range values {
if v != "" {
return v
}
}
return ""
}
type audiobookDetailFile struct {
ID int `json:"id"`
Path string `json:"path"`
DurationSeconds int `json:"duration_seconds"`
Container string `json:"container,omitempty"`
CodecAudio string `json:"codec_audio,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
AudioChannels int `json:"audio_channels,omitempty"`
Chapters []models.MediaChapter `json:"chapters,omitempty"`
}
type audiobookDetailProgress struct {
PositionSeconds float64 `json:"position_seconds"`
Completed bool `json:"completed"`
UpdatedAt string `json:"updated_at"`
}
func audiobookDetailFiles(files []*models.MediaFile) []audiobookDetailFile {
out := make([]audiobookDetailFile, 0, len(files))
for _, f := range files {
out = append(out, audiobookDetailFile{
ID: f.ID,
Path: f.FilePath,
DurationSeconds: f.Duration,
Container: f.Container,
CodecAudio: f.CodecAudio,
Bitrate: f.Bitrate,
AudioChannels: f.AudioChannels,
Chapters: f.Chapters,
})
}
return out
}
+183
View File
@@ -0,0 +1,183 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/recommendations"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// AudiobookHandler serves silo-native audiobook endpoints under
// /api/v1/audiobooks/*. Sub-plan 3 implements list/detail/progress;
// sub-plan 4's ABS-compat layer is separate.
type AudiobookHandler struct {
Items *catalog.ItemRepository
Files *scanner.FileRepository
StoreProvider userstore.UserStoreProvider
Detail *catalog.DetailService
// Recs is optional. When set, the detail handler uses embedding-based
// nearest-neighbor search for the "You might also like" rail and falls
// back to shared-genre matching only when no embedding exists for the
// source book yet.
Recs *recommendations.Repo
}
// HandleListAudiobooks serves GET /api/v1/audiobooks?limit=&offset=.
// Paginated catalog of media_items with type='audiobook' scoped to the
// caller's accessible libraries.
func (h *AudiobookHandler) HandleListAudiobooks(w http.ResponseWriter, r *http.Request) {
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
if limit <= 0 || limit > 200 {
limit = 50
}
offset, _ := strconv.Atoi(r.URL.Query().Get("offset"))
if offset < 0 {
offset = 0
}
ctx := r.Context()
access := requestAccessFilter(r)
genre := r.URL.Query().Get("genre")
conditions, args, argIdx, empty := audiobookListConditions(access, genre)
if empty {
writeAudiobookListResponse(w, nil, 0, limit, offset)
return
}
pool := h.Files.Pool()
var total int
countSQL := "SELECT COUNT(*) FROM media_items mi WHERE " + strings.Join(conditions, " AND ")
if err := pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "list audiobooks count failed")
return
}
dataArgs := append([]any(nil), args...)
limitIdx := argIdx
dataArgs = append(dataArgs, limit)
offsetIdx := argIdx + 1
dataArgs = append(dataArgs, offset)
rows, err := pool.Query(ctx, fmt.Sprintf(`
SELECT
mi.content_id,
mi.title,
COALESCE(mi.year, 0),
COALESCE(mi.poster_path, ''),
COALESCE(SUM(mf.duration), 0)::int AS duration_seconds,
COALESCE(MIN(mf.container), '') AS container,
COALESCE(MIN(mf.codec_audio), '') AS codec_audio,
COUNT(mf.id) AS file_count
FROM media_items mi
LEFT JOIN media_files mf ON mf.content_id = mi.content_id
WHERE %s
GROUP BY mi.content_id, mi.title, mi.year, mi.poster_path, mi.sort_title
ORDER BY LOWER(mi.sort_title), LOWER(mi.title)
LIMIT $%d OFFSET $%d
`, strings.Join(conditions, " AND "), limitIdx, offsetIdx), dataArgs...)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "list audiobooks failed")
return
}
defer rows.Close()
items := make([]audiobookListItem, 0, limit)
for rows.Next() {
var it audiobookListItem
var posterPath string
if err := rows.Scan(&it.ContentID, &it.Title, &it.Year, &posterPath, &it.DurationSeconds, &it.Container, &it.CodecAudio, &it.FileCount); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "scan audiobook row failed")
return
}
it.PosterURL = h.presignAudiobookPoster(ctx, posterPath)
items = append(items, it)
}
if err := rows.Err(); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "iterate audiobook rows failed")
return
}
writeAudiobookListResponse(w, items, total, limit, offset)
}
func audiobookListConditions(access catalog.AccessFilter, genre string) ([]string, []any, int, bool) {
conditions := []string{"mi.type = 'audiobook'"}
var args []any
argIdx := 1
if genre != "" {
conditions = append(conditions, fmt.Sprintf("$%d = ANY(mi.genres)", argIdx))
args = append(args, genre)
argIdx++
}
if access.AllowedLibraryIDs != nil {
if len(access.AllowedLibraryIDs) == 0 {
return nil, nil, 1, true
}
conditions = append(conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
AND mil.media_folder_id = ANY($%d)
)`, argIdx))
args = append(args, access.AllowedLibraryIDs)
argIdx++
}
if len(access.DisabledLibraryIDs) > 0 {
if access.AllowedLibraryIDs == nil {
conditions = append(conditions, `
EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
)`)
}
conditions = append(conditions, fmt.Sprintf(`
NOT EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id
AND mil.media_folder_id = ANY($%d)
)`, argIdx))
args = append(args, access.DisabledLibraryIDs)
argIdx++
}
catalog.ApplySectionAccessFilter("mi", catalog.AccessFilter{MaxContentRating: access.MaxContentRating}, &conditions, &args, &argIdx)
return conditions, args, argIdx, false
}
func writeAudiobookListResponse(w http.ResponseWriter, items []audiobookListItem, total, limit, offset int) {
if items == nil {
items = []audiobookListItem{}
}
resp := struct {
Items []audiobookListItem `json:"items"`
Total int `json:"total"`
Limit int `json:"limit"`
Offset int `json:"offset"`
}{Items: items, Total: total, Limit: limit, Offset: offset}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
type audiobookListItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year"`
PosterURL string `json:"poster_url,omitempty"`
DurationSeconds int `json:"duration_seconds"`
Container string `json:"container,omitempty"`
CodecAudio string `json:"codec_audio,omitempty"`
FileCount int `json:"file_count"`
}
func (h *AudiobookHandler) presignAudiobookPoster(ctx context.Context, path string) string {
if h == nil || h.Detail == nil || path == "" {
return ""
}
return h.Detail.PresignURL(ctx, cardThumbnailPath(path), "card")
}
@@ -0,0 +1,113 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type audiobookProgressRequest struct {
PositionSeconds float64 `json:"position_seconds"`
MediaFileID int `json:"media_file_id,omitempty"`
}
// HandleReportAudiobookProgress serves POST /api/v1/audiobooks/{id}/progress.
// UPSERTs into user_watch_progress for the caller's (user_id, profile_id, content_id).
func (h *AudiobookHandler) HandleReportAudiobookProgress(w http.ResponseWriter, r *http.Request) {
contentID := chi.URLParam(r, "id")
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "id is required")
return
}
var req audiobookProgressRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid request body")
return
}
if req.PositionSeconds < 0 {
writeError(w, http.StatusBadRequest, "bad_request", "position_seconds must be >= 0")
return
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if profileID == "" {
profileID = r.Header.Get("X-Profile-Id")
}
if userID == 0 || profileID == "" {
writeError(w, http.StatusUnauthorized, "unauthorized", "authentication required")
return
}
if h.Items == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "catalog unavailable")
return
}
items, err := h.Items.GetByIDsWithAccess(r.Context(), []string{contentID}, requestAccessFilter(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "check access failed")
return
}
if len(items) == 0 || items[0] == nil || items[0].Type != "audiobook" {
writeError(w, http.StatusNotFound, "not_found", "audiobook not found")
return
}
durationSeconds, err := h.audiobookProgressDuration(r.Context(), contentID, req.MediaFileID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "load files failed")
return
}
if h.StoreProvider == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "store unavailable")
return
}
store, err := h.StoreProvider.ForUser(r.Context(), userID)
if err != nil || store == nil {
writeError(w, http.StatusInternalServerError, "internal_error", "store unavailable")
return
}
if err := store.SetProgress(r.Context(), profileID, contentID, req.PositionSeconds, durationSeconds, userstore.ProgressThresholds{}); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "save progress failed")
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *AudiobookHandler) audiobookProgressDuration(ctx context.Context, contentID string, mediaFileID int) (float64, error) {
if h == nil || h.Files == nil {
return 0, nil
}
files, err := h.Files.GetByContentID(ctx, contentID)
if err != nil {
return 0, err
}
return audiobookDurationForProgress(files, mediaFileID), nil
}
func audiobookDurationForProgress(files []*models.MediaFile, mediaFileID int) float64 {
var total int
for _, f := range files {
if f == nil || f.Duration <= 0 {
continue
}
if mediaFileID > 0 && f.ID == mediaFileID {
return float64(f.Duration)
}
total += f.Duration
}
if total <= 0 {
return 0
}
return float64(total)
}
+61
View File
@@ -0,0 +1,61 @@
package handlers
import (
"context"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestPresignAudiobookPosterDoesNotExposeRawKeysWithoutPresigner(t *testing.T) {
h := &AudiobookHandler{}
got := h.presignAudiobookPoster(context.Background(), "metadata/audiobooks/book-1/poster/original.webp")
if got != "" {
t.Fatalf("PosterURL = %q, want empty without presigner", got)
}
}
func TestAudiobookListConditionsIncludeAccessPredicates(t *testing.T) {
conditions, args, _, empty := audiobookListConditions(catalog.AccessFilter{
AllowedLibraryIDs: []int{1, 2},
DisabledLibraryIDs: []int{9},
MaxContentRating: "PG-13",
}, "Fantasy")
if empty {
t.Fatal("empty = true, want false")
}
sql := strings.Join(conditions, " AND ")
for _, want := range []string{
"mi.type = 'audiobook'",
"ANY(mi.genres)",
"EXISTS",
"NOT EXISTS",
"mi.content_rating = ANY",
} {
if !strings.Contains(sql, want) {
t.Fatalf("conditions missing %q in:\n%s", want, sql)
}
}
if len(args) != 4 {
t.Fatalf("len(args) = %d, want 4", len(args))
}
}
func TestAudiobookDurationForProgress(t *testing.T) {
files := []*models.MediaFile{
{ID: 10, Duration: 120},
{ID: 11, Duration: 180},
}
if got := audiobookDurationForProgress(files, 10); got != 120 {
t.Fatalf("selected file duration = %v, want 120", got)
}
if got := audiobookDurationForProgress(files, 0); got != 300 {
t.Fatalf("total duration = %v, want 300", got)
}
if got := audiobookDurationForProgress([]*models.MediaFile{{ID: 1}}, 1); got != 0 {
t.Fatalf("unknown duration = %v, want 0", got)
}
}
+57 -94
View File
@@ -36,7 +36,6 @@ type autoscanStore interface {
CountAutoscanScans(ctx context.Context, filter autoscan.ScanListFilter) (int, error)
ListEvents(ctx context.Context, filter autoscan.EventListFilter) ([]autoscan.EventWithRuns, error)
CountEvents(ctx context.Context, filter autoscan.EventListFilter) (int, error)
ListRunningEvents(ctx context.Context) ([]autoscan.Event, error)
GetQueueSummary(ctx context.Context) (autoscan.QueueSummary, error)
LatestEventAt(ctx context.Context) (*time.Time, error)
}
@@ -310,7 +309,7 @@ func (h *AutoscanHandler) HandleDeleteConnection(w http.ResponseWriter, r *http.
// credentials: the connection link is by id only.
type autoscanSourceResponse struct {
ID string `json:"id"`
PluginID string `json:"plugin_id"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
@@ -332,7 +331,7 @@ func sourceResponse(s autoscan.Source) autoscanSourceResponse {
config := normalizeSourceConfig(s.SourceConfig)
return autoscanSourceResponse{
ID: s.ID,
PluginID: s.PluginID,
InstallationID: s.InstallationID,
CapabilityID: s.CapabilityID,
ConnectionID: s.ConnectionID,
Enabled: s.Enabled,
@@ -365,9 +364,10 @@ func (h *AutoscanHandler) HandleListSources(w http.ResponseWriter, r *http.Reque
// --- Available scan-source plugins (Add-source picker) ---
type autoscanScanSourcePluginResponse struct {
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
DisplayName string `json:"display_name"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
PluginID string `json:"plugin_id"`
DisplayName string `json:"display_name"`
}
// HandleListAvailableScanSources returns every installed scan_source capability
@@ -381,9 +381,10 @@ func (h *AutoscanHandler) HandleListAvailableScanSources(w http.ResponseWriter,
out := make([]autoscanScanSourcePluginResponse, 0, len(available))
for _, a := range available {
out = append(out, autoscanScanSourcePluginResponse{
PluginID: a.PluginID,
CapabilityID: a.CapabilityID,
DisplayName: a.DisplayName,
InstallationID: a.InstallationID,
CapabilityID: a.CapabilityID,
PluginID: a.PluginID,
DisplayName: a.DisplayName,
})
}
writeJSON(w, http.StatusOK, struct {
@@ -393,12 +394,12 @@ func (h *AutoscanHandler) HandleListAvailableScanSources(w http.ResponseWriter,
// --- Create source ---
// autoscanCreateSourceInput is the create payload: it names the scan_source
// capability to bind (plugin_id + capability_id) plus the initial
// binding/scheduling fields. Unlike the update payload, identity is supplied
// here rather than read from an existing row.
// autoscanCreateSourceInput is the create payload: it names the installed
// scan_source capability to bind (installation_id + capability_id) plus the
// initial binding/scheduling fields. Unlike the update payload, identity is
// supplied here rather than read from an existing row.
type autoscanCreateSourceInput struct {
PluginID string `json:"plugin_id"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
@@ -419,11 +420,6 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
return
}
capID := strings.TrimSpace(in.CapabilityID)
pluginID := strings.TrimSpace(in.PluginID)
if pluginID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "plugin_id is required")
return
}
if capID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "capability_id is required")
return
@@ -436,7 +432,7 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
// Validate the (plugin_id, capability_id) is a currently-installed
// Validate the (installation_id, capability_id) is a currently-installed
// scan_source capability — a source may only be created against an installed
// plugin capability.
available, err := h.svc.ListAvailableScanSources(r.Context())
@@ -444,13 +440,13 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeAutoscanError(w, err)
return
}
if !scanSourceInstalled(available, pluginID, capID) {
writeError(w, http.StatusBadRequest, "bad_request", "plugin_id + capability_id is not a currently-installed scan_source capability")
if !scanSourceInstalled(available, in.InstallationID, capID) {
writeError(w, http.StatusBadRequest, "bad_request", "installation_id + capability_id is not a currently-installed scan_source capability")
return
}
connArg := normalizeConnectionID(in.ConnectionID)
created, err := h.repo.CreateSource(r.Context(), autoscan.Source{
PluginID: pluginID,
InstallationID: in.InstallationID,
CapabilityID: capID,
ConnectionID: connArg,
Enabled: in.Enabled,
@@ -466,18 +462,18 @@ func (h *AutoscanHandler) HandleCreateSource(w http.ResponseWriter, r *http.Requ
writeJSON(w, http.StatusCreated, sourceResponse(created))
}
// scanSourceInstalled reports whether (pluginID, capabilityID) is in the
// scanSourceInstalled reports whether (installationID, capabilityID) is in the
// set of currently-installed scan_source capabilities.
func scanSourceInstalled(available []autoscan.AvailableScanSource, pluginID, capabilityID string) bool {
func scanSourceInstalled(available []autoscan.AvailableScanSource, installationID int, capabilityID string) bool {
for _, a := range available {
if a.PluginID == pluginID && a.CapabilityID == capabilityID {
if a.InstallationID == installationID && a.CapabilityID == capabilityID {
return true
}
}
return false
}
// autoscanSourceInput is the source write payload. The (plugin_id,
// autoscanSourceInput is the source write payload. The (installation_id,
// capability_id) identity is read from the existing source row, so only the
// schedulable/binding fields are accepted.
//
@@ -552,7 +548,7 @@ func (h *AutoscanHandler) HandleUpdateSource(w http.ResponseWriter, r *http.Requ
// connection_id is a full-state field: nil means unbind, a UUID string
// means bind. A whitespace-only string is normalised to nil (unbound).
connArg := normalizeConnectionID(in.ConnectionID)
// Update by id; identity (plugin_id, capability_id) is immutable and
// Update by id; identity (installation_id, capability_id) is immutable and
// preserved by the repo. A missing source maps to 404.
updated, err := h.repo.UpdateSource(r.Context(), autoscan.Source{
ID: id,
@@ -585,8 +581,9 @@ func normalizeConnectionID(in *string) *string {
}
// HandleDeleteSource removes a source row. This is the operator's escape hatch
// for an orphaned source (one whose scan_source plugin was uninstalled or no
// longer resolves cleanly). An unknown id maps to 404 via autoscan.ErrNotFound.
// for an orphaned source (one whose scan_source plugin was uninstalled): the
// poll loop skips such rows quietly, and this endpoint clears them. An unknown
// id maps to 404 via autoscan.ErrNotFound.
func (h *AutoscanHandler) HandleDeleteSource(w http.ResponseWriter, r *http.Request) {
id := strings.TrimSpace(chi.URLParam(r, "id"))
if err := h.repo.DeleteSource(r.Context(), id); err != nil {
@@ -713,7 +710,7 @@ type autoscanEventScanRunResponse struct {
type autoscanEventResponse struct {
ID int64 `json:"id"`
SourceID *string `json:"source_id"`
PluginID string `json:"plugin_id"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
StartedAt time.Time `json:"started_at"`
CompletedAt time.Time `json:"completed_at"`
@@ -756,7 +753,7 @@ func eventResponse(event autoscan.EventWithRuns) autoscanEventResponse {
return autoscanEventResponse{
ID: e.ID,
SourceID: e.SourceID,
PluginID: e.PluginID,
InstallationID: e.InstallationID,
CapabilityID: e.CapabilityID,
StartedAt: e.StartedAt,
CompletedAt: e.CompletedAt,
@@ -797,9 +794,9 @@ func (h *AutoscanHandler) HandleListEvents(w http.ResponseWriter, r *http.Reques
filter.Offset = offset
}
switch filter.Status {
case "", autoscan.EventStatusRunning, autoscan.EventStatusSuccess, autoscan.EventStatusError, autoscan.EventStatusUnresolved:
case "", autoscan.EventStatusSuccess, autoscan.EventStatusError, autoscan.EventStatusUnresolved:
default:
writeError(w, http.StatusBadRequest, "bad_request", "status must be running, success, error, or unresolved")
writeError(w, http.StatusBadRequest, "bad_request", "status must be success, error, or unresolved")
return
}
@@ -838,7 +835,7 @@ type autoscanScanResponse struct {
CompletedAt *time.Time `json:"completed_at,omitempty"`
AutoscanEventID *int64 `json:"autoscan_event_id,omitempty"`
SourceID *string `json:"source_id,omitempty"`
PluginID string `json:"plugin_id,omitempty"`
InstallationID *int `json:"installation_id,omitempty"`
CapabilityID string `json:"capability_id,omitempty"`
EventStatus string `json:"event_status,omitempty"`
EventCompletedAt *time.Time `json:"event_completed_at,omitempty"`
@@ -865,7 +862,7 @@ func autoscanScanRunResponse(scan autoscan.ScanWithEvent) autoscanScanResponse {
CompletedAt: scan.CompletedAt,
AutoscanEventID: scan.AutoscanEventID,
SourceID: scan.SourceID,
PluginID: scan.PluginID,
InstallationID: scan.InstallationID,
CapabilityID: scan.CapabilityID,
EventStatus: string(scan.EventStatus),
EventCompletedAt: scan.EventCompletedAt,
@@ -926,35 +923,24 @@ func (h *AutoscanHandler) HandleListScans(w http.ResponseWriter, r *http.Request
// --- Status ---
type autoscanStatusSource struct {
ID string `json:"id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
Label string `json:"label"`
PathRewrites []autoscan.PathRewrite `json:"path_rewrites"`
LastRunAt *time.Time `json:"last_run_at,omitempty"`
LastError *string `json:"last_error,omitempty"`
}
type autoscanRunningPollResponse struct {
ID int64 `json:"id"`
SourceID *string `json:"source_id"`
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
StartedAt time.Time `json:"started_at"`
ElapsedMS int64 `json:"elapsed_ms"`
MarkerBefore *string `json:"marker_before,omitempty"`
ID string `json:"id"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
ConnectionID *string `json:"connection_id"`
Enabled bool `json:"enabled"`
Label string `json:"label"`
PathRewrites []autoscan.PathRewrite `json:"path_rewrites"`
LastRunAt *time.Time `json:"last_run_at,omitempty"`
LastError *string `json:"last_error,omitempty"`
}
type autoscanStatusResponse struct {
Enabled bool `json:"enabled"`
Sources []autoscanStatusSource `json:"sources"`
RunningPolls []autoscanRunningPollResponse `json:"running_polls"`
ActiveScans int `json:"active_scans"`
AcceptedScans int `json:"accepted_scans"`
RunningScans int `json:"running_scans"`
LatestEventAt *time.Time `json:"latest_event_at,omitempty"`
Enabled bool `json:"enabled"`
Sources []autoscanStatusSource `json:"sources"`
ActiveScans int `json:"active_scans"`
AcceptedScans int `json:"accepted_scans"`
RunningScans int `json:"running_scans"`
LatestEventAt *time.Time `json:"latest_event_at,omitempty"`
}
func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
@@ -974,11 +960,6 @@ func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
writeAutoscanError(w, err)
return
}
runningEvents, err := h.repo.ListRunningEvents(ctx)
if err != nil {
writeAutoscanError(w, err)
return
}
latestEventAt, err := h.repo.LatestEventAt(ctx)
if err != nil {
writeAutoscanError(w, err)
@@ -991,38 +972,20 @@ func (h *AutoscanHandler) HandleStatus(w http.ResponseWriter, r *http.Request) {
rewrites = []autoscan.PathRewrite{}
}
trimmed = append(trimmed, autoscanStatusSource{
ID: src.ID,
PluginID: src.PluginID,
CapabilityID: src.CapabilityID,
ConnectionID: src.ConnectionID,
Enabled: src.Enabled,
Label: src.Label,
PathRewrites: rewrites,
LastRunAt: src.LastRunAt,
LastError: src.LastError,
})
}
now := time.Now()
runningPolls := make([]autoscanRunningPollResponse, 0, len(runningEvents))
for _, event := range runningEvents {
elapsed := now.Sub(event.StartedAt)
if elapsed < 0 {
elapsed = 0
}
runningPolls = append(runningPolls, autoscanRunningPollResponse{
ID: event.ID,
SourceID: event.SourceID,
PluginID: event.PluginID,
CapabilityID: event.CapabilityID,
StartedAt: event.StartedAt,
ElapsedMS: elapsed.Milliseconds(),
MarkerBefore: event.MarkerBefore,
ID: src.ID,
InstallationID: src.InstallationID,
CapabilityID: src.CapabilityID,
ConnectionID: src.ConnectionID,
Enabled: src.Enabled,
Label: src.Label,
PathRewrites: rewrites,
LastRunAt: src.LastRunAt,
LastError: src.LastError,
})
}
writeJSON(w, http.StatusOK, autoscanStatusResponse{
Enabled: s.Enabled,
Sources: trimmed,
RunningPolls: runningPolls,
ActiveScans: queue.Active,
AcceptedScans: queue.Accepted,
RunningScans: queue.Running,
+22 -52
View File
@@ -31,7 +31,6 @@ type fakeAutoscanStore struct {
countScansFn func(autoscan.ScanListFilter) (int, error)
listEventsFn func(autoscan.EventListFilter) ([]autoscan.EventWithRuns, error)
countEventsFn func(autoscan.EventListFilter) (int, error)
listRunningFn func() ([]autoscan.Event, error)
queueSummaryFn func() (autoscan.QueueSummary, error)
latestEventAtFn func() (*time.Time, error)
}
@@ -141,13 +140,6 @@ func (f *fakeAutoscanStore) CountEvents(_ context.Context, filter autoscan.Event
return 0, nil
}
func (f *fakeAutoscanStore) ListRunningEvents(context.Context) ([]autoscan.Event, error) {
if f.listRunningFn != nil {
return f.listRunningFn()
}
return nil, nil
}
func (f *fakeAutoscanStore) GetQueueSummary(context.Context) (autoscan.QueueSummary, error) {
if f.queueSummaryFn != nil {
return f.queueSummaryFn()
@@ -281,7 +273,7 @@ func TestAutoscanHandleListSourcesListsOnly(t *testing.T) {
store := &fakeAutoscanStore{
listSourcesFn: func() ([]autoscan.Source, error) {
listed = true
return []autoscan.Source{{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr"}}, nil
return []autoscan.Source{{ID: "src-1", InstallationID: 1, CapabilityID: "arr"}}, nil
},
}
h := NewAutoscanHandler(store, &fakeAutoscanTriggerer{})
@@ -308,7 +300,7 @@ func TestAutoscanHandleListSourcesListsOnly(t *testing.T) {
func TestAutoscanHandleListAvailableScanSources(t *testing.T) {
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{PluginID: "sonarr", CapabilityID: "arr", DisplayName: "Sonarr"},
{InstallationID: 1, CapabilityID: "arr", PluginID: "sonarr", DisplayName: "Sonarr"},
}}
h := NewAutoscanHandler(&fakeAutoscanStore{}, trig)
@@ -339,19 +331,19 @@ func TestAutoscanHandleCreateSourceSucceeds(t *testing.T) {
},
}
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{PluginID: "silo.autoscan.arr", CapabilityID: "arr"},
{InstallationID: 1, CapabilityID: "arr"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","connection_id":"conn-1","enabled":true}`, "")
`{"installation_id":1,"capability_id":"arr","connection_id":"conn-1","enabled":true}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
if rec.Code != http.StatusCreated {
t.Fatalf("status = %d, want 201; body=%s", rec.Code, rec.Body.String())
}
if got.PluginID != "silo.autoscan.arr" || got.CapabilityID != "arr" || got.ConnectionID == nil || *got.ConnectionID != "conn-1" {
if got.InstallationID != 1 || got.CapabilityID != "arr" || got.ConnectionID == nil || *got.ConnectionID != "conn-1" {
t.Fatalf("unexpected created source: %+v", got)
}
var body autoscanSourceResponse
@@ -371,14 +363,14 @@ func TestAutoscanHandleCreateSourceRejectsUnknownCapability(t *testing.T) {
return s, nil
},
}
// Installed set does NOT include ("silo.autoscan.arr", "arr").
// Installed set does NOT include (1, "arr").
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{PluginID: "silo.autoscan.other", CapabilityID: "other"},
{InstallationID: 2, CapabilityID: "other"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","enabled":false}`, "")
`{"installation_id":1,"capability_id":"arr","enabled":false}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
@@ -402,12 +394,12 @@ func TestAutoscanHandleCreateSourceEnableWithoutConnectionSucceeds(t *testing.T)
},
}
trig := &fakeAutoscanTriggerer{available: []autoscan.AvailableScanSource{
{PluginID: "silo.autoscan.arr", CapabilityID: "arr"},
{InstallationID: 1, CapabilityID: "arr"},
}}
h := NewAutoscanHandler(store, trig)
req := newAutoscanRequest("POST", "/api/v1/admin/autoscan/sources",
`{"plugin_id":"silo.autoscan.arr","capability_id":"arr","enabled":true}`, "")
`{"installation_id":1,"capability_id":"arr","enabled":true}`, "")
rec := httptest.NewRecorder()
h.HandleCreateSource(rec, req)
@@ -623,7 +615,7 @@ func TestAutoscanHandleUpdateSourceEnableWithoutConnectionSucceeds(t *testing.T)
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -649,7 +641,7 @@ func TestAutoscanHandleUpdateSourceEnableWithConnectionSucceeds(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -677,7 +669,7 @@ func TestAutoscanHandleUpdateSourceBindConnectionSucceeds(t *testing.T) {
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
// Source starts with no connection.
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -705,7 +697,7 @@ func TestAutoscanHandleUpdateSourceUnbindConnectionSucceeds(t *testing.T) {
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
// Source starts with a connection already bound.
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -735,7 +727,7 @@ func TestAutoscanHandleUpdateSourceUnbindWhileEnabledSucceeds(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -933,7 +925,7 @@ func TestAutoscanHandleUpdateSourceRoundTripsPathRewrites(t *testing.T) {
var got autoscan.Source
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
got = s
@@ -998,7 +990,7 @@ func TestAutoscanHandleUpdateSourceRejectsBlankRewrite(t *testing.T) {
upserted := false
store := &fakeAutoscanStore{
getSourceFn: func(id string) (autoscan.Source, error) {
return autoscan.Source{ID: id, PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
return autoscan.Source{ID: id, InstallationID: 1, CapabilityID: "arr", ConnectionID: ptr("conn-1")}, nil
},
updateSourceFn: func(s autoscan.Source) (autoscan.Source, error) {
upserted = true
@@ -1028,7 +1020,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
eventCompleted := completed.Add(-1 * time.Second)
eventID := int64(42)
sourceID := "src-1"
pluginID := "silo.autoscan.cephfs"
installationID := 7
var gotFilter autoscan.ScanListFilter
var gotCountFilter autoscan.ScanListFilter
store := &fakeAutoscanStore{
@@ -1048,7 +1040,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
},
AutoscanEventID: &eventID,
SourceID: &sourceID,
PluginID: pluginID,
InstallationID: &installationID,
CapabilityID: "cephfs",
EventStatus: autoscan.EventStatusSuccess,
EventCompletedAt: &eventCompleted,
@@ -1092,7 +1084,7 @@ func TestAutoscanHandleListScansSerializesFiltersAndEventContext(t *testing.T) {
if scan.AutoscanEventID == nil || *scan.AutoscanEventID != eventID || scan.EventStatus != "success" {
t.Fatalf("event context = %+v", scan)
}
if scan.SourceID == nil || *scan.SourceID != sourceID || scan.PluginID != pluginID || scan.CapabilityID != "cephfs" {
if scan.SourceID == nil || *scan.SourceID != sourceID || scan.InstallationID == nil || *scan.InstallationID != installationID || scan.CapabilityID != "cephfs" {
t.Fatalf("source context = %+v", scan)
}
}
@@ -1136,7 +1128,7 @@ func TestAutoscanHandleListEventsSerializesFiltersAndRuns(t *testing.T) {
Event: autoscan.Event{
ID: 42,
SourceID: &sourceID,
PluginID: "silo.autoscan.cephfs",
InstallationID: 7,
CapabilityID: "cephfs",
StartedAt: started,
CompletedAt: completed,
@@ -1266,30 +1258,18 @@ func TestAutoscanHandleUpdateSourceNormalizesLabel(t *testing.T) {
func TestAutoscanHandleStatusReturnsTrimmedSources(t *testing.T) {
latestEventAt := time.Date(2026, 6, 4, 14, 30, 0, 0, time.UTC)
runningStartedAt := time.Now().Add(-90 * time.Second)
sourceID := "src-1"
store := &fakeAutoscanStore{
getSettingsFn: func() (autoscan.Settings, error) {
return autoscan.Settings{Enabled: true}, nil
},
listSourcesFn: func() ([]autoscan.Source, error) {
return []autoscan.Source{
{ID: "src-1", PluginID: "silo.autoscan.cephfs", CapabilityID: "scan_source", ConnectionID: ptr("conn-1"), Enabled: true},
{ID: "src-1", InstallationID: 7, CapabilityID: "scan_source", ConnectionID: ptr("conn-1"), Enabled: true},
}, nil
},
queueSummaryFn: func() (autoscan.QueueSummary, error) {
return autoscan.QueueSummary{Active: 3, Accepted: 2, Running: 1}, nil
},
listRunningFn: func() ([]autoscan.Event, error) {
return []autoscan.Event{{
ID: 44,
SourceID: &sourceID,
PluginID: "silo.autoscan.cephfs",
CapabilityID: "cephfs",
StartedAt: runningStartedAt,
Status: autoscan.EventStatusRunning,
}}, nil
},
latestEventAtFn: func() (*time.Time, error) {
return &latestEventAt, nil
},
@@ -1316,16 +1296,6 @@ func TestAutoscanHandleStatusReturnsTrimmedSources(t *testing.T) {
if body.ActiveScans != 3 || body.AcceptedScans != 2 || body.RunningScans != 1 {
t.Errorf("queue summary = %+v", body)
}
if len(body.RunningPolls) != 1 {
t.Fatalf("running_polls = %+v", body.RunningPolls)
}
poll := body.RunningPolls[0]
if poll.ID != 44 || poll.SourceID == nil || *poll.SourceID != sourceID || poll.PluginID != "silo.autoscan.cephfs" || poll.CapabilityID != "cephfs" {
t.Fatalf("running poll identity = %+v", poll)
}
if poll.ElapsedMS <= 0 {
t.Fatalf("running poll elapsed_ms = %d, want > 0", poll.ElapsedMS)
}
if body.LatestEventAt == nil || !body.LatestEventAt.Equal(latestEventAt) {
t.Errorf("latest_event_at = %v, want %v", body.LatestEventAt, latestEventAt)
}
-3
View File
@@ -221,9 +221,6 @@ func (h *CatalogHandler) HandleGetCatalogFacetSearch(w http.ResponseWriter, r *h
writeError(w, http.StatusBadRequest, "bad_request", "limit must be a positive integer")
return
}
if n > 100 {
n = 100
}
limit = n
}
+1 -1
View File
@@ -497,7 +497,7 @@ func (h *CatalogResourceHandler) enrichItemDetail(r *http.Request, detail *catal
detail.SeasonUserData = h.items.getAggregateUserData(r, episodes)
}
}
case "movie", "episode", "audiobook":
case "movie", "episode":
detail.SeasonUserData = h.items.getLeafUserData(r, detail.ContentID)
applyEffectiveEditionPreference(detail.SeasonUserData, &detail.EffectiveVersionEditionKey)
}
-83
View File
@@ -225,9 +225,6 @@ type sortMetricsResponse struct {
ProgressRatio *float64 `json:"progress_ratio,omitempty"`
ViewedAt string `json:"viewed_at,omitempty"`
PlayCount *int `json:"play_count,omitempty"`
Author string `json:"author,omitempty"`
Narrator string `json:"narrator,omitempty"`
SeriesName string `json:"series_name,omitempty"`
}
// browseResponse is the paginated response for the /items endpoint.
@@ -894,90 +891,10 @@ func (h *ItemsHandler) listSortMetrics(
}
case "progress", "date_viewed", "plays":
h.listUserSortMetrics(ctx, items, sortField, store, profileID, metrics)
case "author", "narrator", "series":
h.listAudiobookSortMetrics(ctx, items, sortField, metrics)
}
return metrics
}
func (h *ItemsHandler) listAudiobookSortMetrics(
ctx context.Context,
items []*models.MediaItem,
sortField string,
metrics map[string]*sortMetricsResponse,
) {
if h == nil || h.browseRepo == nil || h.browseRepo.Pool() == nil || len(items) == 0 {
return
}
contentIDs := uniqueItemContentIDs(items)
if len(contentIDs) == 0 {
return
}
var query string
switch sortField {
case "author":
query = audiobookPersonSortMetricQuery(int(models.PersonKindAuthor))
case "narrator":
query = audiobookPersonSortMetricQuery(int(models.PersonKindNarrator))
case "series":
query = `
SELECT target.content_id, BTRIM(s.series_name) AS value
FROM unnest($1::text[]) WITH ORDINALITY AS target(content_id, ord)
JOIN audiobook_series s ON s.content_id = target.content_id
WHERE BTRIM(s.series_name) <> ''`
default:
return
}
rows, err := h.browseRepo.Pool().Query(ctx, query, contentIDs)
if err != nil {
return
}
defer rows.Close()
for rows.Next() {
var contentID, value string
if err := rows.Scan(&contentID, &value); err != nil {
return
}
value = strings.TrimSpace(value)
if contentID == "" || value == "" {
continue
}
resp := metrics[contentID]
if resp == nil {
resp = &sortMetricsResponse{}
metrics[contentID] = resp
}
switch sortField {
case "author":
resp.Author = value
case "narrator":
resp.Narrator = value
case "series":
resp.SeriesName = value
}
}
}
func audiobookPersonSortMetricQuery(kind int) string {
return fmt.Sprintf(`
SELECT target.content_id, person.name AS value
FROM unnest($1::text[]) WITH ORDINALITY AS target(content_id, ord)
JOIN LATERAL (
SELECT BTRIM(p.name) AS name
FROM item_people ip
JOIN people p ON p.id = ip.person_id
WHERE ip.content_id = target.content_id
AND ip.kind = %d
AND p.name IS NOT NULL
AND BTRIM(p.name) <> ''
ORDER BY p.name ASC
LIMIT 1
) person ON TRUE`, kind)
}
func (h *ItemsHandler) listUserSortMetrics(
ctx context.Context,
items []*models.MediaItem,
+2 -9
View File
@@ -574,11 +574,6 @@ func (h *LibraryHandler) HandleCreateLibrary(w http.ResponseWriter, r *http.Requ
if seedErr := h.SectionRepo.SeedDefaults(r.Context(), "library", &folder.ID, sections.DefaultLibrarySectionsForType(&folder.ID, folder.Type)); seedErr != nil {
slog.Warn("seed default sections for new library", "library_id", folder.ID, "error", seedErr)
}
if sections.IsAudiobookLibraryType(folder.Type) {
if _, seedErr := h.SectionRepo.EnsureHomeContinueListeningSection(r.Context()); seedErr != nil {
slog.Warn("ensure home continue listening section", "library_id", folder.ID, "error", seedErr)
}
}
if _, seedErr := h.SectionRepo.CreateGeneratedHomeLibraryRecentSections(r.Context(), folder.ID, folder.Name, folder.Type); seedErr != nil {
slog.Warn("seed generated home sections for new library", "library_id", folder.ID, "error", seedErr)
}
@@ -2041,10 +2036,8 @@ func (h *LibraryHandler) seedDefaultChain(ctx context.Context, libraryType strin
levels = []string{"series", "season", "episode"}
case "movies", "movie":
levels = []string{"movie"}
case "audiobooks", "audiobook":
levels = []string{"audiobook"}
case "mixed":
levels = []string{"movie", "series", "season", "episode", "audiobook"}
levels = []string{"movie", "series", "season", "episode"}
default:
return nil
}
@@ -2060,7 +2053,7 @@ func (h *LibraryHandler) seedDefaultChain(ctx context.Context, libraryType strin
var candidates []candidate
for _, c := range caps {
defaultPriority := metadata.LookupDefaultPriority(ctx, h.ChainRepo.Pool(), c.PluginInstallationID, c.CapabilityID, level)
defaultPriority := metadata.LookupDefaultPriority(ctx, h.ChainRepo.Pool(), c.PluginInstallationID, level)
if defaultPriority > 0 {
candidates = append(candidates, candidate{
installationID: c.PluginInstallationID,
+3 -16
View File
@@ -49,8 +49,6 @@ type SessionManagerInterface interface {
SetEffectiveMediaFileID(sessionID string, fileID int) error
SetTranscodeNodeURL(sessionID, url string) error
SetWebSocket(sessionID string, connected bool) error
SetRealtimeConnection(sessionID string, connected bool) error
SetProgressPersistenceDisabled(sessionID string, disabled bool) error
StopSession(sessionID string) error
GetSession(sessionID string) (*playback.Session, error)
}
@@ -259,7 +257,6 @@ type startPlaybackRequest struct {
StartPosition *float64 `json:"start_position,omitempty"`
AudioTrackIndex *int `json:"audio_track_index,omitempty"`
PreserveDirectAudioSelection bool `json:"preserve_direct_audio_selection,omitempty"`
DisableProgressPersistence bool `json:"disable_progress_persistence,omitempty"`
CodecsVideo []string `json:"codecs_video"`
CodecsAudio []string `json:"codecs_audio"`
Containers []string `json:"containers"`
@@ -764,9 +761,6 @@ func (h *PlaybackHandler) persistProgress(ctx context.Context, session *playback
if h.StoreProvider == nil || h.fileResolver == nil {
return
}
if session == nil || session.DisableProgressPersistence {
return
}
file, err := h.loadFileByPreferredID(ctx, requestedMediaFileID(session), session.MediaFileID)
targetID := playbackProgressTarget(file)
@@ -805,7 +799,7 @@ func (h *PlaybackHandler) persistStopAndHistory(ctx context.Context, session *pl
if h.StoreProvider == nil || h.fileResolver == nil {
return watchstate.PlaybackStopResult{}
}
if session == nil || session.DisableProgressPersistence || session.Position <= 0 {
if session == nil || session.Position <= 0 {
return watchstate.PlaybackStopResult{}
}
@@ -1287,13 +1281,6 @@ func (h *PlaybackHandler) HandleStartPlayback(w http.ResponseWriter, r *http.Req
return
}
setPlaybackSessionLogContext(r, session.ID)
if req.DisableProgressPersistence {
if err := h.sessionMgr.SetProgressPersistenceDisabled(session.ID, true); err != nil {
slog.Error("failed to disable progress persistence", "session", session.ID, "error", err)
} else {
session.DisableProgressPersistence = true
}
}
if err := h.sessionMgr.UpdateAudioTrack(session.ID, audioTrackIndex, session.PlayMethod); err != nil {
slog.Error("failed to set audio track", "session", session.ID, "error", err)
@@ -1333,7 +1320,7 @@ func (h *PlaybackHandler) HandleStartPlayback(w http.ResponseWriter, r *http.Req
} else {
h.restoreSessionProgress(r.Context(), session, file)
}
if !session.DisableProgressPersistence && h.WatchScrobbler != nil && effectiveFile != nil {
if h.WatchScrobbler != nil && effectiveFile != nil {
targetID := playbackProgressTarget(effectiveFile)
if targetID != "" {
event := h.scrobbleEventForSession(r.Context(), session, targetID, float64(effectiveFile.Duration), session.Position)
@@ -1562,7 +1549,7 @@ func (h *PlaybackHandler) HandleUpdateProgress(w http.ResponseWriter, r *http.Re
// Persist progress to UserStore (best-effort).
if sess, getErr := h.sessionMgr.GetSession(sessionID); getErr == nil {
h.persistProgress(r.Context(), sess)
if !sess.DisableProgressPersistence && h.WatchScrobbler != nil && wasPaused != sess.IsPaused {
if h.WatchScrobbler != nil && wasPaused != sess.IsPaused {
if file, loadErr := h.loadFileByPreferredID(r.Context(), requestedMediaFileID(sess), sess.MediaFileID); loadErr == nil && file != nil {
targetID := playbackProgressTarget(file)
if targetID != "" {
-62
View File
@@ -129,10 +129,6 @@ func (failingSessionManager) SetTranscodeNodeURL(string, string) error { return
func (failingSessionManager) SetWebSocket(string, bool) error { return nil }
func (failingSessionManager) SetRealtimeConnection(string, bool) error { return nil }
func (failingSessionManager) SetProgressPersistenceDisabled(string, bool) error { return nil }
func (failingSessionManager) StopSession(string) error { return nil }
func (failingSessionManager) GetSession(string) (*playback.Session, error) { return nil, nil }
@@ -287,64 +283,6 @@ func TestHandleStartPlayback_UsesExplicitZeroStartPositionInsteadOfStoredResume(
}
}
func TestPlaybackSessionProgressPersistenceCanBeDisabled(t *testing.T) {
store := newPlaybackTestStore(t)
if err := store.SetProgress(context.Background(), "profile-1", "book-1", 500, 1200, userstore.ProgressThresholds{}); err != nil {
t.Fatalf("seed progress: %v", err)
}
file := &models.MediaFile{
ID: 42,
ContentID: "book-1",
FilePath: writePlaybackTestMediaFile(t, "book.m4b"),
Duration: 600,
}
sessionMgr := playback.NewSessionManager(0, 0)
handler := NewPlaybackHandler(sessionMgr, testPlaybackFileResolver{file: file})
handler.StoreProvider = testUserStoreProvider{store: store}
handler.ItemAccess = allowAllPlaybackItemAccess{}
req := httptest.NewRequest("POST", "/api/v1/playback/start", strings.NewReader(`{"file_id":42,"profile_id":"profile-1","play_method":"direct","start_position":120,"disable_progress_persistence":true}`))
req = req.WithContext(newAuthorizedPlaybackContext())
rr := httptest.NewRecorder()
handler.HandleStartPlayback(rr, req)
if rr.Code != 201 {
t.Fatalf("start status = %d, body = %s", rr.Code, rr.Body.String())
}
var resp playbackSessionResponse
if err := json.NewDecoder(rr.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
progressReq := httptest.NewRequest("POST", "/api/v1/playback/"+resp.SessionID+"/progress", strings.NewReader(`{"position":240,"is_paused":false}`))
progressReq = progressReq.WithContext(newAuthorizedPlaybackContext())
progressReq = withPlaybackRouteParam(progressReq, "session_id", resp.SessionID)
progressRR := httptest.NewRecorder()
handler.HandleUpdateProgress(progressRR, progressReq)
if progressRR.Code != 204 {
t.Fatalf("progress status = %d, body = %s", progressRR.Code, progressRR.Body.String())
}
stopReq := httptest.NewRequest("DELETE", "/api/v1/playback/"+resp.SessionID, nil)
stopReq = stopReq.WithContext(newAuthorizedPlaybackContext())
stopReq = withPlaybackRouteParam(stopReq, "session_id", resp.SessionID)
stopRR := httptest.NewRecorder()
handler.HandleStopPlayback(stopRR, stopReq)
if stopRR.Code != 204 {
t.Fatalf("stop status = %d, body = %s", stopRR.Code, stopRR.Body.String())
}
progress, err := store.GetProgress(context.Background(), "profile-1", "book-1")
if err != nil {
t.Fatalf("get progress: %v", err)
}
if progress == nil || progress.PositionSeconds != 500 || progress.DurationSeconds != 1200 {
t.Fatalf("progress after disabled session = %+v, want 500/1200", progress)
}
}
func TestBuildTranscodeStartResponse_UnifiedSeekAnywhere(t *testing.T) {
file := &models.MediaFile{Duration: 3600}
+25 -107
View File
@@ -169,27 +169,22 @@ type pluginConfigSchemaJSON struct {
}
type pluginAdminFormJSON struct {
Fields []pluginAdminFormFieldJSON `json:"fields"`
SubmitLabel string `json:"submit_label,omitempty"`
Sections []pluginAdminFormSectionJSON `json:"sections,omitempty"`
Fields []pluginAdminFormFieldJSON `json:"fields"`
SubmitLabel string `json:"submit_label,omitempty"`
}
type pluginAdminFormFieldJSON struct {
Key string `json:"key"`
Label string `json:"label"`
Description string `json:"description,omitempty"`
Control string `json:"control"`
Placeholder string `json:"placeholder,omitempty"`
Required bool `json:"required"`
Secret bool `json:"secret"`
Multiline bool `json:"multiline"`
DefaultValue any `json:"default_value,omitempty"`
Options []pluginAdminFormOptionJSON `json:"options,omitempty"`
Rows int32 `json:"rows,omitempty"`
DynamicOptions bool `json:"dynamic_options,omitempty"`
ShowWhen []pluginAdminFormConditionJSON `json:"show_when,omitempty"`
Validation *pluginAdminFormValidationJSON `json:"validation,omitempty"`
ExclusiveGroupField string `json:"exclusive_group_field,omitempty"`
Key string `json:"key"`
Label string `json:"label"`
Description string `json:"description,omitempty"`
Control string `json:"control"`
Placeholder string `json:"placeholder,omitempty"`
Required bool `json:"required"`
Secret bool `json:"secret"`
Multiline bool `json:"multiline"`
DefaultValue any `json:"default_value,omitempty"`
Options []pluginAdminFormOptionJSON `json:"options,omitempty"`
Rows int32 `json:"rows,omitempty"`
}
type pluginAdminFormOptionJSON struct {
@@ -198,31 +193,6 @@ type pluginAdminFormOptionJSON struct {
Description string `json:"description,omitempty"`
}
type pluginAdminFormConditionJSON struct {
Field string `json:"field"`
Equals []string `json:"equals"`
}
type pluginAdminFormValidationJSON struct {
HasMin bool `json:"has_min,omitempty"`
Min float64 `json:"min,omitempty"`
HasMax bool `json:"has_max,omitempty"`
Max float64 `json:"max,omitempty"`
Pattern string `json:"pattern,omitempty"`
MinLength int32 `json:"min_length,omitempty"`
MaxLength int32 `json:"max_length,omitempty"`
}
type pluginAdminFormSectionJSON struct {
Key string `json:"key"`
Title string `json:"title"`
Description string `json:"description,omitempty"`
Collapsible bool `json:"collapsible"`
CollapsedDefault bool `json:"collapsed_default"`
FieldKeys []string `json:"field_keys"`
ShowWhen []pluginAdminFormConditionJSON `json:"show_when,omitempty"`
}
type pluginCapabilityJSON struct {
Type string `json:"type"`
ID string `json:"id"`
@@ -690,7 +660,7 @@ func (h *PluginHandler) syncMetadataProviders(ctx context.Context, installation
continue
}
if err := h.chainRepo.AppendProviderToAllChains(ctx, installation.ID, cap.ID, func(level string) int {
return metadata.LookupDefaultPriority(ctx, h.chainRepo.Pool(), installation.ID, cap.ID, level)
return metadata.LookupDefaultPriority(ctx, h.chainRepo.Pool(), installation.ID, level)
}); err != nil {
slog.Warn("failed to append provider to library chains",
"installation_id", installation.ID,
@@ -1372,78 +1342,26 @@ func adminFormToJSON(form *pluginv1.AdminFormDescriptor) *pluginAdminFormJSON {
if field.GetDefaultValue() != nil {
defaultValue = field.GetDefaultValue().AsInterface()
}
var validation *pluginAdminFormValidationJSON
if v := field.GetValidation(); v != nil {
validation = &pluginAdminFormValidationJSON{
HasMin: v.GetHasMin(),
Min: v.GetMin(),
HasMax: v.GetHasMax(),
Max: v.GetMax(),
Pattern: v.GetPattern(),
MinLength: v.GetMinLength(),
MaxLength: v.GetMaxLength(),
}
}
fields = append(fields, pluginAdminFormFieldJSON{
Key: field.GetKey(),
Label: field.GetLabel(),
Description: field.GetDescription(),
Control: strings.TrimPrefix(field.GetControl().String(), "ADMIN_FORM_CONTROL_"),
Placeholder: field.GetPlaceholder(),
Required: field.GetRequired(),
Secret: field.GetSecret(),
Multiline: field.GetMultiline(),
DefaultValue: defaultValue,
Options: options,
Rows: field.GetRows(),
DynamicOptions: field.GetDynamicOptions(),
ShowWhen: adminFormConditionsToJSON(field.GetShowWhen()),
Validation: validation,
ExclusiveGroupField: field.GetExclusiveGroupField(),
})
}
sections := make([]pluginAdminFormSectionJSON, 0, len(form.GetSections()))
for _, section := range form.GetSections() {
if section == nil {
continue
}
sections = append(sections, pluginAdminFormSectionJSON{
Key: section.GetKey(),
Title: section.GetTitle(),
Description: section.GetDescription(),
Collapsible: section.GetCollapsible(),
CollapsedDefault: section.GetCollapsedDefault(),
FieldKeys: append([]string(nil), section.GetFieldKeys()...),
ShowWhen: adminFormConditionsToJSON(section.GetShowWhen()),
Key: field.GetKey(),
Label: field.GetLabel(),
Description: field.GetDescription(),
Control: strings.TrimPrefix(field.GetControl().String(), "ADMIN_FORM_CONTROL_"),
Placeholder: field.GetPlaceholder(),
Required: field.GetRequired(),
Secret: field.GetSecret(),
Multiline: field.GetMultiline(),
DefaultValue: defaultValue,
Options: options,
Rows: field.GetRows(),
})
}
return &pluginAdminFormJSON{
Fields: fields,
SubmitLabel: form.GetSubmitLabel(),
Sections: sections,
}
}
func adminFormConditionsToJSON(conditions []*pluginv1.AdminFormCondition) []pluginAdminFormConditionJSON {
if len(conditions) == 0 {
return nil
}
out := make([]pluginAdminFormConditionJSON, 0, len(conditions))
for _, condition := range conditions {
if condition == nil {
continue
}
out = append(out, pluginAdminFormConditionJSON{
Field: condition.GetField(),
Equals: append([]string(nil), condition.GetEquals()...),
})
}
if len(out) == 0 {
return nil
}
return out
}
func capabilitiesToJSON(descriptors []*pluginv1.CapabilityDescriptor) []pluginCapabilityJSON {
response := make([]pluginCapabilityJSON, 0, len(descriptors))
for _, descriptor := range descriptors {
@@ -1,216 +0,0 @@
package handlers
import (
"encoding/json"
"reflect"
"strings"
"testing"
pluginv1 "github.com/Silo-Server/silo-plugin-sdk/pkg/pluginproto/silo/plugin/v1"
)
// TestAdminFormToJSONSchemaFields locks the bridge between the SDK proto
// AdminForm descriptor and the JSON the frontend SchemaForm consumes. Without
// this, the schema-driven request-connection form silently loses its
// dynamic dropdowns (dynamic_options), conditional visibility (show_when),
// validation, sections, and the MULTI_SELECT control mapping, which makes the
// new form non-functional (required fields can never be set, save stays
// disabled).
func TestAdminFormToJSONSchemaFields(t *testing.T) {
descriptor := &pluginv1.AdminFormDescriptor{
SubmitLabel: "Connect",
Fields: []*pluginv1.AdminFormField{
{
Key: "quality_profile_ids",
Label: "Quality Profiles",
Control: pluginv1.AdminFormControl_ADMIN_FORM_CONTROL_MULTI_SELECT,
DynamicOptions: true,
ExclusiveGroupField: "service_kind",
ShowWhen: []*pluginv1.AdminFormCondition{
{Field: "service_type", Equals: []string{"radarr", "sonarr"}},
},
Validation: &pluginv1.AdminFormValidation{
HasMin: true,
Min: 1,
HasMax: true,
Max: 10,
Pattern: "^[0-9]+$",
MinLength: 1,
MaxLength: 64,
},
},
},
Sections: []*pluginv1.AdminFormSection{
{
Key: "advanced",
Title: "Advanced",
Description: "Optional tuning",
Collapsible: true,
CollapsedDefault: true,
FieldKeys: []string{"quality_profile_ids"},
ShowWhen: []*pluginv1.AdminFormCondition{
{Field: "service_type", Equals: []string{"radarr"}},
},
},
},
}
form := adminFormToJSON(descriptor)
if form == nil {
t.Fatal("adminFormToJSON returned nil")
}
raw, err := json.Marshal(form)
if err != nil {
t.Fatalf("marshal admin form: %v", err)
}
var decoded struct {
Fields []struct {
Control string `json:"control"`
DynamicOptions bool `json:"dynamic_options"`
ShowWhen []struct {
Field string `json:"field"`
Equals []string `json:"equals"`
} `json:"show_when"`
Validation *struct {
HasMin bool `json:"has_min"`
Min float64 `json:"min"`
HasMax bool `json:"has_max"`
Max float64 `json:"max"`
Pattern string `json:"pattern"`
MinLength int32 `json:"min_length"`
MaxLength int32 `json:"max_length"`
} `json:"validation"`
ExclusiveGroupField string `json:"exclusive_group_field"`
} `json:"fields"`
Sections []struct {
Key string `json:"key"`
Title string `json:"title"`
Description string `json:"description"`
Collapsible bool `json:"collapsible"`
CollapsedDefault bool `json:"collapsed_default"`
FieldKeys []string `json:"field_keys"`
ShowWhen []struct {
Field string `json:"field"`
Equals []string `json:"equals"`
} `json:"show_when"`
} `json:"sections"`
}
if err := json.Unmarshal(raw, &decoded); err != nil {
t.Fatalf("unmarshal admin form: %v\njson: %s", err, raw)
}
if len(decoded.Fields) != 1 {
t.Fatalf("expected 1 field, got %d", len(decoded.Fields))
}
field := decoded.Fields[0]
// Control must serialize to the SHORT form the frontend switches on.
if field.Control != "MULTI_SELECT" {
t.Errorf("control = %q, want %q", field.Control, "MULTI_SELECT")
}
if !field.DynamicOptions {
t.Error("dynamic_options = false, want true")
}
if field.ExclusiveGroupField != "service_kind" {
t.Errorf("exclusive_group_field = %q, want %q", field.ExclusiveGroupField, "service_kind")
}
if len(field.ShowWhen) != 1 || field.ShowWhen[0].Field != "service_type" ||
len(field.ShowWhen[0].Equals) != 2 {
t.Errorf("field show_when not serialized correctly: %+v", field.ShowWhen)
}
if field.Validation == nil {
t.Fatal("validation not serialized")
}
if !field.Validation.HasMin || field.Validation.Min != 1 ||
!field.Validation.HasMax || field.Validation.Max != 10 ||
field.Validation.Pattern != "^[0-9]+$" ||
field.Validation.MinLength != 1 || field.Validation.MaxLength != 64 {
t.Errorf("validation not serialized correctly: %+v", field.Validation)
}
if len(decoded.Sections) != 1 {
t.Fatalf("expected 1 section, got %d", len(decoded.Sections))
}
section := decoded.Sections[0]
if section.Key != "advanced" || section.Title != "Advanced" ||
section.Description != "Optional tuning" || !section.Collapsible ||
!section.CollapsedDefault {
t.Errorf("section scalar fields not serialized correctly: %+v", section)
}
if len(section.FieldKeys) != 1 || section.FieldKeys[0] != "quality_profile_ids" {
t.Errorf("section field_keys not serialized correctly: %+v", section.FieldKeys)
}
if len(section.ShowWhen) != 1 || section.ShowWhen[0].Field != "service_type" ||
len(section.ShowWhen[0].Equals) != 1 {
t.Errorf("section show_when not serialized correctly: %+v", section.ShowWhen)
}
}
// adminFormDTOMappingRenames records intentional proto-field -> DTO-field name
// renames. The DTO field names currently mirror the proto field names 1:1, so
// this is empty; add an entry only when a DTO field is deliberately named
// differently from its proto source (the test then accepts the renamed target).
var adminFormDTOMappingRenames = map[string]string{}
// protoInternalFields are the protobuf-runtime bookkeeping fields every
// generated message struct carries. They are never serialized to the client
// DTO, so the completeness guard skips them.
var protoInternalFields = map[string]bool{
"state": true,
"sizeCache": true,
"unknownFields": true,
}
// TestAdminFormDTOCompleteness is a guard against the C1-class bug: the
// hand-written adminFormToJSON allowlist (pluginAdminFormFieldJSON /
// pluginAdminFormSectionJSON / pluginAdminFormJSON) silently dropping a NEW
// proto field added to AdminFormField / AdminFormDescriptor / AdminFormSection
// later. It reflects over each proto message's exported fields and asserts the
// corresponding JSON DTO struct has a field mapping to it (by identical Go name,
// or via an explicit rename in adminFormDTOMappingRenames). When the next proto
// field lands, this FAILS naming the un-mapped field, forcing the serializer +
// DTO to be updated in lockstep. (We deliberately do not switch the serializer
// to protojson — the short-form control-string MULTI_SELECT mapping makes that
// risky; this guard is the proportionate fix.)
func TestAdminFormDTOCompleteness(t *testing.T) {
cases := []struct {
name string
proto reflect.Type
dto reflect.Type
}{
{"AdminFormField", reflect.TypeOf(pluginv1.AdminFormField{}), reflect.TypeOf(pluginAdminFormFieldJSON{})},
{"AdminFormDescriptor", reflect.TypeOf(pluginv1.AdminFormDescriptor{}), reflect.TypeOf(pluginAdminFormJSON{})},
{"AdminFormSection", reflect.TypeOf(pluginv1.AdminFormSection{}), reflect.TypeOf(pluginAdminFormSectionJSON{})},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
dtoFields := map[string]bool{}
for i := 0; i < tc.dto.NumField(); i++ {
dtoFields[tc.dto.Field(i).Name] = true
}
for i := 0; i < tc.proto.NumField(); i++ {
pf := tc.proto.Field(i)
if !pf.IsExported() {
continue // unexported protobuf-internal state
}
if protoInternalFields[pf.Name] || strings.HasPrefix(pf.Name, "XXX_") {
continue
}
want := pf.Name
if mapped, ok := adminFormDTOMappingRenames[pf.Name]; ok {
want = mapped
}
if !dtoFields[want] {
t.Errorf("proto field %s.%s has no corresponding field %q in the JSON DTO %s; "+
"adminFormToJSON would silently drop it. Add the field to the DTO + serializer "+
"(or record an intentional rename in adminFormDTOMappingRenames).",
tc.name, pf.Name, want, tc.dto.Name())
}
}
})
}
}
+5 -27
View File
@@ -29,13 +29,6 @@ type rateLimitConfigResponse struct {
IPReqPerMinute float64 `json:"ip_requests_per_minute"`
IPBurst int `json:"ip_burst"`
AuthEndpoints map[string]authEndpointConfigResponse `json:"auth_endpoints"`
// Active reports whether a limiter is running in this process. The
// limiter is constructed at startup, so config saved while it is absent
// (or a backend change while it is running) needs a restart to apply.
Active bool `json:"active"`
// ActiveBackend is the backend the running limiter actually uses, which
// can differ from Backend until the server restarts.
ActiveBackend string `json:"active_backend,omitempty"`
}
type tierConfigResponse struct {
@@ -71,10 +64,6 @@ func (h *RateLimitHandler) HandleGetConfig(w http.ResponseWriter, r *http.Reques
IPReqPerMinute: cfg.IPReqPerMinute,
IPBurst: cfg.IPBurst,
AuthEndpoints: make(map[string]authEndpointConfigResponse),
Active: h.mw != nil,
}
if h.mw != nil {
resp.ActiveBackend = h.mw.ActiveBackend()
}
for name, tier := range cfg.Tiers {
resp.Tiers[name] = tierConfigResponse{
@@ -162,21 +151,10 @@ func (h *RateLimitHandler) HandleUpdateConfig(w http.ResponseWriter, r *http.Req
}
}
// The limiter is constructed at startup, so enabling while it is absent
// or switching backend while it runs only takes effect after a restart.
// Everything else hot-reloads below.
restartRequired := false
if h.mw == nil {
restartRequired = req.Enabled
} else {
if (req.Backend == "memory" || req.Backend == "redis") && req.Backend != h.mw.ActiveBackend() {
restartRequired = true
}
// Hot-reload: apply new config immediately on this instance
if err := h.mw.Reload(r.Context()); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Config saved but reload failed")
return
}
// Hot-reload: apply new config immediately on this instance
if err := h.mw.Reload(r.Context()); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Config saved but reload failed")
return
}
// Publish for multi-instance reload (if EventBus is available/backed by Redis)
@@ -187,5 +165,5 @@ func (h *RateLimitHandler) HandleUpdateConfig(w http.ResponseWriter, r *http.Req
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(map[string]any{"status": "ok", "restart_required": restartRequired})
json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
}
-140
View File
@@ -1,140 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/ratelimit"
)
type fakeRateLimitStore struct {
values map[string]string
}
func newFakeRateLimitStore() *fakeRateLimitStore {
return &fakeRateLimitStore{values: make(map[string]string)}
}
func (s *fakeRateLimitStore) Get(_ context.Context, key string) (string, error) {
return s.values[key], nil
}
func (s *fakeRateLimitStore) Set(_ context.Context, key, value string) error {
s.values[key] = value
return nil
}
func (s *fakeRateLimitStore) GetAll(_ context.Context) (map[string]string, error) {
out := make(map[string]string, len(s.values))
for k, v := range s.values {
out[k] = v
}
return out, nil
}
func newRunningRateLimitMiddleware(t *testing.T, store ratelimit.SettingsStore) *ratelimit.Middleware {
t.Helper()
perKey := ratelimit.NewMemoryLimiter()
global := ratelimit.NewMemoryLimiter()
t.Cleanup(func() {
perKey.Close()
global.Close()
})
mw := ratelimit.NewMiddleware(perKey, global, store, true)
if err := mw.Init(context.Background()); err != nil {
t.Fatalf("init middleware: %v", err)
}
return mw
}
func putRateLimitConfig(t *testing.T, h *RateLimitHandler, body string) map[string]any {
t.Helper()
req := httptest.NewRequest(http.MethodPut, "/admin/rate-limits/config", strings.NewReader(body))
rec := httptest.NewRecorder()
h.HandleUpdateConfig(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode PUT response: %v", err)
}
return resp
}
func getRateLimitConfig(t *testing.T, h *RateLimitHandler) rateLimitConfigResponse {
t.Helper()
req := httptest.NewRequest(http.MethodGet, "/admin/rate-limits/config", nil)
rec := httptest.NewRecorder()
h.HandleGetConfig(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("GET status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp rateLimitConfigResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode GET response: %v", err)
}
return resp
}
func TestRateLimitHandlerWithoutRunningLimiter(t *testing.T) {
// The limiter is nil when rate limiting was disabled at boot. The config
// endpoints must still work so an admin can re-enable it from the UI.
store := newFakeRateLimitStore()
store.values["ratelimit.enabled"] = "false"
h := NewRateLimitHandler(store, nil, nil)
got := getRateLimitConfig(t, h)
if got.Enabled {
t.Error("GET enabled = true, want false")
}
if got.Active {
t.Error("GET active = true, want false when no limiter is running")
}
resp := putRateLimitConfig(t, h, `{"enabled":true,"backend":"redis"}`)
if resp["restart_required"] != true {
t.Errorf("enabling without a running limiter: restart_required = %v, want true", resp["restart_required"])
}
if store.values["ratelimit.enabled"] != "true" {
t.Errorf("ratelimit.enabled = %q, want \"true\"", store.values["ratelimit.enabled"])
}
if store.values["ratelimit.backend"] != "redis" {
t.Errorf("ratelimit.backend = %q, want \"redis\"", store.values["ratelimit.backend"])
}
resp = putRateLimitConfig(t, h, `{"enabled":false,"backend":"redis"}`)
if resp["restart_required"] != false {
t.Errorf("saving disabled config without a running limiter: restart_required = %v, want false", resp["restart_required"])
}
}
func TestRateLimitHandlerWithRunningLimiter(t *testing.T) {
store := newFakeRateLimitStore()
mw := newRunningRateLimitMiddleware(t, store)
h := NewRateLimitHandler(store, mw, nil)
got := getRateLimitConfig(t, h)
if !got.Active {
t.Error("GET active = false, want true with a running limiter")
}
if got.ActiveBackend != "memory" {
t.Errorf("GET active_backend = %q, want \"memory\"", got.ActiveBackend)
}
// Toggling enabled hot-reloads; no restart needed when the backend matches.
resp := putRateLimitConfig(t, h, `{"enabled":false,"backend":"memory"}`)
if resp["restart_required"] != false {
t.Errorf("toggle on running limiter: restart_required = %v, want false", resp["restart_required"])
}
// Switching backend only takes effect at boot.
resp = putRateLimitConfig(t, h, `{"enabled":true,"backend":"redis"}`)
if resp["restart_required"] != true {
t.Errorf("backend change: restart_required = %v, want true", resp["restart_required"])
}
}
+53 -39
View File
@@ -38,7 +38,7 @@ type RequestService interface {
CreateIntegration(ctx context.Context, viewer mediarequests.Viewer, integration mediarequests.Integration) (*mediarequests.Integration, error)
UpdateIntegration(ctx context.Context, viewer mediarequests.Viewer, integration mediarequests.Integration) (*mediarequests.Integration, error)
DeleteIntegration(ctx context.Context, viewer mediarequests.Viewer, id string) error
LoadIntegrationOptions(ctx context.Context, viewer mediarequests.Viewer, integration mediarequests.Integration) (map[string][]mediarequests.RouterOption, error)
LoadIntegrationOptions(ctx context.Context, viewer mediarequests.Viewer, integration mediarequests.Integration) (*mediarequests.IntegrationOptions, error)
ListStudios(ctx context.Context, viewer mediarequests.Viewer) ([]mediarequests.DiscoverBrandCard, error)
ListNetworks(ctx context.Context, viewer mediarequests.Viewer) ([]mediarequests.DiscoverBrandCard, error)
@@ -588,36 +588,62 @@ func parsePositivePathInt(w http.ResponseWriter, r *http.Request, key string) (i
}
type requestIntegrationResponse struct {
ID string `json:"id"`
Name string `json:"name"`
CapabilityID string `json:"capability_id"`
InstallationID *int `json:"installation_id,omitempty"`
SupportedMediaTypes []string `json:"supported_media_types"`
PluginConfig map[string]any `json:"plugin_config"`
Enabled bool `json:"enabled"`
BaseURL string `json:"base_url"`
HasAPIKey bool `json:"has_api_key"`
LastCheckAt *time.Time `json:"last_check_at,omitempty"`
LastCheckStatus string `json:"last_check_status,omitempty"`
LastCheckError string `json:"last_check_error,omitempty"`
UpdatedAt time.Time `json:"updated_at"`
ID string `json:"id"`
Name string `json:"name"`
Kind string `json:"kind"`
Enabled bool `json:"enabled"`
Is4K bool `json:"is_4k"`
IsDefault bool `json:"is_default"`
IsDefault4K bool `json:"is_default_4k"`
AnimeEnabled bool `json:"anime_enabled"`
AnimeQualityProfileID *int `json:"anime_quality_profile_id,omitempty"`
AnimeRootFolder string `json:"anime_root_folder,omitempty"`
AnimeTags []int `json:"anime_tags"`
BaseURL string `json:"base_url"`
HasAPIKey bool `json:"has_api_key"`
RootFolder string `json:"root_folder"`
QualityProfileID *int `json:"quality_profile_id,omitempty"`
Tags []int `json:"tags"`
Options map[string]any `json:"options"`
LastCheckAt *time.Time `json:"last_check_at,omitempty"`
LastCheckStatus string `json:"last_check_status,omitempty"`
LastCheckError string `json:"last_check_error,omitempty"`
UpdatedAt time.Time `json:"updated_at"`
}
func requestIntegrationResponseFrom(integration mediarequests.Integration) requestIntegrationResponse {
// Normalize nil slices so they serialize as [] (not null); the frontend
// types them as number[] and indexes/maps over them.
tags := integration.Tags
if tags == nil {
tags = []int{}
}
animeTags := integration.AnimeTags
if animeTags == nil {
animeTags = []int{}
}
return requestIntegrationResponse{
ID: integration.ID,
Name: integration.Name,
CapabilityID: integration.CapabilityID,
InstallationID: integration.InstallationID,
SupportedMediaTypes: integration.SupportedMediaTypes,
PluginConfig: integration.PluginConfig,
Enabled: integration.Enabled,
BaseURL: integration.BaseURL,
HasAPIKey: strings.TrimSpace(integration.APIKeyRef) != "",
LastCheckAt: integration.LastCheckAt,
LastCheckStatus: integration.LastCheckStatus,
LastCheckError: integration.LastCheckError,
UpdatedAt: integration.UpdatedAt,
ID: integration.ID,
Name: integration.Name,
Kind: integration.Kind,
Enabled: integration.Enabled,
Is4K: integration.Is4K,
IsDefault: integration.IsDefault,
IsDefault4K: integration.IsDefault4K,
AnimeEnabled: integration.AnimeEnabled,
AnimeQualityProfileID: integration.AnimeQualityProfileID,
AnimeRootFolder: integration.AnimeRootFolder,
AnimeTags: animeTags,
BaseURL: integration.BaseURL,
HasAPIKey: strings.TrimSpace(integration.APIKeyRef) != "",
RootFolder: integration.RootFolder,
QualityProfileID: integration.QualityProfileID,
Tags: tags,
Options: integration.Options,
LastCheckAt: integration.LastCheckAt,
LastCheckStatus: integration.LastCheckStatus,
LastCheckError: integration.LastCheckError,
UpdatedAt: integration.UpdatedAt,
}
}
@@ -630,18 +656,6 @@ func toIntegrationResponses(integrations []mediarequests.Integration) []requestI
}
func writeRequestServiceError(w http.ResponseWriter, err error) {
// Plugin/instance validation failures carry inline field/form errors; surface
// them as a structured 400 so any handler routing through here renders them
// inline. Checked first because *ValidationError does not wrap a sentinel.
var verr *mediarequests.ValidationError
if errors.As(err, &verr) {
writeJSON(w, http.StatusBadRequest, map[string]any{
"error": "validation_failed",
"field_errors": verr.FieldErrors,
"form_error": verr.FormError,
})
return
}
var quota mediarequests.QuotaError
switch {
case errors.As(err, &quota):
+1 -1
View File
@@ -139,7 +139,7 @@ func (f *fakeRequestService) DeleteIntegration(context.Context, mediarequests.Vi
return nil
}
func (f *fakeRequestService) LoadIntegrationOptions(context.Context, mediarequests.Viewer, mediarequests.Integration) (map[string][]mediarequests.RouterOption, error) {
func (f *fakeRequestService) LoadIntegrationOptions(context.Context, mediarequests.Viewer, mediarequests.Integration) (*mediarequests.IntegrationOptions, error) {
return nil, nil
}
+6 -23
View File
@@ -32,7 +32,7 @@ type SectionHandler struct {
StoreProvider userstore.UserStoreProvider
UserRepo *auth.UserRepository
DetailSvc *catalog.DetailService
Settings catalog.SettingsStore
Settings *catalog.ServerSettingsRepo
CollectionRepo *catalog.LibraryCollectionRepository
}
@@ -157,11 +157,8 @@ func validateSectionConfig(sectionType sections.SectionType, config json.RawMess
}
cfg := sections.ParseConfigFilters(config)
if cfg.FilterType != "" && cfg.FilterType != "movie" && cfg.FilterType != "series" && cfg.FilterType != "audiobook" {
return "filter_type must be 'movie', 'series', or 'audiobook'", false
}
if _, err := sections.ParseContinueType(config); err != nil {
return err.Error(), false
if cfg.FilterType != "" && cfg.FilterType != "movie" && cfg.FilterType != "series" {
return "filter_type must be 'movie' or 'series'", false
}
var rawLibraryConfig struct {
@@ -1411,8 +1408,8 @@ func (h *SectionHandler) maybeInjectNextUp(ctx context.Context, resolved []secti
return resolved
}
// injectNextUpSection inserts a synthetic SectionNextUp entry after the
// contiguous continue rows that start with the video Continue Watching row.
// injectNextUpSection inserts a synthetic SectionNextUp entry immediately after
// SectionContinueWatching in the resolved sections list.
func injectNextUpSection(resolved []sections.ResolvedSection) []sections.ResolvedSection {
nextUp := sections.ResolvedSection{
ID: "system-next-up",
@@ -1421,20 +1418,6 @@ func injectNextUpSection(resolved []sections.ResolvedSection) []sections.Resolve
ItemLimit: 20,
}
for i, s := range resolved {
if s.SectionType == sections.SectionContinueWatching && sections.ContinueTypeFromConfig(s.Config) == sections.ContinueTypeWatching {
insertAt := i + 1
for insertAt < len(resolved) && resolved[insertAt].SectionType == sections.SectionContinueWatching {
insertAt++
}
result := make([]sections.ResolvedSection, 0, len(resolved)+1)
result = append(result, resolved[:insertAt]...)
result = append(result, nextUp)
result = append(result, resolved[insertAt:]...)
return result
}
}
for i, s := range resolved {
if s.SectionType == sections.SectionContinueWatching {
result := make([]sections.ResolvedSection, 0, len(resolved)+1)
@@ -1445,7 +1428,7 @@ func injectNextUpSection(resolved []sections.ResolvedSection) []sections.Resolve
}
}
// If no resume row is found, prepend.
// If no continue_watching found, prepend
return append([]sections.ResolvedSection{nextUp}, resolved...)
}
+1 -1
View File
@@ -13,7 +13,7 @@ const SectionsAllowProfileCustomSettingKey = "sections.allow_profile_custom_sect
// SectionSettingsHandler exposes GET/PUT for the sections-related server setting.
type SectionSettingsHandler struct {
Settings catalog.SettingsStore
Settings *catalog.ServerSettingsRepo
}
type sectionsSettingResponse struct {
-60
View File
@@ -282,66 +282,6 @@ func TestBuildSectionsResponseBatchResolvesImageURLs(t *testing.T) {
}
}
func TestValidateSectionConfigAcceptsContinueTypes(t *testing.T) {
tests := []string{
`{"continue_type":"watching"}`,
`{"continue_type":"listening"}`,
`{"continue_type":"reading"}`,
`{"filter_type":"audiobook"}`,
}
for _, config := range tests {
t.Run(config, func(t *testing.T) {
if msg, ok := validateSectionConfig(sections.SectionContinueWatching, []byte(config)); !ok {
t.Fatalf("validateSectionConfig(%s) rejected config: %s", config, msg)
}
})
}
}
func TestValidateSectionConfigRejectsUnknownContinueType(t *testing.T) {
msg, ok := validateSectionConfig(sections.SectionContinueWatching, []byte(`{"continue_type":"scrolling"}`))
if ok {
t.Fatal("validateSectionConfig accepted unknown continue_type")
}
if msg != "continue_type must be 'watching', 'listening', or 'reading'" {
t.Fatalf("message = %q", msg)
}
}
func TestInjectNextUpAfterContiguousContinueRows(t *testing.T) {
in := []sections.ResolvedSection{
{
ID: "cw",
SectionType: sections.SectionContinueWatching,
Title: "Continue Watching",
Config: sections.ContinueTypeConfig(sections.ContinueTypeWatching),
},
{
ID: "cl",
SectionType: sections.SectionContinueWatching,
Title: "Continue Listening",
Config: sections.ContinueTypeConfig(sections.ContinueTypeListening),
},
{ID: "recent", SectionType: sections.SectionRecentlyAdded, Title: "Recently Added"},
}
got := injectNextUpSection(in)
gotIDs := make([]string, 0, len(got))
for _, section := range got {
gotIDs = append(gotIDs, section.ID)
}
wantIDs := []string{"cw", "cl", "system-next-up", "recent"}
if len(gotIDs) != len(wantIDs) {
t.Fatalf("section ids = %v, want %v", gotIDs, wantIDs)
}
for i := range wantIDs {
if gotIDs[i] != wantIDs[i] {
t.Fatalf("section ids = %v, want %v", gotIDs, wantIDs)
}
}
}
func TestDropEmptySeasonalSectionsRemovesOnlyEmptySeasonal(t *testing.T) {
in := []sections.SectionWithItems{
// empty seasonal — drop
-139
View File
@@ -1,139 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"log/slog"
"net/http"
"time"
"github.com/google/uuid"
"github.com/Silo-Server/silo-server/internal/playback"
)
var ErrServerRestartAlreadyRequested = errors.New("server restart already requested")
type ServerControlHandler struct {
requestRestart func(ctx context.Context) error
commands *playback.CommandDispatcher
}
type serverRestartRequest struct {
Reason string `json:"reason"`
Title string `json:"title"`
Message string `json:"message"`
}
type serverRestartResponse struct {
Status string `json:"status"`
Message string `json:"message"`
NotifiedSessions int `json:"notified_sessions"`
}
func NewServerControlHandler(
requestRestart func(ctx context.Context) error,
commands *playback.CommandDispatcher,
) *ServerControlHandler {
return &ServerControlHandler{
requestRestart: requestRestart,
commands: commands,
}
}
// HandleRestart handles POST /admin/server/restart.
func (h *ServerControlHandler) HandleRestart(w http.ResponseWriter, r *http.Request) {
if h == nil || h.requestRestart == nil {
writeError(w, http.StatusServiceUnavailable, "service_unavailable", "Server restart is unavailable")
return
}
var req serverRestartRequest
if err := decodeOptionalJSONBody(r, &req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
notifiedSessions := h.notifyPlaybackSessions(req)
if err := h.requestRestart(context.Background()); err != nil {
if errors.Is(err, ErrServerRestartAlreadyRequested) {
writeJSON(w, http.StatusAccepted, serverRestartResponse{
Status: "already_requested",
Message: "Server restart is already in progress.",
NotifiedSessions: notifiedSessions,
})
return
}
slog.Error("server restart request failed", "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to request server restart")
return
}
writeJSON(w, http.StatusAccepted, serverRestartResponse{
Status: "restart_requested",
Message: "Server restart requested. The process will shut down gracefully.",
NotifiedSessions: notifiedSessions,
})
}
func (h *ServerControlHandler) notifyPlaybackSessions(req serverRestartRequest) int {
if h == nil || h.commands == nil {
return 0
}
title := req.Title
if title == "" {
title = "Server restarting"
}
message := req.Message
if message == "" {
message = "The server is restarting now. Playback may reconnect shortly."
}
payload, err := json.Marshal(map[string]string{
"title": title,
"message": message,
})
if err != nil {
slog.Warn("server restart: failed to build realtime payload", "error", err)
return 0
}
results := h.commands.DispatchToAll(func(session *playback.Session) (playback.CommandEnvelope, time.Duration, func(), error) {
if session == nil {
return playback.CommandEnvelope{}, 0, nil, playback.ErrCommandDispatchUnavailable
}
command, err := playback.NewCommandEnvelope(
session.ID,
uuid.NewString(),
playback.CommandServerRestarting,
payload,
)
if err != nil {
return playback.CommandEnvelope{}, 0, nil, err
}
command.Reason = req.Reason
if command.Reason == "" {
command.Reason = "server_restart_requested"
}
command.IssuedBy = &playback.CommandIssuedBy{Kind: "admin"}
return command, 0, nil, nil
})
delivered := 0
for _, result := range results {
if result.Delivered {
delivered++
continue
}
if result.DispatchErr != nil && !errors.Is(result.DispatchErr, playback.ErrRealtimeConnectionNotFound) {
slog.Warn("server restart: failed to notify playback session",
"session_id", result.SessionID,
"error", result.DispatchErr,
)
}
}
return delivered
}
@@ -1,156 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/playback"
)
type serverControlTestConn struct {
messages []any
}
func (c *serverControlTestConn) WriteJSON(v any) error {
c.messages = append(c.messages, v)
return nil
}
func TestServerControlRestartRequestsShutdown(t *testing.T) {
t.Parallel()
called := 0
handler := NewServerControlHandler(func(context.Context) error {
called++
return nil
}, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
if called != 1 {
t.Fatalf("restart calls = %d, want 1", called)
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.Status != "restart_requested" {
t.Fatalf("status = %q, want restart_requested", resp.Status)
}
if resp.NotifiedSessions != 0 {
t.Fatalf("notified_sessions = %d, want 0", resp.NotifiedSessions)
}
}
func TestServerControlRestartUnavailable(t *testing.T) {
t.Parallel()
handler := NewServerControlHandler(nil, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
}
func TestServerControlRestartAlreadyRequested(t *testing.T) {
t.Parallel()
handler := NewServerControlHandler(func(context.Context) error {
return ErrServerRestartAlreadyRequested
}, nil)
req := httptest.NewRequest(http.MethodPost, "/admin/server/restart", nil)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.Status != "already_requested" {
t.Fatalf("status = %q, want already_requested", resp.Status)
}
}
func TestServerControlRestartNotifiesPlaybackSessions(t *testing.T) {
t.Parallel()
sessionMgr := playback.NewSessionManager(0, 0)
session, err := sessionMgr.StartSession(1, "profile-1", 100, playback.PlayDirect, false)
if err != nil {
t.Fatalf("StartSession: %v", err)
}
realtimeHub := playback.NewRealtimeHub()
conn := &serverControlTestConn{}
registration := realtimeHub.Register(session.ID, conn)
if registration == nil {
t.Fatal("expected realtime registration")
}
defer realtimeHub.Unregister(registration)
dispatcher := playback.NewCommandDispatcher(sessionMgr, realtimeHub, nil)
handler := NewServerControlHandler(func(context.Context) error {
return nil
}, dispatcher)
req := httptest.NewRequest(
http.MethodPost,
"/admin/server/restart",
strings.NewReader(`{"reason":"maintenance","message":"Restarting for maintenance"}`),
)
rec := httptest.NewRecorder()
handler.HandleRestart(rec, req)
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp serverRestartResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.NotifiedSessions != 1 {
t.Fatalf("notified_sessions = %d, want 1", resp.NotifiedSessions)
}
if len(conn.messages) != 1 {
t.Fatalf("messages = %d, want 1", len(conn.messages))
}
command, ok := conn.messages[0].(playback.CommandEnvelope)
if !ok {
t.Fatalf("message type = %T, want playback.CommandEnvelope", conn.messages[0])
}
if command.Name != playback.CommandServerRestarting {
t.Fatalf("command name = %q, want %q", command.Name, playback.CommandServerRestarting)
}
if command.Reason != "maintenance" {
t.Fatalf("reason = %q, want maintenance", command.Reason)
}
var payload map[string]string
if err := json.Unmarshal(command.Payload, &payload); err != nil {
t.Fatalf("decode payload: %v", err)
}
if payload["message"] != "Restarting for maintenance" {
t.Fatalf("payload message = %q, want custom message", payload["message"])
}
}
-9
View File
@@ -20,7 +20,6 @@ const subtitleAppearanceSettingKey = "subtitle_appearance"
const (
libraryPageStateSettingKey = "ui.library_page_state"
rememberLibraryPageStateSettingKey = "ui.remember_library_page_state"
searchMediaScopeSettingKey = "search.media_scope"
)
const (
@@ -172,14 +171,6 @@ var settingsRegistry = map[string]settingSpec{
DefaultValue: "true",
Validate: validateBoolSetting(rememberLibraryPageStateSettingKey),
},
// Preferred default scope for global/catalog search. "video" keeps
// results to movies and series; "all" mixes audiobooks in.
searchMediaScopeSettingKey: {
Scope: scopeUser,
DefaultValue: "video",
Validate: validateEnumSetting(searchMediaScopeSettingKey,
"all", "video", "audiobook"),
},
"player.hdr_enabled": {
Scope: scopeDevice,
DefaultValue: "true",
+2 -4
View File
@@ -17,16 +17,14 @@ import (
type SystemHandler struct {
transcodePool *nodepool.TranscodePool
jwtSecret string
ffmpegPath string
buildInfo buildinfo.Info
}
// NewSystemHandler creates a SystemHandler.
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string, ffmpegPath string) *SystemHandler {
func NewSystemHandler(transcodePool *nodepool.TranscodePool, jwtSecret string) *SystemHandler {
return &SystemHandler{
transcodePool: transcodePool,
jwtSecret: jwtSecret,
ffmpegPath: ffmpegPath,
buildInfo: buildinfo.Current(),
}
}
@@ -47,7 +45,7 @@ func (h *SystemHandler) HandleHWAccel(w http.ResponseWriter, r *http.Request) {
}
}
info := playback.DetectHWAccelWithFFmpeg(h.ffmpegPath)
info := playback.DetectHWAccel()
writeJSON(w, http.StatusOK, info)
}
+5 -5
View File
@@ -32,11 +32,11 @@ type tasteSeedSubmitResponse struct {
// HandleTasteSeedItems handles GET /recommendations/taste-seed/items.
//
// Returns a paginated, hydrated list of posters used for the new-user
// taste-seeding picker. Blends server-watched popularity with rating reliability
// and recency so fresh servers (no watch history yet) still surface recognizable
// content. The user_state field carries the existing is_favorite flag, so the UI
// can pre-select items the profile already favorited.
// Returns a paginated, hydrated list of "popular on this server" posters used
// for the new-user taste-seeding picker. Blends server-watched popularity with
// TMDB rating so fresh servers (no watch history yet) still surface meaningful
// content. The user_state field carries the existing is_favorite flag, so the
// UI can pre-select items the profile already favorited.
func (h *RecommendationsHandler) HandleTasteSeedItems(w http.ResponseWriter, r *http.Request) {
if h.recsRepo == nil || h.Fetcher == nil {
writeJSON(w, http.StatusOK, tasteSeedItemsResponse{Items: []sectionItemResponse{}})
-36
View File
@@ -1,36 +0,0 @@
package api
import (
"context"
"errors"
"github.com/Silo-Server/silo-server/internal/plugins"
mediarequests "github.com/Silo-Server/silo-server/internal/requests"
)
// PluginRequestRouterAdapter adapts plugins.Service to
// mediarequests.RouterClientResolver. The concrete *pluginhost.RequestRouterClient
// satisfies mediarequests.RouterClient, so the adapter names the interface as its
// return type (Go has no return-type covariance).
type PluginRequestRouterAdapter struct {
Svc *plugins.Service
}
func (a PluginRequestRouterAdapter) RequestRouterClient(ctx context.Context, installationID int, capabilityID string) (mediarequests.RouterClient, error) {
if a.Svc == nil {
return nil, errors.New("request router plugin service is not configured")
}
return a.Svc.RequestRouterClient(ctx, installationID, capabilityID)
}
// AttachRequestRouter wires the plugin-backed router provider onto a requests
// service. Both the HTTP handler and the reconcile task call this so the wiring
// lives in one place. With either dependency absent (e.g. a build without the
// plugin service) it leaves the requests service router-less, so fulfillment
// degrades to the existing "no backend configured" failure instead of panicking.
func AttachRequestRouter(svc *mediarequests.Service, pluginService *plugins.Service) {
if svc == nil || pluginService == nil {
return
}
svc.SetRouterProvider(mediarequests.NewPluginRouterProvider(PluginRequestRouterAdapter{pluginService}))
}
-22
View File
@@ -1,22 +0,0 @@
package api
import (
"context"
"testing"
)
// AttachRequestRouter must tolerate absent dependencies (e.g. a build with the
// plugin service disabled) without panicking — fulfillment then degrades to the
// existing "no backend configured" failure rather than crashing the wiring.
func TestAttachRequestRouterNilDependenciesDoesNotPanic(t *testing.T) {
AttachRequestRouter(nil, nil)
}
// A router adapter installed without a plugin service must return a controlled
// error rather than panic on the request path.
func TestPluginRequestRouterAdapterNilServiceReturnsError(t *testing.T) {
adapter := PluginRequestRouterAdapter{}
if _, err := adapter.RequestRouterClient(context.Background(), 1, "request_router.v1"); err == nil {
t.Fatal("RequestRouterClient with nil Svc = nil error, want a controlled error")
}
}
+32 -28
View File
@@ -48,10 +48,11 @@ import (
"github.com/Silo-Server/silo-server/internal/ratelimit"
"github.com/Silo-Server/silo-server/internal/recommendations"
mediarequests "github.com/Silo-Server/silo-server/internal/requests"
"github.com/Silo-Server/silo-server/internal/requests/radarr"
"github.com/Silo-Server/silo-server/internal/requests/sonarr"
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/subtitles"
subtitleai "github.com/Silo-Server/silo-server/internal/subtitles/ai"
@@ -75,7 +76,6 @@ type Dependencies struct {
BootstrapSensitiveValues map[string]string
AppContext context.Context
DB *pgxpool.Pool
SecretCipher *secret.Cipher // at-rest credential cipher (required when DB is set)
FrontendFS fs.FS
S3Public *s3client.Client // public assets bucket client (may be nil)
S3Private *s3client.Client // private internal bucket client (may be nil)
@@ -143,7 +143,6 @@ type Dependencies struct {
PlaybackRealtimeHub *playback.RealtimeHub
OnUserSessionsRevoked func(ctx context.Context, userID int)
OnServerSettingUpdated func(ctx context.Context, key, value string)
RequestServerRestart func(ctx context.Context) error
// UserCollectionSync handles per-profile imported collections (TMDB /
// Trakt / MDBList) — the user-facing analogue of CollectionService.
@@ -165,6 +164,9 @@ type Dependencies struct {
// clients hitting /login, /api/*, /abs/api/*, and /abs/socket.io/* all
// resolve correctly. May be nil; no ABS routes are registered in that case.
ABSHandler absHandler
// AudiobooksEnabled gates native audiobook admin/UI routes behind the same
// server setting as the ABS compatibility listener.
AudiobooksEnabled bool
}
// absHandler is the narrow interface the router needs from the ABS handler.
@@ -223,11 +225,9 @@ func NewRouter(deps Dependencies) chi.Router {
healthHandler := handlers.NewHealthHandler(healthServerName, healthServerID)
// Build server settings repo if DB is available (needed by auth and admin).
// Wrap it in the encrypting decorator so sensitive keys rest as ciphertext
// and every consumer transparently reads plaintext.
var settingsRepo catalog.SettingsStore
var settingsRepo *catalog.ServerSettingsRepo
if deps.DB != nil {
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(deps.DB), deps.SecretCipher)
settingsRepo = catalog.NewServerSettingsRepo(deps.DB)
}
// Build auth handler and auth middleware if DB and config are available.
@@ -433,26 +433,27 @@ func NewRouter(deps Dependencies) chi.Router {
if deps.Config != nil {
tmdbAPIKey = deps.Config.TMDBAPIKey
}
requestsRepo := mediarequests.NewRepository(deps.DB, deps.SecretCipher)
requestsRepo := mediarequests.NewRepository(deps.DB)
requestSvc := mediarequests.NewService(
requestsRepo,
tmdb.NewClient(tmdbAPIKey, 40),
mediarequests.NewCatalogPresence(itemRepo, providerIDRepo),
)
AttachRequestRouter(requestSvc, deps.PluginService)
requestSvc.SetRequesterIdentityResolver(plugins.RequesterIdentityFromLookup(plugins.NewPgUserIdentityLookup(deps.DB)))
requestSvc.SetSecretResolver(settingsRepo)
requestSvc.SetFulfillmentAdapters(radarr.NewClient(nil), sonarr.NewClient(nil))
if viewerResolver != nil {
requestSvc.SetEntitlementResolver(mediarequests.NewAccessEntitlements(viewerResolver))
}
requestHandler = handlers.NewRequestsHandler(requestSvc)
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
autoscanRepo := autoscan.NewRepository(deps.DB)
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && deps.PluginService != nil {
autoscanSvc := BuildAutoscanService(
autoscanRepo,
deps.PluginService,
plugins.NewInstallationStore(deps.DB),
requestsRepo,
settingsRepo,
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
@@ -567,7 +568,7 @@ func NewRouter(deps Dependencies) chi.Router {
// Create subtitleRepo early — only needs DB, shared with playback handler and subtitle search handler.
var subtitleRepo *subtitles.PgRepository
if deps.DB != nil {
subtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
subtitleRepo = subtitles.NewPgRepository(deps.DB)
}
// Notifier that pushes "subtitle ready" events to active sessions when an AI
@@ -578,7 +579,6 @@ func NewRouter(deps Dependencies) chi.Router {
// Build playback handler if session manager is available.
var playbackHandler *handlers.PlaybackHandler
var adminPlaybackControlHandler *handlers.AdminPlaybackControlHandler
var playbackCommandDispatcher *playback.CommandDispatcher
var streamHandler *handlers.StreamHandler
var watchTogetherHandler *handlers.WatchTogetherHandler
if deps.SessionMgr != nil {
@@ -662,7 +662,6 @@ func NewRouter(deps Dependencies) chi.Router {
playbackHandler.RealtimeHub = realtimeHub
playbackHandler.CommandTracker = commandTracker
playbackHandler.CommandDispatcher = playback.NewCommandDispatcher(deps.SessionMgr, realtimeHub, commandTracker)
playbackCommandDispatcher = playbackHandler.CommandDispatcher
playbackHandler.IntroAnalyzer = deps.IntroAnalyzer
playbackHandler.IntroRepository = deps.IntroRepository
playbackHandler.MarkerRegistry = deps.MarkerRegistry
@@ -701,8 +700,6 @@ func NewRouter(deps Dependencies) chi.Router {
streamHandler.FFmpegPath = deps.Config.Playback.FFmpegPath
}
serverControlHandler := handlers.NewServerControlHandler(deps.RequestServerRestart, playbackCommandDispatcher)
// Build admin handler if we have a user repo.
var adminHandler *handlers.AdminHandler
var catalogSeedHandler *handlers.CatalogSeedHandler
@@ -1001,7 +998,6 @@ func NewRouter(deps Dependencies) chi.Router {
libraryCollectionService.TraktTokenResolver = &traktCollectionTokenResolver{
pool: deps.DB,
settings: settingsRepo,
cipher: deps.SecretCipher,
provider: watchtrakt.NewProvider(nil, ""),
}
}
@@ -1161,7 +1157,7 @@ func NewRouter(deps Dependencies) chi.Router {
var historyImportHandler *handlers.HistoryImportHandler
var historyImportSvc *historyimport.Service
if deps.DB != nil {
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyRepo := historyimport.NewRepository(deps.DB)
historyImportSvc = historyimport.NewService(deps.AppContext, historyRepo, deps.UserStoreProvider)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, providerIDRepo)
historyImportSvc.SetStableIdentityResolver(historyIdentity)
@@ -1170,7 +1166,7 @@ func NewRouter(deps Dependencies) chi.Router {
}
historyImportHandler = handlers.NewHistoryImportHandler(historyImportSvc)
if deps.UserStoreProvider != nil {
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB, deps.SecretCipher), historyRepo, deps.UserStoreProvider)
webhookSyncSvc := webhooksync.NewService(webhooksync.NewRepository(deps.DB), historyRepo, deps.UserStoreProvider)
webhookSyncSvc.SetStableIdentityResolver(historyIdentity)
webhookSyncHandler = handlers.NewWebhookSyncHandler(webhookSyncSvc)
}
@@ -1813,6 +1809,20 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/sections/recipes", recipeHandler.HandleList)
r.Get("/sections/recipes/{type}/candidates", recipeHandler.HandleCandidates)
// Audiobook endpoints (no profile required — catalog-level list/detail).
if deps.AudiobooksEnabled && itemRepo != nil {
audiobookHandler := &handlers.AudiobookHandler{
Items: itemRepo,
Files: deps.FileRepo,
Detail: detailSvc,
StoreProvider: deps.UserStoreProvider,
Recs: recsRepoForStale,
}
r.Get("/audiobooks", audiobookHandler.HandleListAudiobooks)
r.Get("/audiobooks/{id}", audiobookHandler.HandleGetAudiobook)
r.Post("/audiobooks/{id}/progress", audiobookHandler.HandleReportAudiobookProgress)
}
// Section endpoints (profile-scoped).
if sectionHandler != nil {
r.Group(func(r chi.Router) {
@@ -1910,7 +1920,6 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/playback-history", adminHandler.HandleListPlaybackHistory)
r.Get("/unmatched", adminHandler.HandleListUnmatched)
r.Get("/stats", adminHandler.HandleGetStats)
r.Post("/server/restart", serverControlHandler.HandleRestart)
r.Get("/settings/sensitive-status", adminHandler.HandleGetSensitiveStatus)
r.Post("/settings/check/{kind}", adminHandler.HandleCheckSettingsConnection)
if sectionSettingsHandler != nil {
@@ -2104,12 +2113,10 @@ func NewRouter(deps Dependencies) chi.Router {
// System inspection.
{
sysJWTSecret := ""
sysFFmpegPath := ""
if deps.Config != nil {
sysJWTSecret = deps.Config.Auth.JWTSecret
sysFFmpegPath = deps.Config.Playback.FFmpegPath
}
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret, sysFFmpegPath)
systemHandler := handlers.NewSystemHandler(deps.TranscodePool, sysJWTSecret)
r.Route("/system", func(r chi.Router) {
r.Get("/build", systemHandler.HandleBuildInfo)
r.Get("/hw-accel", systemHandler.HandleHWAccel)
@@ -2156,11 +2163,8 @@ func NewRouter(deps Dependencies) chi.Router {
})
}
// Rate limit admin routes. Mounted even when the limiter is not
// running (deps.RateLimitMW == nil) so admins can always reach the
// config — otherwise disabling rate limiting and restarting would
// permanently lock the settings page out of re-enabling it.
if settingsRepo != nil {
// Rate limit admin routes
if deps.RateLimitMW != nil && settingsRepo != nil {
rateLimitHandler := handlers.NewRateLimitHandler(settingsRepo, deps.RateLimitMW, deps.EventBus)
r.Route("/rate-limits", func(r chi.Router) {
r.Get("/config", rateLimitHandler.HandleGetConfig)
@@ -11,15 +11,13 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/watchsync"
watchtrakt "github.com/Silo-Server/silo-server/internal/watchsync/providers/trakt"
)
type traktCollectionTokenResolver struct {
pool *pgxpool.Pool
settings catalog.SettingsStore // encrypting decorator: decrypts trakt client_secret
cipher *secret.Cipher // decrypts watch_provider_connections tokens read via raw SQL
settings *catalog.ServerSettingsRepo
provider *watchtrakt.Provider
}
@@ -99,7 +97,7 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
LIMIT 1
`, profileID)
var conn watchsync.Connection
err := row.Scan(
if err := row.Scan(
&conn.ID,
&conn.Provider,
&conn.UserID,
@@ -109,45 +107,24 @@ func (r *traktCollectionTokenResolver) loadConnection(ctx context.Context, profi
&conn.AccessToken,
&conn.RefreshToken,
&conn.TokenExpiresAt,
)
if err != nil {
); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return watchsync.Connection{}, errors.New("trakt connection not found for profile")
}
return watchsync.Connection{}, fmt.Errorf("load trakt connection: %w", err)
}
// This resolver reads watch_provider_connections directly (bypassing the
// watchsync repo), so it must apply the same at-rest decryption the repo
// does, bound to the same connection identity (watchsync.TokenAAD).
if conn.AccessToken, err = r.cipher.DecryptIfEncrypted(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt access token: %w", err)
}
if conn.RefreshToken, err = r.cipher.DecryptIfEncrypted(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID)); err != nil {
return watchsync.Connection{}, fmt.Errorf("decrypt trakt refresh token: %w", err)
}
return conn, nil
}
func (r *traktCollectionTokenResolver) updateTokens(ctx context.Context, conn watchsync.Connection) error {
// Encrypt the refreshed tokens inline, bound to the connection identity, so
// the raw-SQL write matches what the watchsync repo (and this resolver's read
// path) store.
accessToken, err := r.cipher.Encrypt(conn.AccessToken, watchsync.TokenAAD("access_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt access token: %w", err)
}
refreshToken, err := r.cipher.Encrypt(conn.RefreshToken, watchsync.TokenAAD("refresh_token", conn.Provider, conn.UserID, conn.ProfileID))
if err != nil {
return fmt.Errorf("encrypt trakt refresh token: %w", err)
}
_, err = r.pool.Exec(ctx, `
_, err := r.pool.Exec(ctx, `
UPDATE watch_provider_connections
SET access_token = $2,
refresh_token = $3,
token_expires_at = $4,
updated_at = now()
WHERE id = $1::uuid
`, conn.ID, accessToken, refreshToken, conn.TokenExpiresAt)
`, conn.ID, conn.AccessToken, conn.RefreshToken, conn.TokenExpiresAt)
if err != nil {
return fmt.Errorf("update refreshed trakt connection tokens: %w", err)
}
@@ -18,8 +18,7 @@ func (h *Handler) handleAuthorDetail(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
slog.Warn("abs author access resolution failed", "err", err, "id", id)
http.Error(w, "forbidden", http.StatusForbidden)
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
author, err := h.deps.MediaStore.GetAuthorByID(r.Context(), id, access)
@@ -48,8 +47,7 @@ func (h *Handler) handleSeriesDetail(w http.ResponseWriter, r *http.Request) {
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
slog.Warn("abs series access resolution failed", "err", err, "id", id)
http.Error(w, "forbidden", http.StatusForbidden)
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
series, err := h.deps.MediaStore.GetSeriesByName(r.Context(), id, access)
@@ -44,9 +44,7 @@ func TestAuthor_Detail_ReturnsBooks(t *testing.T) {
t.Fatalf("status = %d", rec.Code)
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
_ = json.Unmarshal(rec.Body.Bytes(), &got)
if got["name"] != "Brandon Sanderson" {
t.Errorf("name = %v", got["name"])
}
@@ -80,9 +78,7 @@ func TestSeries_Detail_ReturnsBooks(t *testing.T) {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
_ = json.Unmarshal(rec.Body.Bytes(), &got)
if got["name"] != "Mistborn" {
t.Errorf("name = %v", got["name"])
}
+1 -6
View File
@@ -61,12 +61,7 @@ func (h *Handler) handleUpsertBookmark(reason string) http.HandlerFunc {
return
}
item, err := h.deps.MediaStore.GetAudiobookByID(r.Context(), itemID, access)
if err != nil {
slog.Error("abs bookmark item lookup failed", "err", err, "user", a.UserID, "item", itemID)
http.Error(w, "item lookup failed", http.StatusInternalServerError)
return
}
if item == nil {
if err != nil || item == nil {
http.Error(w, "item not found", http.StatusNotFound)
return
}
@@ -4,7 +4,6 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"sort"
@@ -158,14 +157,10 @@ func (p *recordingPublisher) snapshot() []publishedEvent {
// handler tests can drive both the 200 and 404 branches.
type stubMediaStore struct {
noopMediaStore
known map[string]*models.MediaItem // itemID → row (nil means "exists but no row needed")
lookupErr error
known map[string]*models.MediaItem // itemID → row (nil means "exists but no row needed")
}
func (s *stubMediaStore) GetAudiobookByID(_ context.Context, id string, _ catalog.AccessFilter) (*models.MediaItem, error) {
if s.lookupErr != nil {
return nil, s.lookupErr
}
if it, ok := s.known[id]; ok {
if it == nil {
return &models.MediaItem{ContentID: id}, nil
@@ -416,19 +411,6 @@ func TestMissingItem_404(t *testing.T) {
}
}
func TestItemLookup_ErrorFromMediaStore_500(t *testing.T) {
hb := newBookmarksHarness(t, "book-1")
hb.H.deps.MediaStore = &stubMediaStore{
known: map[string]*models.MediaItem{"book-1": nil},
lookupErr: errors.New("media lookup failed"),
}
rec := dispatchBookmark(hb.H, http.MethodPost, "/api/me/item/book-1/bookmark", "book-1", "", []byte(`{"title":"x","time":1}`), "1", "", hb.H.handleUpsertBookmark("bookmark_created"))
if rec.Code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500; body=%s", rec.Code, rec.Body.String())
}
}
func TestInvalidBody_400(t *testing.T) {
hb := newBookmarksHarness(t, "book-1")
rec := dispatchBookmark(hb.H, http.MethodPost, "/api/me/item/book-1/bookmark", "book-1", "", []byte(`{not json`), "1", "", hb.H.handleUpsertBookmark("bookmark_created"))
+3 -14
View File
@@ -6,7 +6,6 @@ import (
"io"
"log/slog"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/oklog/ulid/v2"
@@ -40,12 +39,7 @@ func (h *Handler) handleCreateCollection(w http.ResponseWriter, r *http.Request)
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
if body.Name == nil {
http.Error(w, "name required", http.StatusBadRequest)
return
}
name := strings.TrimSpace(*body.Name)
if name == "" {
if body.Name == nil || *body.Name == "" {
http.Error(w, "name required", http.StatusBadRequest)
return
}
@@ -54,7 +48,7 @@ func (h *Handler) handleCreateCollection(w http.ResponseWriter, r *http.Request)
ID: ulid.Make().String(),
UserID: a.UserID,
ProfileID: a.ProfileID,
Name: name,
Name: *body.Name,
}
if body.Description != nil {
c.Description = *body.Description
@@ -273,12 +267,7 @@ func (h *Handler) handleUpdateCollection(w http.ResponseWriter, r *http.Request)
return
}
if body.Name != nil {
name := strings.TrimSpace(*body.Name)
if name == "" {
http.Error(w, "name required", http.StatusBadRequest)
return
}
c.Name = name
c.Name = *body.Name
}
if body.Description != nil {
c.Description = *body.Description
@@ -25,7 +25,7 @@ import (
// and profile_id; we mirror that so List can filter correctly).
type memCollectionStore struct {
mu sync.Mutex
rows map[string]Collection // id -> row
rows map[string]Collection // id -> row
items map[string][]CollectionItem // collection_id -> items
}
@@ -281,16 +281,8 @@ func TestCollection_List_DoesNotLeakOtherUsers(t *testing.T) {
// User 2 lists.
rec := dispatchABSWithParams(http.MethodGet, "/api/collections", nil, nil, "2", "", hb.H.handleListCollections)
var env map[string]any
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if err := json.Unmarshal(rec.Body.Bytes(), &env); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
list, ok := env["collections"].([]any)
if !ok {
t.Fatalf("response missing 'collections' key; body=%s", rec.Body.String())
}
_ = json.Unmarshal(rec.Body.Bytes(), &env)
list, _ := env["collections"].([]any)
if len(list) != 0 {
t.Errorf("user 2 sees %d collections, want 0", len(list))
}
@@ -303,16 +295,8 @@ func TestCollection_List_ProfileIsolation(t *testing.T) {
_ = dispatchABSWithParams(http.MethodPost, "/api/collections", nil, []byte(`{"name":"A"}`), "1", pA, hb.H.handleCreateCollection)
rec := dispatchABSWithParams(http.MethodGet, "/api/collections", nil, nil, "1", pB, hb.H.handleListCollections)
var env map[string]any
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if err := json.Unmarshal(rec.Body.Bytes(), &env); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
list, ok := env["collections"].([]any)
if !ok {
t.Fatalf("response missing 'collections' key; body=%s", rec.Body.String())
}
_ = json.Unmarshal(rec.Body.Bytes(), &env)
list, _ := env["collections"].([]any)
if len(list) != 0 {
t.Errorf("profile B sees %d collections, want 0", len(list))
}
@@ -32,12 +32,7 @@ func (h *Handler) setHideFromContinue(w http.ResponseWriter, r *http.Request, hi
return
}
item, err := h.deps.MediaStore.GetAudiobookByID(r.Context(), itemID, access)
if err != nil {
slog.Error("abs continue item lookup failed", "err", err, "user", a.UserID, "item", itemID)
http.Error(w, "item lookup failed", http.StatusInternalServerError)
return
}
if item == nil {
if err != nil || item == nil {
http.Error(w, "item not found", http.StatusNotFound)
return
}
@@ -3,7 +3,6 @@ package abs
import (
"context"
"encoding/json"
"errors"
"net/http"
"sync"
"testing"
@@ -74,18 +73,3 @@ func TestContinue_UnknownItem_404(t *testing.T) {
t.Errorf("status = %d, want 404", rec.Code)
}
}
func TestContinue_MediaLookupError_500(t *testing.T) {
prog := &recordingProgressFake{}
media := &stubMediaStore{
known: map[string]*models.MediaItem{"book-1": nil},
lookupErr: errors.New("media lookup failed"),
}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
rec := dispatchABSWithParams(http.MethodGet, "/api/me/progress/book-1/remove-from-continue-listening",
map[string]string{"itemId": "book-1"}, nil, "1", "", h.handleRemoveFromContinueListening)
if rec.Code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500; body=%s", rec.Code, rec.Body.String())
}
}
+9 -13
View File
@@ -151,18 +151,14 @@ func (h *Handler) handleDeleteItemProgress(w http.ResponseWriter, r *http.Reques
// returns the empty progress shape so the client can store offline state
// without raising an error.
func (h *Handler) handleSetEpisodeProgress(w http.ResponseWriter, r *http.Request) {
if a, ok := absAuthFrom(r); !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
writeJSON(w, http.StatusOK, map[string]any{
"libraryItemId": chi.URLParam(r, "libraryItemId"),
"episodeId": chi.URLParam(r, "episodeId"),
"currentTime": 0.0,
"duration": 0.0,
"isFinished": false,
"progress": 0.0,
"lastUpdate": 0,
"libraryItemId": chi.URLParam(r, "libraryItemId"),
"episodeId": chi.URLParam(r, "episodeId"),
"currentTime": 0.0,
"duration": 0.0,
"isFinished": false,
"progress": 0.0,
"lastUpdate": 0,
})
}
@@ -186,8 +182,8 @@ func (h *Handler) handleEbookFile(w http.ResponseWriter, _ *http.Request) {
// optimistic update succeeds.
func (h *Handler) handleEbookStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{
"libraryItemId": chi.URLParam(r, "id"),
"fileId": chi.URLParam(r, "fileid"),
"libraryItemId": chi.URLParam(r, "id"),
"fileId": chi.URLParam(r, "fileid"),
"isSupplementary": false,
})
}
+4 -8
View File
@@ -13,20 +13,16 @@ import (
)
const (
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
absJWTSecretKey = "audiobooks.abs.jwt_secret"
absDefaultAccessTTL = 24 * time.Hour
absDefaultRefreshTTL = 30 * 24 * time.Hour
)
// ABSConfigProvider implements abs.ConfigProvider using silo's server_settings
// table. The ABS JWT secret is generated once on first read and persisted for
// the lifetime of the deployment.
type ABSConfigProvider struct {
// Settings is the encrypting settings decorator in production, so the ABS
// JWT secret (a SensitiveSettingKey) rests as ciphertext and is transparently
// decrypted here. Typed as the interface so both the raw repo and the
// decorator satisfy it.
Settings catalog.SettingsStore
Settings *catalog.ServerSettingsRepo
// secretCache holds the generated secret so we only hit the DB once per
// process lifetime. Protected by mu.
+8 -46
View File
@@ -44,8 +44,7 @@ const (
// defaultEnrichBatchSize is the maximum number of audiobook items processed
// per sweep invocation. Keeps latency bounded for large libraries.
// Override with SILO_AUDIOBOOK_ENRICH_BATCH_SIZE.
defaultEnrichBatchSize = 250
defaultEnrichBatchSize = 50
// defaultEnrichWorkers is the default fan-out used by Enricher.Run.
// Network-bound: each worker holds one provider HTTP call at a time, so
// 4 is enough to mask single-request latency without hammering plugins.
@@ -53,28 +52,18 @@ const (
defaultEnrichWorkers = 4
)
// audiobookEnrichBatchSize returns the configured maximum sweep size.
func audiobookEnrichBatchSize() int {
if v := os.Getenv("SILO_AUDIOBOOK_ENRICH_BATCH_SIZE"); v != "" {
if parsed, err := strconv.Atoi(v); err == nil && parsed > 0 {
return parsed
}
}
return defaultEnrichBatchSize
}
// audiobookEnrichWorkers returns the configured number of parallel enrichment
// workers, capped to the active batch size so workers never outnumber the batch
// they drain.
func audiobookEnrichWorkers(batchSize int) int {
// workers, capped to defaultEnrichBatchSize so workers never outnumber the
// batch they drain.
func audiobookEnrichWorkers() int {
n := defaultEnrichWorkers
if v := os.Getenv("SILO_AUDIOBOOK_ENRICH_WORKERS"); v != "" {
if parsed, err := strconv.Atoi(v); err == nil && parsed > 0 {
n = parsed
}
}
if batchSize > 0 && n > batchSize {
n = batchSize
if n > defaultEnrichBatchSize {
n = defaultEnrichBatchSize
}
return n
}
@@ -116,7 +105,6 @@ func NewEnricher(
personRepo *catalog.PersonRepository,
providerIDs *catalog.ProviderIDRepository,
) *Enricher {
batchSize := audiobookEnrichBatchSize()
return &Enricher{
pool: pool,
chainRepo: chainRepo,
@@ -124,8 +112,8 @@ func NewEnricher(
itemRepo: itemRepo,
personRepo: personRepo,
providerIDs: providerIDs,
batchSize: batchSize,
workers: audiobookEnrichWorkers(batchSize),
batchSize: defaultEnrichBatchSize,
workers: audiobookEnrichWorkers(),
}
}
@@ -178,32 +166,6 @@ func (e *Enricher) Run(ctx context.Context) (int, error) {
return enriched, nil
}
// HasPendingItems reports whether a scheduled sweep has any audiobook rows to
// process. It mirrors claimBatch's eligibility predicate without loading rows
// or provider IDs, so the scheduler can skip no-op executions without adding
// task-history noise.
func (e *Enricher) HasPendingItems(ctx context.Context) (bool, error) {
if e == nil || e.pool == nil || e.chainRepo == nil {
return false, nil
}
var exists bool
err := e.pool.QueryRow(ctx, `
SELECT EXISTS(
SELECT 1
FROM media_items mi
WHERE mi.type = 'audiobook'
AND (mi.poster_path IS NULL OR mi.poster_path = '')
AND mi.last_refreshed IS NULL
LIMIT 1
)
`).Scan(&exists)
if err != nil {
return false, fmt.Errorf("checking pending audiobook enrichment: %w", err)
}
return exists, nil
}
// runBatch processes items in parallel using e.workers goroutines. The
// enrichFn parameter exists for testing: production calls pass e.enrichItem.
// Returns the count of items where enrichFn returned nil.
-44
View File
@@ -63,50 +63,6 @@ func TestEnricherRunFansOut(t *testing.T) {
}
}
func TestNewEnricherUsesConfiguredBatchSize(t *testing.T) {
t.Setenv("SILO_AUDIOBOOK_ENRICH_BATCH_SIZE", "123")
t.Setenv("SILO_AUDIOBOOK_ENRICH_WORKERS", "8")
e := NewEnricher(nil, nil, nil, nil, nil, nil)
if e.batchSize != 123 {
t.Fatalf("batchSize = %d, want 123", e.batchSize)
}
if e.workers != 8 {
t.Fatalf("workers = %d, want 8", e.workers)
}
}
func TestNewEnricherCapsWorkersToConfiguredBatchSize(t *testing.T) {
t.Setenv("SILO_AUDIOBOOK_ENRICH_BATCH_SIZE", "50")
t.Setenv("SILO_AUDIOBOOK_ENRICH_WORKERS", "100")
e := NewEnricher(nil, nil, nil, nil, nil, nil)
if e.batchSize != 50 {
t.Fatalf("batchSize = %d, want 50", e.batchSize)
}
if e.workers != e.batchSize {
t.Fatalf("workers = %d, want capped batchSize %d", e.workers, e.batchSize)
}
}
func TestAudiobookEnrichWorkersCapsAtBatchSize(t *testing.T) {
t.Setenv("SILO_AUDIOBOOK_ENRICH_WORKERS", "8")
if got := audiobookEnrichWorkers(3); got != 3 {
t.Fatalf("audiobookEnrichWorkers(3) = %d, want 3", got)
}
}
func TestAudiobookEnrichBatchSizeIgnoresInvalidEnv(t *testing.T) {
t.Setenv("SILO_AUDIOBOOK_ENRICH_BATCH_SIZE", "nope")
if got := audiobookEnrichBatchSize(); got != defaultEnrichBatchSize {
t.Fatalf("audiobookEnrichBatchSize() = %d, want %d", got, defaultEnrichBatchSize)
}
}
func TestCacheRemotePosterCachesProviderURL(t *testing.T) {
cacher := &fakeAudiobookImageCacher{}
e := &Enricher{imageCacher: cacher}
+4 -22
View File
@@ -106,23 +106,6 @@ func (r *Refresher) WithHTTPClient(hc *http.Client) *Refresher {
return r
}
// HasDue reports whether RefreshDue would attempt at least one feed. The task
// scheduler uses this to avoid recording empty maintenance runs when no podcast
// feeds exist or every feed is still inside its refresh window.
func (r *Refresher) HasDue(ctx context.Context, s Store) (bool, error) {
feeds, err := s.ListPodcastFeeds(ctx)
if err != nil {
return false, fmt.Errorf("list podcast feeds: %w", err)
}
now := time.Now()
for _, f := range feeds {
if shouldRefreshFeed(f, now) {
return true, nil
}
}
return false, nil
}
// RefreshDue walks every podcast_feeds row and refreshes those whose
// last_refreshed_at + refresh_interval has elapsed. Per-feed failures are
// logged at Warn and do not abort the walk. Returns the count of feeds
@@ -135,7 +118,10 @@ func (r *Refresher) RefreshDue(ctx context.Context, s Store) (int, error) {
now := time.Now()
attempted := 0
for _, f := range feeds {
if !shouldRefreshFeed(f, now) {
if f.FeedURL == "" {
continue
}
if !isDue(f, now) {
continue
}
attempted++
@@ -150,10 +136,6 @@ func (r *Refresher) RefreshDue(ctx context.Context, s Store) (int, error) {
return attempted, nil
}
func shouldRefreshFeed(f PodcastFeed, now time.Time) bool {
return f.FeedURL != "" && isDue(f, now)
}
// RefreshOne refreshes a single podcast feed. Public so the admin
// force-refresh endpoint can call it directly. Records success or failure
// in podcast_feeds.last_refresh_error regardless of outcome.
+14 -11
View File
@@ -21,8 +21,10 @@ type SettingsReader interface {
GetString(ctx context.Context, key string) (string, error)
}
// Service owns the Audiobookshelf compatibility adapters. Native audiobook
// libraries are normal catalog media_items and do not depend on this service.
// Service is the audiobooks feature's top-level orchestrator. Sub-plan 1
// exposes only Enabled(); subsequent sub-plans hang additional methods
// off Service as new capabilities (scanner branches, ABS handlers, etc.)
// come online.
type Service struct {
settings SettingsReader
ABSHandler *abs.Handler
@@ -42,8 +44,8 @@ type ABSHandlerDeps struct {
Pool *pgxpool.Pool
Items *catalog.ItemRepository
Files *scanner.FileRepository
Settings catalog.SettingsStore // encrypting decorator in production
Auth absAuthAdapter // see BuildABSHandler below
Settings *catalog.ServerSettingsRepo
Auth absAuthAdapter // see BuildABSHandler below
AccessResolver abs.AccessResolver
// Recs is the recommendations repository used to power the
// /items/{id}/similar endpoint via embedding nearest-neighbor search.
@@ -162,7 +164,7 @@ func (s *Service) BuildABSHandler(deps ABSHandlerDeps) *abs.Handler {
if deps.Detail == nil {
return ""
}
return deps.Detail.PresignImageURL(ctx, path, "poster", variant)
return deps.Detail.PresignURL(ctx, path, variant)
},
})
s.ABSHandler = h
@@ -179,16 +181,17 @@ func buildABSRecommender(deps ABSHandlerDeps) abs.Recommender {
return &ABSRecommender{Pool: deps.Pool, Recs: deps.Recs}
}
// ABSCompatEnabled reports whether the Audiobookshelf compatibility listener
// should be started. Native audiobook libraries intentionally do not read this
// setting.
func (s *Service) ABSCompatEnabled(ctx context.Context) (bool, error) {
// Enabled reports whether the audiobooks feature flag (set by
// 160_audiobooks_feature_flag and toggled by operators) is currently true.
// Any value other than the literal string "true" reads as false; this matches how silo
// treats other boolean server_settings rows.
func (s *Service) Enabled(ctx context.Context) (bool, error) {
if s == nil || s.settings == nil {
return false, nil
}
value, err := s.settings.GetString(ctx, "audiobookshelf_compat.enabled")
value, err := s.settings.GetString(ctx, "audiobooks.enabled")
if err != nil {
return false, fmt.Errorf("read audiobookshelf_compat.enabled: %w", err)
return false, fmt.Errorf("read audiobooks.enabled: %w", err)
}
return value == "true", nil
}
+16 -70
View File
@@ -3,14 +3,7 @@ package audiobooks
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"unsafe"
"github.com/Silo-Server/silo-server/internal/audiobooks/abs"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/scanner"
)
type fakeSettingsReader struct {
@@ -19,13 +12,13 @@ type fakeSettingsReader struct {
}
func (f *fakeSettingsReader) GetString(_ context.Context, key string) (string, error) {
if key != "audiobookshelf_compat.enabled" {
if key != "audiobooks.enabled" {
return "", errors.New("unexpected key: " + key)
}
return f.value, f.err
}
func TestServiceABSCompatEnabledReadsFlag(t *testing.T) {
func TestServiceEnabledReadsFlag(t *testing.T) {
cases := []struct {
name string
stored string
@@ -40,91 +33,44 @@ func TestServiceABSCompatEnabledReadsFlag(t *testing.T) {
tc := tc
t.Run(tc.name, func(t *testing.T) {
svc := New(&fakeSettingsReader{value: tc.stored})
got, err := svc.ABSCompatEnabled(context.Background())
got, err := svc.Enabled(context.Background())
if err != nil {
t.Fatalf("ABSCompatEnabled returned error: %v", err)
t.Fatalf("Enabled returned error: %v", err)
}
if got != tc.want {
t.Fatalf("ABSCompatEnabled = %v, want %v", got, tc.want)
t.Fatalf("Enabled = %v, want %v", got, tc.want)
}
})
}
}
func TestServiceABSCompatEnabledPropagatesError(t *testing.T) {
func TestServiceEnabledPropagatesError(t *testing.T) {
wantErr := errors.New("db down")
svc := New(&fakeSettingsReader{err: wantErr})
_, err := svc.ABSCompatEnabled(context.Background())
_, err := svc.Enabled(context.Background())
if !errors.Is(err, wantErr) {
t.Fatalf("ABSCompatEnabled error = %v, want %v wrapped", err, wantErr)
t.Fatalf("Enabled error = %v, want %v wrapped", err, wantErr)
}
}
func TestServiceABSCompatEnabledNilReceiverReturnsFalse(t *testing.T) {
func TestServiceEnabledNilReceiverReturnsFalse(t *testing.T) {
var svc *Service
got, err := svc.ABSCompatEnabled(context.Background())
got, err := svc.Enabled(context.Background())
if err != nil {
t.Fatalf("ABSCompatEnabled returned error: %v", err)
t.Fatalf("Enabled returned error: %v", err)
}
if got {
t.Fatal("ABSCompatEnabled = true, want false")
t.Fatal("Enabled = true, want false")
}
}
func TestServiceABSCompatEnabledNilSettingsReturnsFalse(t *testing.T) {
func TestServiceEnabledNilSettingsReturnsFalse(t *testing.T) {
svc := New(nil)
got, err := svc.ABSCompatEnabled(context.Background())
got, err := svc.Enabled(context.Background())
if err != nil {
t.Fatalf("ABSCompatEnabled returned error: %v", err)
t.Fatalf("Enabled returned error: %v", err)
}
if got {
t.Fatal("ABSCompatEnabled = true, want false")
t.Fatal("Enabled = true, want false")
}
}
func TestBuildABSHandlerCoverResolverUsesPosterVariant(t *testing.T) {
resolver := &recordingImageResolver{}
detail := &catalog.DetailService{}
detail.SetImageResolver(resolver)
handler := New(nil).BuildABSHandler(ABSHandlerDeps{
Items: &catalog.ItemRepository{},
Files: &scanner.FileRepository{},
Detail: detail,
})
coverResolver := absCoverResolverForTest(t, handler)
got := coverResolver(context.Background(), "local/audiobooks/book-1/poster/original.webp", "card")
if !strings.Contains(got, "/w500.webp") {
t.Fatalf("resolved URL = %q, want w500 poster variant", got)
}
if resolver.variant != "featured" {
t.Fatalf("resolver variant = %q, want featured", resolver.variant)
}
}
func absCoverResolverForTest(t *testing.T, handler *abs.Handler) func(context.Context, string, string) string {
t.Helper()
field := reflect.ValueOf(handler).Elem().FieldByName("deps").FieldByName("CoverResolver")
return reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).Elem().Interface().(func(context.Context, string, string) string)
}
type recordingImageResolver struct {
path string
variant string
}
func (r *recordingImageResolver) ResolveImageURL(_ context.Context, path string, variant string) string {
r.path = path
r.variant = variant
return "resolved://" + path
}
func (r *recordingImageResolver) ResolveImageURLs(_ context.Context, paths []string, variant string) map[string]string {
out := make(map[string]string, len(paths))
for _, path := range paths {
out[path] = r.ResolveImageURL(context.Background(), path, variant)
}
return out
}
+27 -13
View File
@@ -13,27 +13,31 @@ type ResolvedConnection struct {
}
// RequestIntegrationLookup resolves a soft-linked Requests integration to its
// base URL and (already-decrypted) api key.
// base URL and encrypted api-key reference.
type RequestIntegrationLookup interface {
Get(ctx context.Context, integrationID string) (baseURL, apiKey string, err error)
Get(ctx context.Context, integrationID string) (baseURL, apiKeyRef string, err error)
}
// SecretResolver resolves an encrypted api-key reference to its plaintext value.
type SecretResolver interface {
Get(ctx context.Context, ref string) (string, error)
}
type ConnectionResolver struct {
requests RequestIntegrationLookup
secrets SecretResolver
}
func NewConnectionResolver(r RequestIntegrationLookup) *ConnectionResolver {
return &ConnectionResolver{requests: r}
func NewConnectionResolver(r RequestIntegrationLookup, s SecretResolver) *ConnectionResolver {
return &ConnectionResolver{requests: r, secrets: s}
}
// Resolve turns a stored Connection into concrete credentials. When the
// connection is linked to a Requests integration the live base URL + key are
// read from there (the Requests repo decrypts it); otherwise the connection's
// own fields are used (the autoscan repo already decrypted api_key_ref on read).
// Either way the returned key is plaintext — there is no longer a separate
// secret-resolution step.
// connection is linked to a Requests integration the live base URL + key ref are
// read from there; otherwise the connection's own fields are used. The api-key
// ref is then resolved to plaintext via the secrets resolver.
func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (ResolvedConnection, error) {
baseURL, apiKey := c.BaseURL, c.APIKeyRef
baseURL, apiKeyRef := c.BaseURL, c.APIKeyRef
// A pointer-to-empty-string request_integration_id is NOT a live link (it can
// arise from a both-NULL orphan or a stripped link). Treat empty/whitespace as
// "no link" so we don't call requests.Get("") and use the connection's own
@@ -42,11 +46,21 @@ func (cr *ConnectionResolver) Resolve(ctx context.Context, c Connection) (Resolv
if cr.requests == nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: no lookup configured", *c.RequestIntegrationID)
}
u, key, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
u, ref, err := cr.requests.Get(ctx, *c.RequestIntegrationID)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: linked requests integration %q: %w", *c.RequestIntegrationID, err)
}
baseURL, apiKey = u, key
baseURL, apiKeyRef = u, ref
}
return ResolvedConnection{BaseURL: baseURL, APIKey: strings.TrimSpace(apiKey)}, nil
apiKey := strings.TrimSpace(apiKeyRef)
if cr.secrets != nil && apiKey != "" {
resolved, err := cr.secrets.Get(ctx, apiKey)
if err != nil {
return ResolvedConnection{}, fmt.Errorf("autoscan: resolve api key: %w", err)
}
if resolved = strings.TrimSpace(resolved); resolved != "" {
apiKey = resolved
}
}
return ResolvedConnection{BaseURL: baseURL, APIKey: apiKey}, nil
}
+36 -20
View File
@@ -6,10 +6,9 @@ import (
"testing"
)
// fakeRequestLookup implements RequestIntegrationLookup from a static map. The
// returned apiKey is already plaintext (the Requests repo decrypts on read).
// fakeRequestLookup implements RequestIntegrationLookup from a static map.
type fakeRequestLookup struct {
entries map[string]struct{ baseURL, apiKey string }
entries map[string]struct{ baseURL, apiKeyRef string }
err error
}
@@ -21,17 +20,30 @@ func (f fakeRequestLookup) Get(_ context.Context, id string) (string, string, er
if !ok {
return "", "", errors.New("integration not found: " + id)
}
return e.baseURL, e.apiKey, nil
return e.baseURL, e.apiKeyRef, nil
}
// fakeSecrets implements SecretResolver from a static map; unknown refs resolve
// to empty (so the ref itself is used).
type fakeSecrets struct {
values map[string]string
err error
}
func (f fakeSecrets) Get(_ context.Context, ref string) (string, error) {
if f.err != nil {
return "", f.err
}
return f.values[ref], nil
}
func TestResolveConnectionOwnCredentials(t *testing.T) {
// The autoscan repo already decrypted api_key_ref, so the connection carries
// the literal key; Resolve returns it verbatim (trimmed).
r := NewConnectionResolver(fakeRequestLookup{})
secrets := fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "OWNKEY",
APIKeyRef: "ownref",
})
if err != nil {
t.Fatalf("Resolve own: %v", err)
@@ -42,10 +54,11 @@ func TestResolveConnectionOwnCredentials(t *testing.T) {
}
func TestResolveConnectionLinked(t *testing.T) {
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKey string }{
"req-1": {baseURL: "http://req:7878", apiKey: "REQKEY"},
lookup := fakeRequestLookup{entries: map[string]struct{ baseURL, apiKeyRef string }{
"req-1": {baseURL: "http://req:7878", apiKeyRef: "rk"},
}}
r := NewConnectionResolver(lookup)
secrets := fakeSecrets{values: map[string]string{"rk": "REQKEY"}}
r := NewConnectionResolver(lookup, secrets)
id := "req-1"
got, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id})
@@ -58,7 +71,7 @@ func TestResolveConnectionLinked(t *testing.T) {
}
func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{})
r := NewConnectionResolver(fakeRequestLookup{}, fakeSecrets{})
id := "gone"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when linked Requests integration is missing")
@@ -66,7 +79,7 @@ func TestResolveConnectionLinkedMissingErrors(t *testing.T) {
}
func TestResolveConnectionLinkedLookupErrors(t *testing.T) {
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")})
r := NewConnectionResolver(fakeRequestLookup{err: errors.New("boom")}, fakeSecrets{})
id := "req-1"
if _, err := r.Resolve(context.Background(), Connection{RequestIntegrationID: &id}); err == nil {
t.Fatal("expected error when the lookup itself fails")
@@ -79,12 +92,13 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
// requests.Get(""). The lookup here errors on any call, so reaching it fails.
r := NewConnectionResolver(
fakeRequestLookup{err: errors.New("lookup must not be called")},
fakeSecrets{values: map[string]string{"ownref": "OWNKEY"}},
)
for _, empty := range []string{"", " "} {
id := empty
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: "OWNKEY",
APIKeyRef: "ownref",
RequestIntegrationID: &id,
})
if err != nil {
@@ -96,18 +110,20 @@ func TestResolveConnectionEmptyIntegrationIDIsNotALink(t *testing.T) {
}
}
func TestResolveConnectionTrimsKey(t *testing.T) {
// Resolve trims surrounding whitespace from the (decrypted) key.
r := NewConnectionResolver(fakeRequestLookup{})
func TestResolveConnectionTrimsAndFallsBackOnEmptySecret(t *testing.T) {
// Whitespace-padded ref; secret resolves to whitespace-only -> fall back to
// the trimmed ref (parity with requests.resolveAPIKey).
secrets := fakeSecrets{values: map[string]string{"ownref": " "}}
r := NewConnectionResolver(fakeRequestLookup{}, secrets)
got, err := r.Resolve(context.Background(), Connection{
BaseURL: "http://own:8989",
APIKeyRef: " OWNKEY ",
APIKeyRef: " ownref ",
})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if got.APIKey != "OWNKEY" {
t.Fatalf("expected trimmed key, got %q", got.APIKey)
if got.APIKey != "ownref" {
t.Fatalf("expected trimmed ref fallback, got %q", got.APIKey)
}
}
+39 -9
View File
@@ -9,17 +9,19 @@ import (
// enriched with the metadata the Add-source picker needs (plugin id + a
// human-friendly display name).
type DiscoveredSource struct {
InstallationID int
CapabilityID string
// PluginID is the installation's plugin id (e.g. "sonarr"); empty when the
// lister cannot supply it.
PluginID string
CapabilityID string
PluginID string
// DisplayName is a human-friendly label for the capability (from the
// capability's manifest display_name, falling back to plugin/capability ids).
DisplayName string
}
// ScanSourceLister enumerates every installed scan_source.v1 capability so the
// engine can offer them in the Add-source picker.
// engine can (a) offer them in the Add-source picker and (b) detect orphaned
// source rows whose plugin has been uninstalled.
type ScanSourceLister interface {
// ListScanSources returns one entry per installed scan_source.v1 capability,
// enriched with plugin id + display name.
@@ -29,9 +31,17 @@ type ScanSourceLister interface {
// AvailableScanSource is one installed scan_source capability an operator can
// create a source against (the Add-source picker list).
type AvailableScanSource struct {
PluginID string `json:"plugin_id"`
CapabilityID string `json:"capability_id"`
DisplayName string `json:"display_name"`
InstallationID int `json:"installation_id"`
CapabilityID string `json:"capability_id"`
PluginID string `json:"plugin_id"`
DisplayName string `json:"display_name"`
}
// installedKey identifies an installed scan_source capability for set
// membership tests (orphan detection in PollOnce).
type installedKey struct {
InstallationID int
CapabilityID string
}
// ListAvailableScanSources enumerates every installed scan_source capability so
@@ -49,10 +59,30 @@ func (s *Service) ListAvailableScanSources(ctx context.Context) ([]AvailableScan
out := make([]AvailableScanSource, 0, len(discovered))
for _, d := range discovered {
out = append(out, AvailableScanSource{
PluginID: d.PluginID,
CapabilityID: d.CapabilityID,
DisplayName: d.DisplayName,
InstallationID: d.InstallationID,
CapabilityID: d.CapabilityID,
PluginID: d.PluginID,
DisplayName: d.DisplayName,
})
}
return out, nil
}
// installedScanSources returns the set of currently-installed scan_source
// capabilities so PollOnce can skip orphaned source rows (their plugin is
// gone). A nil set means the set is unavailable (no lister) — callers must treat
// that as "discovery unavailable" and NOT prune.
func (s *Service) installedScanSources(ctx context.Context) (map[installedKey]struct{}, error) {
if s.lister == nil {
return nil, nil
}
discovered, err := s.lister.ListScanSources(ctx)
if err != nil {
return nil, fmt.Errorf("list scan sources: %w", err)
}
present := make(map[installedKey]struct{}, len(discovered))
for _, d := range discovered {
present[installedKey{InstallationID: d.InstallationID, CapabilityID: d.CapabilityID}] = struct{}{}
}
return present, nil
}
+32 -3
View File
@@ -14,8 +14,8 @@ func (f fakeLister) ListScanSources(context.Context) ([]DiscoveredSource, error)
func TestListAvailableScanSourcesEnumeratesInstalled(t *testing.T) {
lister := fakeLister{sources: []DiscoveredSource{
{PluginID: "sonarr", CapabilityID: "arr-a", DisplayName: "Sonarr"},
{PluginID: "radarr", CapabilityID: "arr-b", DisplayName: "Radarr"},
{InstallationID: 1, CapabilityID: "arr-a", PluginID: "sonarr", DisplayName: "Sonarr"},
{InstallationID: 2, CapabilityID: "arr-b", PluginID: "radarr", DisplayName: "Radarr"},
}}
svc := &Service{lister: lister}
@@ -26,7 +26,7 @@ func TestListAvailableScanSourcesEnumeratesInstalled(t *testing.T) {
if len(available) != 2 {
t.Fatalf("expected 2 available, got %d: %+v", len(available), available)
}
if available[0] != (AvailableScanSource{PluginID: "sonarr", CapabilityID: "arr-a", DisplayName: "Sonarr"}) {
if available[0] != (AvailableScanSource{InstallationID: 1, CapabilityID: "arr-a", PluginID: "sonarr", DisplayName: "Sonarr"}) {
t.Fatalf("unexpected first available: %+v", available[0])
}
}
@@ -41,3 +41,32 @@ func TestListAvailableScanSourcesNilListerEmpty(t *testing.T) {
t.Fatalf("nil lister must return empty, got %+v", available)
}
}
func TestInstalledScanSourcesSetMembership(t *testing.T) {
lister := fakeLister{sources: []DiscoveredSource{
{InstallationID: 1, CapabilityID: "arr-a"},
}}
svc := &Service{lister: lister}
present, err := svc.installedScanSources(context.Background())
if err != nil {
t.Fatalf("installedScanSources: %v", err)
}
if _, ok := present[installedKey{1, "arr-a"}]; !ok {
t.Fatalf("expected 1/arr-a present: %+v", present)
}
if _, ok := present[installedKey{2, "arr-b"}]; ok {
t.Fatalf("did not expect 2/arr-b present: %+v", present)
}
}
func TestInstalledScanSourcesNilListerNilSet(t *testing.T) {
svc := &Service{lister: nil}
present, err := svc.installedScanSources(context.Background())
if err != nil {
t.Fatalf("installedScanSources: %v", err)
}
if present != nil {
t.Fatalf("nil lister must return a nil set, got %+v", present)
}
}
-4
View File
@@ -5,7 +5,3 @@ import "errors"
// ErrNotFound is returned when an autoscan operation references a connection or
// source that does not exist. The admin API surfaces it as a 404.
var ErrNotFound = errors.New("autoscan: not found")
// ErrPollAlreadyRunning is returned when a source already has an active poll
// event. Callers should treat it as a no-op skip, not a source failure.
var ErrPollAlreadyRunning = errors.New("autoscan: source poll already running")
+2 -2
View File
@@ -69,7 +69,7 @@ func TestTestConnectionNoProbeConfigured(t *testing.T) {
func TestSuggestRewritesMatchesArrRootsToSiloFolders(t *testing.T) {
store := &fakeStore{
sources: []Source{
{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1")},
{ID: "src-1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1")},
},
}
arr := &fakeArrClient{roots: []string{"/data/tv"}}
@@ -87,7 +87,7 @@ func TestSuggestRewritesMatchesArrRootsToSiloFolders(t *testing.T) {
func TestSuggestRewritesNoConnection(t *testing.T) {
store := &fakeStore{
sources: []Source{{ID: "src-1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil}},
sources: []Source{{ID: "src-1", InstallationID: 1, CapabilityID: "arr", ConnectionID: nil}},
}
svc := &Service{store: store, connres: passthroughConnRes{}, rootFolders: &fakeArrClient{}, folders: fakeFolderLister{}}
+5 -5
View File
@@ -9,7 +9,7 @@ import (
// ScanSourceProvider yields changed paths for one source. The engine calls
// PollChanges; production wraps the plugins.Service scan_source resolver.
type ScanSourceProvider interface {
PollChanges(ctx context.Context, pluginID, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) (changes []Change, nextMarker string, err error)
PollChanges(ctx context.Context, installationID int, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) (changes []Change, nextMarker string, err error)
}
// PollChangesClient is the slice of *pluginhost.ScanSourceClient used here. It
@@ -20,10 +20,10 @@ type PollChangesClient interface {
PollChanges(ctx context.Context, req *pluginv1.PollChangesRequest) (*pluginv1.PollChangesResponse, error)
}
// ScanSourceResolver yields a per-(plugin, capability) scan-source client.
// ScanSourceResolver yields a per-(installation, capability) scan-source client.
// Exported for the same cross-package adapter reason as PollChangesClient.
type ScanSourceResolver interface {
ScanSourceClient(ctx context.Context, pluginID, capabilityID string) (PollChangesClient, error)
ScanSourceClient(ctx context.Context, installationID int, capabilityID string) (PollChangesClient, error)
}
type pluginProvider struct{ resolver ScanSourceResolver }
@@ -34,8 +34,8 @@ func NewPluginProvider(resolver ScanSourceResolver) ScanSourceProvider {
return &pluginProvider{resolver: resolver}
}
func (p *pluginProvider) PollChanges(ctx context.Context, pluginID, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
client, err := p.resolver.ScanSourceClient(ctx, pluginID, capabilityID)
func (p *pluginProvider) PollChanges(ctx context.Context, installationID int, capabilityID, marker string, conn ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
client, err := p.resolver.ScanSourceClient(ctx, installationID, capabilityID)
if err != nil {
return nil, "", err
}
+7 -18
View File
@@ -17,17 +17,10 @@ func (c *capturingClient) PollChanges(_ context.Context, req *pluginv1.PollChang
return &pluginv1.PollChangesResponse{SourcePaths: []string{"/mnt/media/x"}, NextMarker: "m2"}, nil
}
// capturingResolver yields a fixed PollChangesClient and records the requested
// stable scan-source identity.
type capturingResolver struct {
client PollChangesClient
lastPluginID string
lastCapID string
}
// capturingResolver yields a fixed PollChangesClient.
type capturingResolver struct{ client PollChangesClient }
func (r *capturingResolver) ScanSourceClient(_ context.Context, pluginID, capabilityID string) (PollChangesClient, error) {
r.lastPluginID = pluginID
r.lastCapID = capabilityID
func (r capturingResolver) ScanSourceClient(context.Context, int, string) (PollChangesClient, error) {
return r.client, nil
}
@@ -35,12 +28,11 @@ func (r *capturingResolver) ScanSourceClient(_ context.Context, pluginID, capabi
// delivered to the plugin on the PollChangesRequest.
func TestPluginProviderPopulatesConnection(t *testing.T) {
client := &capturingClient{}
resolver := &capturingResolver{client: client}
prov := NewPluginProvider(resolver)
prov := NewPluginProvider(capturingResolver{client})
conn := ResolvedConnection{BaseURL: "https://arr.example", APIKey: "secret-key"}
sourceConfig := map[string]string{"exclusions": ".downloads"}
changes, next, err := prov.PollChanges(context.Background(), "silo.autoscan.arr", "cap", "m1", conn, sourceConfig)
changes, next, err := prov.PollChanges(context.Background(), 1, "cap", "m1", conn, sourceConfig)
if err != nil {
t.Fatalf("PollChanges: %v", err)
}
@@ -53,9 +45,6 @@ func TestPluginProviderPopulatesConnection(t *testing.T) {
if client.last.GetCapabilityId() != "cap" || client.last.GetMarker() != "m1" {
t.Fatalf("unexpected request fields: cap=%q marker=%q", client.last.GetCapabilityId(), client.last.GetMarker())
}
if resolver.lastPluginID != "silo.autoscan.arr" || resolver.lastCapID != "cap" {
t.Fatalf("resolver identity = %q/%q", resolver.lastPluginID, resolver.lastCapID)
}
rc := client.last.GetConnection()
if rc == nil {
t.Fatal("expected connection to be populated on the request")
@@ -70,9 +59,9 @@ func TestPluginProviderPopulatesConnection(t *testing.T) {
func TestPluginProviderPrefersStructuredChanges(t *testing.T) {
client := &capturingStructuredClient{}
prov := NewPluginProvider(&capturingResolver{client: client})
prov := NewPluginProvider(capturingResolver{client: client})
changes, next, err := prov.PollChanges(context.Background(), "silo.autoscan.arr", "cap", "m1", ResolvedConnection{}, nil)
changes, next, err := prov.PollChanges(context.Background(), 1, "cap", "m1", ResolvedConnection{}, nil)
if err != nil {
t.Fatalf("PollChanges: %v", err)
}
+37 -181
View File
@@ -9,38 +9,14 @@ import (
"time"
"unicode/utf8"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
type Repository struct {
pool *pgxpool.Pool
cipher *secret.Cipher
}
type Repository struct{ pool *pgxpool.Pool }
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// connectionAPIKeyAAD binds an autoscan_connections api_key_ref ciphertext to
// its row id.
func connectionAPIKeyAAD(id string) string {
return secret.RowAAD("autoscan_connections", "api_key_ref", id)
}
// encryptAPIKey encrypts a non-empty, trimmed key bound to the connection id;
// an empty key returns "" so NULL/keep-existing semantics are preserved.
func (r *Repository) encryptAPIKey(id, apiKey string) (string, error) {
apiKey = strings.TrimSpace(apiKey)
if apiKey == "" {
return "", nil
}
return r.cipher.Encrypt(apiKey, connectionAPIKeyAAD(id))
}
func NewRepository(pool *pgxpool.Pool) *Repository { return &Repository{pool: pool} }
// --- Settings ---
@@ -88,7 +64,7 @@ func (r *Repository) UpdateSettings(ctx context.Context, s Settings) (Settings,
const connectionColumns = `id, name, kind, base_url, api_key_ref, request_integration_id`
func (r *Repository) scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
func scanConnection(row interface{ Scan(...any) error }) (Connection, error) {
var c Connection
var baseURL, apiKeyRef, reqIntegrationID *string
if err := row.Scan(&c.ID, &c.Name, &c.Kind, &baseURL, &apiKeyRef, &reqIntegrationID); err != nil {
@@ -101,13 +77,6 @@ func (r *Repository) scanConnection(row interface{ Scan(...any) error }) (Connec
c.APIKeyRef = *apiKeyRef
}
c.RequestIntegrationID = reqIntegrationID
// Decrypt the stored key (read-path contract): legacy plaintext passes
// through, enc:v1: decrypts, corrupt ciphertext errors.
apiKey, err := r.cipher.DecryptIfEncrypted(c.APIKeyRef, connectionAPIKeyAAD(c.ID))
if err != nil {
return Connection{}, fmt.Errorf("decrypt autoscan connection %s api key: %w", c.ID, err)
}
c.APIKeyRef = apiKey
return c, nil
}
@@ -120,22 +89,12 @@ func nullable(s string) any {
}
func (r *Repository) CreateConnection(ctx context.Context, c Connection) (Connection, error) {
// Generate the id in Go (rather than the DB default) so the api_key_ref
// ciphertext can be AAD-bound to the row id at insert time.
id := strings.TrimSpace(c.ID)
if id == "" {
id = uuid.NewString()
}
apiKeyRef, err := r.encryptAPIKey(id, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_connections (id, name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5, $6)
INSERT INTO autoscan_connections (name, kind, base_url, api_key_ref, request_integration_id)
VALUES ($1, $2, $3, $4, $5)
RETURNING `+connectionColumns,
id, c.Name, c.Kind, nullable(c.BaseURL), nullable(apiKeyRef), c.RequestIntegrationID)
out, err := r.scanConnection(row)
c.Name, c.Kind, nullable(c.BaseURL), nullable(c.APIKeyRef), c.RequestIntegrationID)
out, err := scanConnection(row)
if err != nil {
return Connection{}, fmt.Errorf("create autoscan connection: %w", err)
}
@@ -149,11 +108,6 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
// next poll. Mirrors requests.UpdateIntegration's CASE-WHEN keep-semantics.
// Pass the raw trimmed string (not nullable()) so the empty-string sentinel
// reaches the CASE.
// Encrypt the incoming key; an empty result preserves the keep-existing CASE.
apiKeyRef, err := r.encryptAPIKey(c.ID, c.APIKeyRef)
if err != nil {
return Connection{}, fmt.Errorf("encrypt autoscan api key: %w", err)
}
row := r.pool.QueryRow(ctx, `
UPDATE autoscan_connections
SET name = $2, kind = $3, base_url = $4,
@@ -161,8 +115,8 @@ func (r *Repository) UpdateConnection(ctx context.Context, c Connection) (Connec
request_integration_id = $6, updated_at = now()
WHERE id = $1
RETURNING `+connectionColumns,
c.ID, c.Name, c.Kind, nullable(c.BaseURL), apiKeyRef, c.RequestIntegrationID)
out, err := r.scanConnection(row)
c.ID, c.Name, c.Kind, nullable(c.BaseURL), c.APIKeyRef, c.RequestIntegrationID)
out, err := scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, c.ID)
@@ -197,7 +151,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
defer rows.Close()
var out []Connection
for rows.Next() {
c, err := r.scanConnection(rows)
c, err := scanConnection(rows)
if err != nil {
return nil, err
}
@@ -209,7 +163,7 @@ func (r *Repository) ListConnections(ctx context.Context) ([]Connection, error)
func (r *Repository) GetConnection(ctx context.Context, id string) (Connection, error) {
row := r.pool.QueryRow(ctx, `SELECT `+connectionColumns+`
FROM autoscan_connections WHERE id = $1`, id)
c, err := r.scanConnection(row)
c, err := scanConnection(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return Connection{}, fmt.Errorf("%w: connection %s", ErrNotFound, id)
@@ -221,14 +175,14 @@ func (r *Repository) GetConnection(ctx context.Context, id string) (Connection,
// --- Sources ---
const sourceColumns = `id, plugin_id, capability_id, connection_id, enabled,
const sourceColumns = `id, installation_id, capability_id, connection_id, enabled,
poll_interval_seconds, path_rewrites, source_config, label, marker, last_run_at, last_error`
func scanSource(row interface{ Scan(...any) error }) (Source, error) {
var s Source
var pathRewrites []byte
var sourceConfig []byte
if err := row.Scan(&s.ID, &s.PluginID, &s.CapabilityID, &s.ConnectionID,
if err := row.Scan(&s.ID, &s.InstallationID, &s.CapabilityID, &s.ConnectionID,
&s.Enabled, &s.PollIntervalSeconds, &pathRewrites, &sourceConfig, &s.Label, &s.Marker, &s.LastRunAt, &s.LastError); err != nil {
return Source{}, err
}
@@ -329,11 +283,11 @@ func (r *Repository) CreateSource(ctx context.Context, s Source) (Source, error)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_sources (
plugin_id, capability_id, connection_id, enabled, poll_interval_seconds, path_rewrites, source_config, label
installation_id, capability_id, connection_id, enabled, poll_interval_seconds, path_rewrites, source_config, label
)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING `+sourceColumns,
s.PluginID, s.CapabilityID, connectionIDArg(s.ConnectionID), s.Enabled, s.PollIntervalSeconds, rewrites, sourceConfig, s.Label)
s.InstallationID, s.CapabilityID, connectionIDArg(s.ConnectionID), s.Enabled, s.PollIntervalSeconds, rewrites, sourceConfig, s.Label)
out, err := scanSource(row)
if err != nil {
if connID, ok := connectionFKViolation(err, s.ConnectionID); ok {
@@ -345,7 +299,7 @@ func (r *Repository) CreateSource(ctx context.Context, s Source) (Source, error)
}
// UpdateSource updates a source's binding/scheduling fields by id. Identity
// (plugin_id, capability_id) and bookkeeping fields (marker/last_run_at/
// (installation_id, capability_id) and bookkeeping fields (marker/last_run_at/
// last_error) are left untouched. An unknown id maps to ErrNotFound; a
// non-existent connection trips the FK constraint and also maps to ErrNotFound.
func (r *Repository) UpdateSource(ctx context.Context, s Source) (Source, error) {
@@ -399,7 +353,7 @@ func connectionFKViolation(err error, connectionID *string) (string, bool) {
func (r *Repository) ListSources(ctx context.Context) ([]Source, error) {
rows, err := r.pool.Query(ctx, `SELECT `+sourceColumns+`
FROM autoscan_sources ORDER BY plugin_id, capability_id`)
FROM autoscan_sources ORDER BY installation_id, capability_id`)
if err != nil {
return nil, fmt.Errorf("list autoscan sources: %w", err)
}
@@ -410,7 +364,7 @@ func (r *Repository) ListSources(ctx context.Context) ([]Source, error) {
func (r *Repository) ListEnabledSources(ctx context.Context) ([]Source, error) {
rows, err := r.pool.Query(ctx, `SELECT `+sourceColumns+`
FROM autoscan_sources WHERE enabled = true
ORDER BY plugin_id, capability_id`)
ORDER BY installation_id, capability_id`)
if err != nil {
return nil, fmt.Errorf("list enabled autoscan sources: %w", err)
}
@@ -508,7 +462,7 @@ func (r *Repository) RecordError(ctx context.Context, sourceID, msg string) erro
return nil
}
const eventColumns = `id, source_id, plugin_id, capability_id, started_at, completed_at,
const eventColumns = `id, source_id, installation_id, capability_id, started_at, completed_at,
duration_ms, status, changes_returned, changes_resolved, targets_claimed, scans_created,
scans_reused, scans_suppressed, error_message, marker_before, marker_after`
@@ -518,7 +472,7 @@ func scanEvent(row interface{ Scan(...any) error }) (Event, error) {
if err := row.Scan(
&e.ID,
&e.SourceID,
&e.PluginID,
&e.InstallationID,
&e.CapabilityID,
&e.StartedAt,
&e.CompletedAt,
@@ -545,80 +499,25 @@ func (r *Repository) CreateEvent(ctx context.Context, in EventCreate) (int64, er
if started.IsZero() {
started = time.Now()
}
sourceID := strings.TrimSpace(in.SourceID)
if sourceID == "" {
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_events (
source_id, plugin_id, capability_id, started_at, completed_at,
duration_ms, status, error_message, marker_before
)
VALUES ($1, $2, $3, $4, $4, 0, $5, $6, $7)
RETURNING id`,
nil,
in.PluginID,
in.CapabilityID,
started,
string(EventStatusRunning),
"",
nullable(in.MarkerBefore),
)
var id int64
if err := row.Scan(&id); err != nil {
return 0, fmt.Errorf("create autoscan event: %w", err)
}
return id, nil
}
tx, err := r.pool.Begin(ctx)
if err != nil {
return 0, fmt.Errorf("begin autoscan event: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock(900173, hashtext($1))`, sourceID); err != nil {
return 0, fmt.Errorf("lock autoscan event: %w", err)
}
var runningID int64
err = tx.QueryRow(ctx, `
SELECT id
FROM autoscan_events
WHERE source_id = $1
AND status = $2
ORDER BY started_at ASC, id ASC
LIMIT 1`,
sourceID,
string(EventStatusRunning),
).Scan(&runningID)
if err == nil {
return 0, fmt.Errorf("%w: source %s event %d", ErrPollAlreadyRunning, sourceID, runningID)
}
if !errors.Is(err, pgx.ErrNoRows) {
return 0, fmt.Errorf("check running autoscan event: %w", err)
}
row := tx.QueryRow(ctx, `
row := r.pool.QueryRow(ctx, `
INSERT INTO autoscan_events (
source_id, plugin_id, capability_id, started_at, completed_at,
source_id, installation_id, capability_id, started_at, completed_at,
duration_ms, status, error_message, marker_before
)
VALUES ($1, $2, $3, $4, $4, 0, $5, $6, $7)
RETURNING id`,
sourceID,
in.PluginID,
nullable(in.SourceID),
in.InstallationID,
in.CapabilityID,
started,
string(EventStatusRunning),
"",
string(EventStatusError),
"poll started but did not finish",
nullable(in.MarkerBefore),
)
var id int64
if err := row.Scan(&id); err != nil {
return 0, fmt.Errorf("create autoscan event: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return 0, fmt.Errorf("commit autoscan event: %w", err)
}
return id, nil
}
@@ -666,25 +565,6 @@ func (r *Repository) FinishEvent(ctx context.Context, in EventFinish) error {
return nil
}
func (r *Repository) MarkInterruptedEvents(ctx context.Context) error {
msg := "poll started but did not finish"
_, err := r.pool.Exec(ctx, `
UPDATE autoscan_events
SET completed_at = now(),
duration_ms = GREATEST(0, EXTRACT(EPOCH FROM (now() - started_at)) * 1000)::bigint,
status = $1,
error_message = $2
WHERE status = $3`,
string(EventStatusError),
msg,
string(EventStatusRunning),
)
if err != nil {
return fmt.Errorf("mark interrupted autoscan events: %w", err)
}
return nil
}
// clampAutoscanLimit bounds a requested page size to a sane window: the
// default page when unset, capped so a single query can never fan out.
func clampAutoscanLimit(limit int) int {
@@ -878,30 +758,6 @@ func (r *Repository) ListEvents(ctx context.Context, filter EventListFilter) ([]
return events, runRows.Err()
}
func (r *Repository) ListRunningEvents(ctx context.Context) ([]Event, error) {
rows, err := r.pool.Query(ctx, `
SELECT `+eventColumns+`
FROM autoscan_events
WHERE status = $1
ORDER BY started_at ASC, id ASC`,
string(EventStatusRunning),
)
if err != nil {
return nil, fmt.Errorf("list running autoscan events: %w", err)
}
defer rows.Close()
events := make([]Event, 0)
for rows.Next() {
event, scanErr := scanEvent(rows)
if scanErr != nil {
return nil, scanErr
}
events = append(events, event)
}
return events, rows.Err()
}
func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilter) ([]ScanWithEvent, error) {
limit := clampAutoscanLimit(filter.Limit)
offset := filter.Offset
@@ -924,15 +780,15 @@ func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilte
sr.trigger,
sr.status,
COALESCE(sr.error_message, ''),
sr.requested_at,
sr.started_at,
sr.completed_at,
sr.autoscan_event_id,
e.source_id,
COALESCE(e.plugin_id, ''),
COALESCE(e.capability_id, ''),
COALESCE(e.status, ''),
e.completed_at
sr.requested_at,
sr.started_at,
sr.completed_at,
sr.autoscan_event_id,
e.source_id,
e.installation_id,
COALESCE(e.capability_id, ''),
COALESCE(e.status, ''),
e.completed_at
FROM scan_runs sr
LEFT JOIN autoscan_events e ON e.id = sr.autoscan_event_id
WHERE `+strings.Join(clauses, " AND ")+`
@@ -962,7 +818,7 @@ func (r *Repository) ListAutoscanScans(ctx context.Context, filter ScanListFilte
&scan.CompletedAt,
&scan.AutoscanEventID,
&scan.SourceID,
&scan.PluginID,
&scan.InstallationID,
&scan.CapabilityID,
&eventStatus,
&scan.EventCompletedAt,
@@ -997,7 +853,7 @@ func (r *Repository) GetQueueSummary(ctx context.Context) (QueueSummary, error)
func (r *Repository) LatestEventAt(ctx context.Context) (*time.Time, error) {
var latest *time.Time
err := r.pool.QueryRow(ctx, `SELECT max(completed_at) FROM autoscan_events WHERE status <> $1`, string(EventStatusRunning)).Scan(&latest)
err := r.pool.QueryRow(ctx, `SELECT max(completed_at) FROM autoscan_events`).Scan(&latest)
if err != nil {
return nil, fmt.Errorf("get latest autoscan event time: %w", err)
}
+35 -22
View File
@@ -91,7 +91,8 @@ func (s *Service) SetSuggesterDeps(rootFolders RootFolderClient, folders FolderL
// from scan_source plugins; connres resolves a source's connection to concrete
// credentials; resolver/queue/suppress drive the resolve→suppress→enqueue loop.
// lister enumerates installed scan_source capabilities so the Add-source picker
// can offer them; it may be nil (the picker then returns an empty list).
// can offer them and PollOnce can skip orphaned source rows; it may be nil
// (the picker then returns an empty list and orphan-pruning is disabled).
func NewService(
store Store,
provider ScanSourceProvider,
@@ -112,15 +113,24 @@ func NewService(
}
}
// PollOnce runs one autoscan cycle. Per-source failures are logged, recorded on
// the source/event, and the loop continues; only settings/listing errors
// propagate. The opaque next marker returned by the
// PollOnce runs one autoscan cycle. Per-source failures are logged and skipped;
// only settings/listing errors propagate. The opaque next marker returned by the
// provider is stored verbatim, but only when the cycle's work is genuinely
// consumed: the provider returned no paths, or it returned paths and at least one
// resolved+enqueued. When paths come back but NONE resolve to a library folder
// (e.g. a freshly-enabled source with unconfigured rewrites) the marker is held
// and an error recorded, so those imports aren't skipped forever.
func (s *Service) PollOnce(ctx context.Context) error {
// Fetch the set of currently-installed scan_source capabilities so we can skip
// orphaned source rows (their plugin was uninstalled). Listing failures are
// non-fatal: a nil set means the installed set is unavailable, in which case
// we do NOT prune orphans — every enabled source is assumed present.
present, derr := s.installedScanSources(ctx)
if derr != nil {
slog.WarnContext(ctx, "autoscan: list installed scan sources failed", "err", derr)
present = nil
}
settings, err := s.store.GetSettings(ctx)
if err != nil {
return err
@@ -136,6 +146,16 @@ func (s *Service) PollOnce(ctx context.Context) error {
now := time.Now()
for _, src := range sources {
// Skip orphaned sources: an enabled source whose scan_source plugin has
// been uninstalled/disabled is no longer in the discovered set, so polling
// it would error every cycle. Skip it quietly (no RecordError) so the spam
// stops; the operator can delete it via the source delete endpoint. A nil
// `present` set means discovery is unavailable — don't prune in that case.
if present != nil {
if _, ok := present[installedKey{InstallationID: src.InstallationID, CapabilityID: src.CapabilityID}]; !ok {
continue
}
}
// Honor the per-source poll interval as a "poll at most every N seconds"
// floor: the global task fires at the default cadence, so a source with a
// longer interval is skipped until enough time has elapsed.
@@ -150,10 +170,7 @@ func (s *Service) PollOnce(ctx context.Context) error {
if src.Marker != nil {
marker = *src.Marker
}
eventID, started := s.createEvent(ctx, src, marker, time.Now())
if !started {
continue
}
eventID := s.createEvent(ctx, src, marker, time.Now())
// A connection is OPTIONAL. Server-based providers (Sonarr/Radarr) bind a
// connection and the resolved {base_url, api_key} is handed to the plugin.
// Other providers (e.g. a filesystem/CephFS watcher) need none and get an
@@ -178,7 +195,7 @@ func (s *Service) PollOnce(ctx context.Context) error {
}
conn = resolved
}
changes, next, perr := s.provider.PollChanges(ctx, src.PluginID, src.CapabilityID, marker, conn, src.SourceConfig)
changes, next, perr := s.provider.PollChanges(ctx, src.InstallationID, src.CapabilityID, marker, conn, src.SourceConfig)
if perr != nil {
slog.WarnContext(ctx, "autoscan: poll changes failed", "source_id", src.ID, "err", perr)
if rerr := s.store.RecordError(ctx, src.ID, perr.Error()); rerr != nil {
@@ -278,26 +295,22 @@ func (s *Service) PollOnce(ctx context.Context) error {
return nil
}
func (s *Service) createEvent(ctx context.Context, src Source, marker string, startedAt time.Time) (int64, bool) {
func (s *Service) createEvent(ctx context.Context, src Source, marker string, startedAt time.Time) int64 {
if s == nil || s.store == nil {
return 0, true
return 0
}
id, err := s.store.CreateEvent(ctx, EventCreate{
SourceID: src.ID,
PluginID: src.PluginID,
CapabilityID: src.CapabilityID,
StartedAt: startedAt,
MarkerBefore: marker,
SourceID: src.ID,
InstallationID: src.InstallationID,
CapabilityID: src.CapabilityID,
StartedAt: startedAt,
MarkerBefore: marker,
})
if err != nil {
if errors.Is(err, ErrPollAlreadyRunning) {
slog.DebugContext(ctx, "autoscan: source poll already running", "source_id", src.ID)
return 0, false
}
slog.WarnContext(ctx, "autoscan: create event failed", "source_id", src.ID, "err", err)
return 0, true
return 0
}
return id, true
return id
}
func (s *Service) finishEvent(ctx context.Context, eventID int64, finish EventFinish) {
+38 -85
View File
@@ -2,7 +2,6 @@ package autoscan
import (
"context"
"errors"
"path/filepath"
"strings"
"testing"
@@ -13,14 +12,13 @@ import (
)
type fakeStore struct {
settings Settings
sources []Source
connection Connection
advanced map[string]string // source ID -> marker
recorded map[string]string // source ID -> error message
createdEvents []EventCreate
events []EventFinish
createEventErr error
settings Settings
sources []Source
connection Connection
advanced map[string]string // source ID -> marker
recorded map[string]string // source ID -> error message
createdEvents []EventCreate
events []EventFinish
}
func (f *fakeStore) GetSettings(context.Context) (Settings, error) { return f.settings, nil }
@@ -53,9 +51,6 @@ func (f *fakeStore) RecordError(_ context.Context, sourceID, msg string) error {
return nil
}
func (f *fakeStore) CreateEvent(_ context.Context, event EventCreate) (int64, error) {
if f.createEventErr != nil {
return 0, f.createEventErr
}
f.createdEvents = append(f.createdEvents, event)
return int64(len(f.createdEvents)), nil
}
@@ -70,20 +65,14 @@ type fakeProvider struct {
changes map[string][]Change // key: capabilityID, structured changes
nextMarker string
err error
errByCap map[string]error
lastConfig map[string]string
calls int
}
func (f *fakeProvider) PollChanges(_ context.Context, _ string, capabilityID, _ string, _ ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
f.calls++
func (f *fakeProvider) PollChanges(_ context.Context, _ int, capabilityID, _ string, _ ResolvedConnection, sourceConfig map[string]string) ([]Change, string, error) {
f.lastConfig = sourceConfig
if f.err != nil {
return nil, "", f.err
}
if err := f.errByCap[capabilityID]; err != nil {
return nil, "", err
}
if changes, ok := f.changes[capabilityID]; ok {
return changes, f.nextMarker, nil
}
@@ -98,7 +87,7 @@ func TestPollOncePassesSourceConfigToProvider(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
SourceConfig: map[string]string{"exclusions": ".downloads\n.recyclebin"},
}},
}
@@ -112,38 +101,6 @@ func TestPollOncePassesSourceConfigToProvider(t *testing.T) {
}
}
func TestPollOnceSkipsSourceWhenPollAlreadyRunning(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
createEventErr: ErrPollAlreadyRunning,
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"cephfs": {"/mnt/media/Movie/movie.mkv"}}, nextMarker: "m1"}
q := &recordingQueuer{}
svc := newService(store, prov, q, allowSuppressor{})
if err := svc.PollOnce(context.Background()); err != nil {
t.Fatalf("PollOnce: %v", err)
}
if prov.calls != 0 {
t.Fatalf("provider calls = %d, want 0", prov.calls)
}
if len(q.enqueued) != 0 {
t.Fatalf("enqueued scans = %+v, want none", q.enqueued)
}
if len(store.events) != 0 {
t.Fatalf("finished events = %+v, want none", store.events)
}
if len(store.recorded) != 0 {
t.Fatalf("recorded source errors = %+v, want none", store.recorded)
}
if len(store.advanced) != 0 {
t.Fatalf("advanced markers = %+v, want none", store.advanced)
}
}
// passthroughConnRes resolves to empty credentials; the engine doesn't inspect
// them in tests (the fake provider ignores conn).
type passthroughConnRes struct{}
@@ -253,7 +210,7 @@ func TestPollOnceEnqueuesDedupedFolders(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -284,7 +241,7 @@ func TestPollOnceRecordsSuccessfulEvent(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true, Marker: &marker,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true, Marker: &marker,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -326,7 +283,7 @@ func TestPollOnceRecordsReusedScanCounts(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -361,7 +318,7 @@ func TestPollOnceAppliesSourceRewritesBeforeEnqueue(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PathRewrites: []PathRewrite{{From: "/data/tv", To: "/mnt/media/tv"}},
}},
}
@@ -390,7 +347,7 @@ func TestPollOnceStructuredFileChangeEnqueuesExactFile(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
PathRewrites: []PathRewrite{{From: "/ceph/tv", To: "/mnt/media/tv"}},
}},
}
@@ -423,7 +380,7 @@ func TestPollOnceStructuredSubtreeChangeEnqueuesExactSubtree(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.cephfs", CapabilityID: "cephfs", Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "cephfs", Enabled: true,
PathRewrites: []PathRewrite{{From: "/ceph/movies", To: "/mnt/media/movies"}},
}},
}
@@ -459,7 +416,7 @@ func TestPollOnceScansDistinctPathsUnderSameFolder(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -489,7 +446,7 @@ func TestPollOnceStoresOpaqueMarkerVerbatim(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
const opaque = "eyJjdXJzb3IiOiJhYmMxMjMifQ==|2026-06-02T14:10:00Z"
@@ -513,7 +470,7 @@ func TestPollOnceHoldsMarkerWhenPathsReturnedButNoneResolve(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{
@@ -566,7 +523,7 @@ func TestPollOnceAdvancesMarkerWhenResolvedButAllSuppressed(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
// rewrite /data/tv -> /mnt/media/tv so fakeResolver resolves the paths.
PathRewrites: []PathRewrite{{From: "/data/tv", To: "/mnt/media/tv"}},
}},
@@ -599,7 +556,7 @@ func TestPollOnceAdvancesMarkerWhenZeroPathsReturned(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {}}, nextMarker: "m1"}
@@ -631,7 +588,7 @@ func TestPollOnceDisabledNoop(t *testing.T) {
func TestPollOnceProviderErrorKeepsMarker(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
sources: []Source{{ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
}
prov := &fakeProvider{err: context.DeadlineExceeded}
q := &recordingQueuer{}
@@ -650,7 +607,7 @@ func TestPollOnceProviderErrorKeepsMarker(t *testing.T) {
func TestPollOnceReleasesClaimOnEnqueueFailure(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
sources: []Source{{ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {"/mnt/media/Show/S01/E01.mkv"}}, nextMarker: "m1"}
sup := &recordingSuppressor{}
@@ -683,7 +640,7 @@ func TestPollOncePollsConnectionlessSource(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: nil, Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: nil, Enabled: true,
}},
}
prov := &fakeProvider{paths: map[string][]string{"arr": {"/mnt/media/Show/S01/E01.mkv"}}, nextMarker: "m1"}
@@ -709,7 +666,7 @@ func TestPollOnceSkipsSourcePolledTooRecently(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PollIntervalSeconds: &interval, LastRunAt: &recent,
}},
}
@@ -733,7 +690,7 @@ func TestPollOnceRunsSourcePastItsInterval(t *testing.T) {
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{{
ID: "s1", PluginID: "silo.autoscan.arr", CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
ID: "s1", InstallationID: 1, CapabilityID: "arr", ConnectionID: strptr("c1"), Enabled: true,
PollIntervalSeconds: &interval, LastRunAt: &old,
}},
}
@@ -751,23 +708,25 @@ func TestPollOnceRunsSourcePastItsInterval(t *testing.T) {
}
}
func TestPollOnceRecordsScanSourceResolutionFailure(t *testing.T) {
// Two enabled sources: one polls normally, while the other simulates the
// production resolver failing because the plugin cannot be resolved.
func TestPollOnceSkipsOrphanedSourceQuietly(t *testing.T) {
// Two enabled sources: only "arr-live" is still discovered; "arr-gone"'s
// plugin has been uninstalled. The orphan must be skipped quietly — no poll,
// no RecordError spam — while the live source still polls normally.
store := &fakeStore{
settings: Settings{Enabled: true, DefaultPollIntervalSeconds: 600, DebounceSeconds: 60},
sources: []Source{
{ID: "live", PluginID: "silo.autoscan.live", CapabilityID: "arr-live", ConnectionID: strptr("c1"), Enabled: true},
{ID: "gone", PluginID: "silo.autoscan.gone", CapabilityID: "arr-gone", ConnectionID: strptr("c1"), Enabled: true},
{ID: "live", InstallationID: 1, CapabilityID: "arr-live", ConnectionID: strptr("c1"), Enabled: true},
{ID: "gone", InstallationID: 2, CapabilityID: "arr-gone", ConnectionID: strptr("c1"), Enabled: true},
},
}
prov := &fakeProvider{paths: map[string][]string{
"arr-live": {"/mnt/media/Show/S01/E01.mkv"},
}, errByCap: map[string]error{
"arr-gone": errors.New("scan source plugin \"silo.autoscan.gone\" is not installed"),
"arr-gone": {"/mnt/media/Other/S01/E01.mkv"},
}, nextMarker: "m1"}
q := &recordingQueuer{}
svc := newService(store, prov, q, allowSuppressor{})
// Lister reports only the live capability; the orphan is absent.
lister := fakeLister{sources: []DiscoveredSource{{InstallationID: 1, CapabilityID: "arr-live"}}}
svc := NewService(store, prov, passthroughConnRes{}, fakeResolver{}, q, allowSuppressor{}, lister)
if err := svc.PollOnce(context.Background()); err != nil {
t.Fatalf("PollOnce: %v", err)
@@ -779,15 +738,9 @@ func TestPollOnceRecordsScanSourceResolutionFailure(t *testing.T) {
t.Fatalf("expected live source marker advanced")
}
if _, ok := store.advanced["gone"]; ok {
t.Fatalf("failed source must NOT advance")
t.Fatalf("orphaned source must NOT poll/advance")
}
if got := store.recorded["gone"]; !strings.Contains(got, "not installed") {
t.Fatalf("failed source error = %q", got)
}
if len(store.events) != 2 {
t.Fatalf("expected one event per source, got %+v", store.events)
}
if store.events[1].Status != EventStatusError || !strings.Contains(store.events[1].ErrorMessage, "not installed") {
t.Fatalf("failed source event = %+v", store.events[1])
if _, ok := store.recorded["gone"]; ok {
t.Fatalf("orphaned source must be skipped quietly, but an error was recorded: %q", store.recorded["gone"])
}
}
+9 -10
View File
@@ -45,11 +45,11 @@ type Change struct {
Scope ChangeScope
}
// Source ties a scan_source plugin capability to a connection plus the
// Source ties a scan_source plugin capability instance to a connection plus the
// host-owned scheduling/bookkeeping state.
type Source struct {
ID string
PluginID string
InstallationID int
CapabilityID string
ConnectionID *string // nil until an operator binds a connection
Enabled bool
@@ -65,7 +65,6 @@ type Source struct {
type EventStatus string
const (
EventStatusRunning EventStatus = "running"
EventStatusSuccess EventStatus = "success"
EventStatusError EventStatus = "error"
EventStatusUnresolved EventStatus = "unresolved"
@@ -74,7 +73,7 @@ const (
type Event struct {
ID int64
SourceID *string
PluginID string
InstallationID int
CapabilityID string
StartedAt time.Time
CompletedAt time.Time
@@ -92,11 +91,11 @@ type Event struct {
}
type EventCreate struct {
SourceID string
PluginID string
CapabilityID string
StartedAt time.Time
MarkerBefore string
SourceID string
InstallationID int
CapabilityID string
StartedAt time.Time
MarkerBefore string
}
type EventFinish struct {
@@ -155,7 +154,7 @@ type ScanWithEvent struct {
ScanRunSummary
AutoscanEventID *int64
SourceID *string
PluginID string
InstallationID *int
CapabilityID string
EventStatus EventStatus
EventCompletedAt *time.Time
-8
View File
@@ -36,7 +36,6 @@ type BrowseFilters struct {
MaxLimit int // optional caller-specific cap; zero keeps the default browse cap
Offset int
SnapshotAt *time.Time // pagination fence: exclude items created after this timestamp
RequireBackdrop bool // only return items with a non-empty backdrop_path (Jellyfin ImageTypes=Backdrop filter)
}
// BrowseResult contains the paginated result of a browse query.
@@ -296,13 +295,6 @@ func (r *BrowseRepository) buildBrowsePlan(filters BrowseFilters) (browseQueryPl
argIdx++
}
// Backdrop presence filter (Jellyfin ImageTypes=Backdrop). Pushed down so
// random/limited selections only ever pick items that actually have a
// backdrop — filtering after the LIMIT would wrongly return empty pages.
if filters.RequireBackdrop {
conditions = append(conditions, "NULLIF(BTRIM(mi.backdrop_path), '') IS NOT NULL")
}
// Library access control: restrict to user's accessible libraries.
needsLibJoin := filters.LibraryID > 0 || filters.LibraryIDs != nil || len(filters.DisabledLibraryIDs) > 0 || filters.Sort == "recently_added"
needsPersonJoin := filters.PersonID > 0
-31
View File
@@ -1,31 +0,0 @@
package catalog
import (
"strings"
"testing"
)
// TestBuildBrowsePlan_RequireBackdrop asserts the ImageTypes=Backdrop filter
// (BrowseFilters.RequireBackdrop) renders the backdrop-presence predicate into
// the WHERE clause, and is absent otherwise. Guards against a future refactor
// of buildBrowsePlan silently dropping the condition.
func TestBuildBrowsePlan_RequireBackdrop(t *testing.T) {
const predicate = "NULLIF(BTRIM(mi.backdrop_path), '') IS NOT NULL"
repo := &BrowseRepository{}
plan, earlyEmpty, err := repo.buildBrowsePlan(BrowseFilters{Type: "movie", RequireBackdrop: true})
if err != nil || earlyEmpty {
t.Fatalf("buildBrowsePlan(RequireBackdrop) err=%v earlyEmpty=%v", err, earlyEmpty)
}
if !strings.Contains(plan.whereClause, predicate) {
t.Fatalf("RequireBackdrop=true: whereClause missing predicate.\ngot: %s", plan.whereClause)
}
plan, _, err = repo.buildBrowsePlan(BrowseFilters{Type: "movie"})
if err != nil {
t.Fatalf("buildBrowsePlan err=%v", err)
}
if strings.Contains(plan.whereClause, predicate) {
t.Fatalf("RequireBackdrop unset: predicate should be absent.\ngot: %s", plan.whereClause)
}
}
+5 -4
View File
@@ -389,11 +389,12 @@ func normalizeCatalogMatch(raw string) string {
}
func parseCatalogMediaScope(raw string) string {
scope := strings.ToLower(strings.TrimSpace(raw))
if scope != "" && IsValidMediaScope(scope) {
return scope
switch strings.ToLower(strings.TrimSpace(raw)) {
case "movie", "series", "episode", "audiobook":
return strings.ToLower(strings.TrimSpace(raw))
default:
return ""
}
return ""
}
func defaultCatalogQuerySort(searchQuery string) QuerySort {
+13 -7
View File
@@ -1086,8 +1086,11 @@ func validateCatalogExactCollectionRequest(req CatalogRequest) error {
}
func validateCatalogOverlayQuery(searchQuery string, def QueryDefinition, ruleFields, sortFields map[string]bool, allowRelevance bool) error {
if !IsValidMediaScope(def.MediaScope) {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', 'episode', 'audiobook', or 'video'", ErrInvalidCatalogRequest)
if def.MediaScope != "" &&
def.MediaScope != "movie" &&
def.MediaScope != "series" &&
def.MediaScope != "episode" {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', or 'episode'", ErrInvalidCatalogRequest)
}
if def.Match != "" && def.Match != "all" && def.Match != "any" {
return fmt.Errorf("%w: match must be 'all' or 'any'", ErrInvalidCatalogRequest)
@@ -1603,7 +1606,12 @@ func catalogSearchAccess(req CatalogRequest, access AccessFilter) (AccessFilter,
MaxContentRating: access.MaxContentRating,
}
return searchAccess, MediaScopeItemTypes(req.Query.MediaScope), false
var itemTypes []string
if req.Query.MediaScope != "" {
itemTypes = []string{req.Query.MediaScope}
}
return searchAccess, itemTypes, false
}
func catalogBrowseFilters(req CatalogRequest, access AccessFilter) (BrowseFilters, bool, error) {
@@ -1613,9 +1621,7 @@ func catalogBrowseFilters(req CatalogRequest, access AccessFilter) (BrowseFilter
}
filters := BrowseFilters{
// BrowseFilters.Type accepts a comma-separated type list, so group
// scopes like "video" expand here rather than leaking downstream.
Type: strings.Join(MediaScopeItemTypes(req.Query.MediaScope), ","),
Type: req.Query.MediaScope,
NamePrefix: req.NamePrefix,
DisabledLibraryIDs: effectiveCatalogDisabledLibraryIDs(req.Query.LibraryIDs, access.DisabledLibraryIDs),
MaxContentRating: access.MaxContentRating,
@@ -1756,7 +1762,7 @@ func filterCatalogItems(items []*models.MediaItem, def QueryDefinition) []*model
if item == nil {
continue
}
if !MediaScopeMatchesItemType(def.MediaScope, item.Type) {
if def.MediaScope != "" && !strings.EqualFold(strings.TrimSpace(item.Type), def.MediaScope) {
continue
}
if catalogDefinitionMatchesItem(item, def) {
-427
View File
@@ -154,51 +154,6 @@ type ItemDetail struct {
EffectiveVersionHDR *bool `json:"effective_version_hdr,omitempty"`
EffectiveVersionCodecVideo *string `json:"effective_version_codec_video,omitempty"`
EffectiveVersionEditionKey *string `json:"effective_version_edition_key,omitempty"`
// Audiobook-specific detail. Present only when Type == "audiobook".
Audiobook *AudiobookDetailExtension `json:"audiobook,omitempty"`
}
type AudiobookDetailExtension struct {
Authors []AudiobookPerson `json:"authors"`
Narrators []AudiobookPerson `json:"narrators"`
Publisher string `json:"publisher,omitempty"`
TotalDurationSeconds int `json:"total_duration_seconds"`
Series *AudiobookSeriesGroup `json:"series,omitempty"`
OtherNarrations []AudiobookNarration `json:"other_narrations"`
Related AudiobookRelatedContent `json:"related"`
}
type AudiobookPerson struct {
PersonID string `json:"person_id,omitempty"`
Name string `json:"name"`
PhotoURL string `json:"photo_url,omitempty"`
PhotoThumbhash string `json:"photo_thumbhash,omitempty"`
}
type AudiobookRelatedContent struct {
AlsoByAuthor []AudiobookRelatedItem `json:"also_by_author"`
Similar []AudiobookRelatedItem `json:"similar"`
}
type AudiobookRelatedItem struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
PosterURL string `json:"poster_url,omitempty"`
SeriesIndex *int `json:"series_index,omitempty"`
}
type AudiobookSeriesGroup struct {
Name string `json:"name,omitempty"`
Entries []AudiobookRelatedItem `json:"entries"`
}
type AudiobookNarration struct {
ContentID string `json:"content_id"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
Narrators []string `json:"narrators"`
}
// ItemUserState is per-profile viewer state included in item detail responses.
@@ -269,7 +224,6 @@ type FileVersion struct {
PresentationKind string `json:"presentation_kind,omitempty"`
PresentationGroupKey string `json:"presentation_group_key,omitempty"`
PresentationPartIndex int `json:"presentation_part_index,omitempty"`
PresentationPartTotal int `json:"presentation_part_total,omitempty"`
MultiEpisodeStart int `json:"multi_episode_start,omitempty"`
MultiEpisodeEnd int `json:"multi_episode_end,omitempty"`
EffectiveAudioTrackIndex *int `json:"effective_audio_track_index,omitempty"`
@@ -879,9 +833,6 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
}
files = FilterMediaFilesByAccess(files, filter)
if item.Type == "audiobook" {
sortAudiobookMediaFiles(files)
}
files = s.preparePlaybackFiles(ctx, files)
detail.Versions, detail.PlaybackVariants, detail.Subtitles, detail.Intro, detail.Credits, detail.Recap, detail.Preview = s.buildPlaybackInfo(
ctx,
@@ -892,10 +843,6 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
detail.OverlaySummary = overlays.BuildSummary(files)
}
if item.Type == "audiobook" {
detail.Audiobook = s.buildAudiobookExtension(ctx, item, detail.Versions, crewCredits, filter)
}
// Series folder paths from confirmed claims when available, otherwise from
// the file links that currently belong to the item. This keeps provisional
// root-scoped series visible for manual match/admin flows without implying
@@ -1005,355 +952,6 @@ func splitCastCrew(credits []PersonCredit) ([]CastCredit, []CrewCredit) {
return cast, crew
}
func (s *DetailService) buildAudiobookExtension(
ctx context.Context,
item *models.MediaItem,
versions []FileVersion,
crew []CrewCredit,
filter AccessFilter,
) *AudiobookDetailExtension {
if item == nil {
return nil
}
totalDuration := 0
for _, version := range versions {
if version.Duration > 0 {
totalDuration += version.Duration
}
}
return &AudiobookDetailExtension{
Authors: audiobookPeopleFromCrew(crew, models.PersonKindAuthor.String()),
Narrators: audiobookPeopleFromCrew(crew, models.PersonKindNarrator.String()),
Publisher: firstNonEmptyString(item.Studios),
TotalDurationSeconds: totalDuration,
Series: s.fetchAudiobookSeries(ctx, item.ContentID, filter),
OtherNarrations: s.fetchAudiobookOtherNarrations(ctx, item.ContentID, filter),
Related: AudiobookRelatedContent{
AlsoByAuthor: s.fetchAudiobookAlsoByAuthor(ctx, item.ContentID, filter),
Similar: s.fetchAudiobookSimilarByGenres(ctx, item.ContentID, filter),
},
}
}
func audiobookPeopleFromCrew(crew []CrewCredit, job string) []AudiobookPerson {
out := make([]AudiobookPerson, 0)
for _, credit := range crew {
if credit.Job != job || strings.TrimSpace(credit.Name) == "" {
continue
}
out = append(out, AudiobookPerson{
PersonID: credit.PersonID,
Name: credit.Name,
PhotoURL: credit.PhotoURL,
PhotoThumbhash: credit.PhotoThumbhash,
})
}
return out
}
func firstNonEmptyString(values []string) string {
for _, value := range values {
if strings.TrimSpace(value) != "" {
return value
}
}
return ""
}
func (s *DetailService) presignAudiobookPosterURL(ctx context.Context, posterPath string) string {
return s.PresignImageURL(ctx, posterPath, "poster", "")
}
func appendAudiobookItemAccessConditions(
alias string,
filter AccessFilter,
conditions *[]string,
args *[]any,
argIdx *int,
) bool {
if filter.AllowedLibraryIDs != nil {
if len(filter.AllowedLibraryIDs) == 0 {
return false
}
*conditions = append(*conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil_allowed
WHERE mil_allowed.content_id = %s.content_id
AND mil_allowed.media_folder_id = ANY($%d)
)`, alias, *argIdx))
*args = append(*args, filter.AllowedLibraryIDs)
*argIdx = *argIdx + 1
}
if len(filter.DisabledLibraryIDs) > 0 {
if filter.AllowedLibraryIDs == nil {
*conditions = append(*conditions, fmt.Sprintf(`
EXISTS (
SELECT 1 FROM media_item_libraries mil_visible
WHERE mil_visible.content_id = %s.content_id
)`, alias))
}
*conditions = append(*conditions, fmt.Sprintf(`
NOT EXISTS (
SELECT 1 FROM media_item_libraries mil_disabled
WHERE mil_disabled.content_id = %s.content_id
AND mil_disabled.media_folder_id = ANY($%d)
)`, alias, *argIdx))
*args = append(*args, filter.DisabledLibraryIDs)
*argIdx = *argIdx + 1
}
ApplySectionAccessFilter(alias, AccessFilter{MaxContentRating: filter.MaxContentRating}, conditions, args, argIdx)
return true
}
func (s *DetailService) fetchAudiobookAlsoByAuthor(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
conditions := []string{
"ip1.content_id = $1",
"ip1.kind = $2",
"m2.type = 'audiobook'",
}
if !appendAudiobookItemAccessConditions("m2", filter, &conditions, &args, &argIdx) {
return []AudiobookRelatedItem{}
}
query := fmt.Sprintf(`
SELECT m2.content_id, m2.title, COALESCE(m2.year, 0), COALESCE(m2.poster_path, '')
FROM item_people ip1
JOIN item_people ip2
ON ip2.person_id = ip1.person_id
AND ip2.kind = ip1.kind
AND ip2.content_id <> ip1.content_id
JOIN media_items m2 ON m2.content_id = ip2.content_id
WHERE %s
ORDER BY m2.year DESC NULLS LAST, LOWER(m2.sort_title)
LIMIT 12
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookRelatedItem{}
}
defer rows.Close()
out := make([]AudiobookRelatedItem, 0, 12)
seen := make(map[string]struct{})
for rows.Next() {
var item AudiobookRelatedItem
var posterPath string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &posterPath); err != nil {
return []AudiobookRelatedItem{}
}
if _, ok := seen[item.ContentID]; ok {
continue
}
seen[item.ContentID] = struct{}{}
item.PosterURL = s.presignAudiobookPosterURL(ctx, posterPath)
out = append(out, item)
}
return out
}
func (s *DetailService) fetchAudiobookSimilarByGenres(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
conditions := []string{
"m.type = 'audiobook'",
"m.content_id <> $1",
"m.genres && (SELECT array_agg(g) FROM this_genres)",
`NOT EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $2
AND ip.person_id IN (SELECT person_id FROM this_author)
)`,
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return []AudiobookRelatedItem{}
}
query := fmt.Sprintf(`
WITH this_genres AS (
SELECT unnest(genres) AS g FROM media_items WHERE content_id = $1
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $1 AND kind = $2
)
SELECT m.content_id, m.title, COALESCE(m.year, 0), COALESCE(m.poster_path, '')
FROM media_items m
WHERE %s
ORDER BY
(
SELECT COUNT(*)
FROM unnest(m.genres) AS genre(value)
WHERE genre.value IN (SELECT g FROM this_genres)
) DESC,
COALESCE(m.year, 0) DESC,
LOWER(m.sort_title)
LIMIT 12
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookRelatedItem{}
}
defer rows.Close()
out := make([]AudiobookRelatedItem, 0, 12)
for rows.Next() {
var item AudiobookRelatedItem
var posterPath string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &posterPath); err != nil {
return []AudiobookRelatedItem{}
}
item.PosterURL = s.presignAudiobookPosterURL(ctx, posterPath)
out = append(out, item)
}
return out
}
func (s *DetailService) fetchAudiobookSeries(ctx context.Context, contentID string, filter AccessFilter) *AudiobookSeriesGroup {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return nil
}
args := []any{contentID}
argIdx := 2
conditions := []string{
"root.content_id = $1",
"m.type = 'audiobook'",
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return nil
}
query := fmt.Sprintf(`
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(m.poster_path, ''),
s.series_name,
s.series_index
FROM audiobook_series root
JOIN audiobook_series s ON LOWER(s.series_name) = LOWER(root.series_name)
JOIN media_items m ON m.content_id = s.content_id
WHERE %s
ORDER BY s.series_index NULLS LAST, LOWER(m.sort_title)
LIMIT 30
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return nil
}
defer rows.Close()
var seriesName string
entries := make([]AudiobookRelatedItem, 0, 16)
for rows.Next() {
var (
item AudiobookRelatedItem
poster string
name string
index *float64
)
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &poster, &name, &index); err != nil {
return nil
}
if seriesName == "" {
seriesName = name
}
if index != nil {
n := int(*index)
if float64(n) == *index {
item.SeriesIndex = &n
}
}
item.PosterURL = s.presignAudiobookPosterURL(ctx, poster)
entries = append(entries, item)
}
if len(entries) < 2 {
return nil
}
return &AudiobookSeriesGroup{Name: seriesName, Entries: entries}
}
func (s *DetailService) fetchAudiobookOtherNarrations(ctx context.Context, contentID string, filter AccessFilter) []AudiobookNarration {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookNarration{}
}
const stripRE = `\s*\(?\s*[-:,]?\s*(UK Version:?|US Version:?)?\s*[Rr]ead [Bb]y [^()]+\)?\s*$`
args := []any{stripRE, contentID, models.PersonKindAuthor, models.PersonKindNarrator}
argIdx := 5
conditions := []string{
"m.type = 'audiobook'",
"m.content_id <> $2",
"trim(regexp_replace(m.title, $1, '', 'g')) = (SELECT norm_title FROM this_book)",
`EXISTS (
SELECT 1 FROM item_people ip
WHERE ip.content_id = m.content_id
AND ip.kind = $3
AND ip.person_id IN (SELECT person_id FROM this_author)
)`,
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return []AudiobookNarration{}
}
query := fmt.Sprintf(`
WITH this_book AS (
SELECT trim(regexp_replace(title, $1, '', 'g')) AS norm_title
FROM media_items WHERE content_id = $2
),
this_author AS (
SELECT person_id FROM item_people WHERE content_id = $2 AND kind = $3
)
SELECT
m.content_id,
m.title,
COALESCE(m.year, 0),
COALESCE(string_agg(DISTINCT NULLIF(BTRIM(p.name), ''), '|||'), '') AS narrators
FROM media_items m
LEFT JOIN item_people ipn ON ipn.content_id = m.content_id AND ipn.kind = $4
LEFT JOIN people p ON p.id = ipn.person_id
WHERE %s
GROUP BY m.content_id, m.title, m.year
ORDER BY m.year DESC NULLS LAST, m.title
LIMIT 8
`, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return []AudiobookNarration{}
}
defer rows.Close()
out := make([]AudiobookNarration, 0, 4)
for rows.Next() {
var item AudiobookNarration
var narrators string
if err := rows.Scan(&item.ContentID, &item.Title, &item.Year, &narrators); err != nil {
return []AudiobookNarration{}
}
item.Narrators = splitDelimitedNames(narrators)
out = append(out, item)
}
return out
}
func splitDelimitedNames(raw string) []string {
if strings.TrimSpace(raw) == "" {
return []string{}
}
parts := strings.Split(raw, "|||")
out := make([]string, 0, len(parts))
for _, part := range parts {
if value := strings.TrimSpace(part); value != "" {
out = append(out, value)
}
}
return out
}
func seriesFolderPathsFromFiles(files []*models.MediaFile) []string {
if len(files) == 0 {
return nil
@@ -1969,30 +1567,6 @@ func markerFromRange(start, end *float64) *Marker {
return &Marker{Start: *start, End: *end}
}
func sortAudiobookMediaFiles(files []*models.MediaFile) {
sort.SliceStable(files, func(i, j int) bool {
a, b := files[i], files[j]
if a == nil || b == nil {
return a != nil
}
if a.PresentationPartIndex > 0 || b.PresentationPartIndex > 0 {
if a.PresentationPartIndex != b.PresentationPartIndex {
if a.PresentationPartIndex == 0 {
return false
}
if b.PresentationPartIndex == 0 {
return true
}
return a.PresentationPartIndex < b.PresentationPartIndex
}
}
if a.FilePath != b.FilePath {
return strings.Compare(a.FilePath, b.FilePath) < 0
}
return a.ID < b.ID
})
}
func (s *DetailService) buildPlaybackInfo(
ctx context.Context,
files []*models.MediaFile,
@@ -2048,7 +1622,6 @@ func (s *DetailService) buildPlaybackInfo(
PresentationKind: f.PresentationKind,
PresentationGroupKey: f.PresentationGroupKey,
PresentationPartIndex: f.PresentationPartIndex,
PresentationPartTotal: f.PresentationPartTotal,
MultiEpisodeStart: f.MultiEpisodeStart,
MultiEpisodeEnd: f.MultiEpisodeEnd,
EffectiveAudioTrackIndex: intPtr(effectiveAudioSelection.Index),
-78
View File
@@ -1,78 +0,0 @@
package catalog
import (
"context"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestSortAudiobookMediaFilesPreservesPresentationPartOrder(t *testing.T) {
files := []*models.MediaFile{
{ID: 30, FilePath: "/books/book/03.mp3", PresentationPartIndex: 3},
{ID: 10, FilePath: "/books/book/01.mp3", PresentationPartIndex: 1},
{ID: 20, FilePath: "/books/book/02.mp3", PresentationPartIndex: 2},
}
sortAudiobookMediaFiles(files)
got := []int{files[0].ID, files[1].ID, files[2].ID}
want := []int{10, 20, 30}
for i := range want {
if got[i] != want[i] {
t.Fatalf("sorted IDs = %v, want %v", got, want)
}
}
}
func TestSortAudiobookMediaFilesFallsBackToPath(t *testing.T) {
files := []*models.MediaFile{
{ID: 20, FilePath: "/books/book/02.mp3"},
{ID: 10, FilePath: "/books/book/01.mp3"},
}
sortAudiobookMediaFiles(files)
got := []int{files[0].ID, files[1].ID}
want := []int{10, 20}
for i := range want {
if got[i] != want[i] {
t.Fatalf("sorted IDs = %v, want %v", got, want)
}
}
}
func TestPresignAudiobookPosterURLUsesPosterVariant(t *testing.T) {
resolver := &recordingCatalogImageResolver{}
detail := &DetailService{}
detail.SetImageResolver(resolver)
got := detail.presignAudiobookPosterURL(context.Background(), "local/audiobooks/book/poster/original.webp")
if !strings.Contains(got, "/w500.webp") {
t.Fatalf("resolved URL = %q, want w500 poster variant", got)
}
if resolver.variant != "featured" {
t.Fatalf("resolver variant = %q, want featured", resolver.variant)
}
}
type recordingCatalogImageResolver struct {
path string
variant string
}
func (r *recordingCatalogImageResolver) ResolveImageURL(_ context.Context, path string, variant string) string {
r.path = path
r.variant = variant
return "resolved://" + path
}
func (r *recordingCatalogImageResolver) ResolveImageURLs(_ context.Context, paths []string, variant string) map[string]string {
out := make(map[string]string, len(paths))
for _, path := range paths {
out[path] = r.ResolveImageURL(context.Background(), path, variant)
}
return out
}
+136 -138
View File
@@ -28,35 +28,65 @@ type RatingFilter struct {
Min float64
// Limit caps the number of rows returned. Zero or negative means no limit.
Limit int
// LibraryID, when non-nil, restricts results to items in that library.
// LibraryID, when non-nil, restricts results to items in that library via
// the media_item_libraries junction table.
LibraryID *int
// Filter carries viewer-level access constraints (content rating ceiling,
// allowed/disabled library sets).
Filter AccessFilter
}
// buildRatingThresholdQuery builds the SQL statement and bind args for ListByRatingThreshold.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold - NULL ratings are excluded implicitly by >=.
// IMDb rating threshold — NULL ratings are excluded implicitly by >=.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.Min)
argIdx++
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, f.LibraryID, f.Filter); !ok {
return "", nil
// Optional single-library pin.
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id = $%d", argIdx))
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
strings.Join(conditions, " AND "),
)
@@ -65,18 +95,6 @@ func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
args = append(args, f.Limit)
}
return query, args
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
query, args := buildRatingThresholdQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing items by rating threshold: %w", err)
@@ -104,50 +122,6 @@ type UnplayedFilter struct {
Filter AccessFilter
}
// buildUnplayedHighRatedQuery builds the SQL statement and bind args for ListUnplayedHighRated.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// Items the user has any watch history entry for are excluded.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
}
return query, args
}
// ListUnplayedHighRated returns high-rated items that the given user/profile has
// never started watching. "Never started" means no row exists in
// user_watch_history for (user_id, profile_id, media_item_id), regardless of
@@ -159,9 +133,67 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
return nil, fmt.Errorf("ListUnplayedHighRated: UserID and ProfileID are required")
}
query, args := buildUnplayedHighRatedQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
var conditions []string
var args []any
argIdx := 1
// IMDb rating threshold.
conditions = append(conditions, fmt.Sprintf("mi.rating_imdb >= $%d", argIdx))
args = append(args, f.MinRating)
argIdx++
// LEFT JOIN exclusion: items the user has any watch history entry for are excluded.
// We use a NOT EXISTS subquery rather than an outer join so it composes cleanly
// with library access conditions on the main query.
conditions = append(conditions, fmt.Sprintf(`NOT EXISTS (
SELECT 1
FROM user_watch_history uwh
WHERE uwh.user_id = $%d
AND uwh.profile_id = $%d
AND uwh.media_item_id = mi.content_id
)`, argIdx, argIdx+1))
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
strings.Join(conditions, " AND "),
)
if f.Limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argIdx)
args = append(args, f.Limit)
}
rows, err := r.pool.Query(ctx, query, args...)
@@ -193,10 +225,13 @@ type ForgottenFavoritesFilter struct {
Filter AccessFilter
}
// buildForgottenFavoritesQuery builds the SQL statement and bind args for ListForgottenFavorites.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
@@ -220,15 +255,39 @@ func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
if ok := appendDiscoveryLibraryScope(&conditions, &args, &argIdx, nil, f.Filter); !ok {
return "", nil
// Library access control.
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id IN (%s)", strings.Join(placeholders, ", ")))
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = fmt.Sprintf("$%d", argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id NOT IN (%s)", strings.Join(placeholders, ", ")))
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := fmt.Sprintf(
"SELECT %s FROM media_items mi WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
qualifiedItemColumns("mi"),
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
fromClause,
strings.Join(conditions, " AND "),
)
@@ -237,21 +296,6 @@ func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
args = append(args, f.Limit)
}
return query, args
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
query, args := buildForgottenFavoritesQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing forgotten favorites: %w", err)
@@ -264,49 +308,3 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
}
return items, nil
}
func appendDiscoveryLibraryScope(
conditions *[]string,
args *[]any,
argIdx *int,
libraryID *int,
filter AccessFilter,
) bool {
if libraryID != nil {
whereSQL, scopeArgs, ok := buildLibraryScopeJoin([]int{*libraryID}, nil, *argIdx, "", "mi.content_id")
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
if filter.AllowedLibraryIDs != nil && len(filter.AllowedLibraryIDs) == 0 {
return false
}
// buildLibraryScopeJoin uses semi-joins: disabled libraries are an item-level
// exclusion, matching the rest of catalog access filtering and avoiding row fanout.
if filter.AllowedLibraryIDs == nil && len(filter.DisabledLibraryIDs) > 0 {
// In deny-only mode, require at least one library membership. Without this,
// stale or provisional media_items rows with no membership pass NOT EXISTS.
*conditions = append(*conditions,
"EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)",
)
}
whereSQL, scopeArgs, ok := buildLibraryScopeJoin(
filter.AllowedLibraryIDs,
filter.DisabledLibraryIDs,
*argIdx,
"",
"mi.content_id",
)
if ok {
*conditions = append(*conditions, whereSQL)
*args = append(*args, scopeArgs...)
*argIdx += len(scopeArgs)
}
return true
}
+224 -146
View File
@@ -4,12 +4,139 @@ package catalog
// discovery_repo.go using the same pure-unit approach used throughout this
// package: build a query string and assert that key fragments are present,
// without needing a live database.
//
// The helpers themselves execute against a real pool; integration coverage is
// provided by whichever recipe resolver exercises them in higher-level tests.
import (
"strings"
"testing"
)
// buildRatingThresholdQuery is the testable extraction of the SQL generation
// logic from ListByRatingThreshold so we can inspect it without a pool.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.Min)
argIdx++
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, "mil.media_folder_id = $2")
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// buildUnplayedHighRatedQuery mirrors the SQL-building portion of
// ListUnplayedHighRated for unit testing.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.MinRating)
argIdx++
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t)")
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// itoa is a local helper to avoid importing strconv.
func itoa(n int) string {
if n == 0 {
return "0"
}
buf := [20]byte{}
pos := len(buf)
for n > 0 {
pos--
buf[pos] = byte('0' + n%10)
n /= 10
}
return string(buf[pos:])
}
// ---------------------------------------------------------------------------
// ListByRatingThreshold SQL generation tests
// ---------------------------------------------------------------------------
@@ -20,7 +147,6 @@ func TestRatingThreshold_BasicQuery(t *testing.T) {
Limit: 20,
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -55,24 +181,25 @@ func TestRatingThreshold_NoLimitOmitsClause(t *testing.T) {
}
}
func TestRatingThreshold_LibraryIDUsesSemiJoin(t *testing.T) {
func TestRatingThreshold_LibraryIDJoin(t *testing.T) {
libID := 42
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
LibraryID: &libID,
})
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected library EXISTS predicate, got:\n%s", query)
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected library id predicate at $2, got:\n%s", query)
if !strings.Contains(query, "mil.media_folder_id = $2") {
t.Fatalf("expected media_folder_id predicate at $2, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args (min + library id slice), got %d", len(args))
t.Fatalf("expected 2 args (min + library id), got %d", len(args))
}
if args[1] != 42 {
t.Fatalf("expected library id 42 at args[1], got %v", args[1])
}
assertIntSliceArg(t, args, 1, []int{42})
}
func TestRatingThreshold_AllowedLibraries(t *testing.T) {
@@ -83,20 +210,18 @@ func TestRatingThreshold_AllowedLibraries(t *testing.T) {
},
})
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($2)") {
t.Fatalf("expected allowed library ANY clause, got:\n%s", query)
if !strings.Contains(query, "mil.media_folder_id IN ($2, $3)") {
t.Fatalf("expected allowed library IN clause, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
if len(args) != 3 {
t.Fatalf("expected 3 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{1, 2})
}
func TestRatingThreshold_DisabledLibrariesUseItemLevelExclusion(t *testing.T) {
func TestRatingThreshold_DisabledLibraries(t *testing.T) {
query, args := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
@@ -104,24 +229,19 @@ func TestRatingThreshold_DisabledLibrariesUseItemLevelExclusion(t *testing.T) {
},
})
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_out.media_folder_id = ANY($2)") {
t.Fatalf("expected disabled library ANY clause, got:\n%s", query)
if !strings.Contains(query, "mil.media_folder_id NOT IN ($2)") {
t.Fatalf("expected disabled library NOT IN clause, got:\n%s", query)
}
if len(args) != 2 {
t.Fatalf("expected 2 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 1, []int{99})
}
func TestRatingThreshold_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
query, _ := buildRatingThresholdQuery(RatingFilter{
Min: 7.0,
Filter: AccessFilter{
AllowedLibraryIDs: []int{},
AllowedLibraryIDs: []int{}, // empty slice → early return
},
})
if query != "" {
@@ -141,7 +261,6 @@ func TestUnplayedHighRated_BasicQuery(t *testing.T) {
ProfileID: "profile-abc",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= $1") {
t.Fatalf("expected rating threshold predicate, got:\n%s", query)
}
@@ -182,14 +301,14 @@ func TestUnplayedHighRated_NoLimitOmitsClause(t *testing.T) {
}
}
func TestUnplayedHighRated_NoLibraryScope(t *testing.T) {
func TestUnplayedHighRated_NoLibraryJoin(t *testing.T) {
query, _ := buildUnplayedHighRatedQuery(UnplayedFilter{
MinRating: 7.0,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
}
}
@@ -203,17 +322,15 @@ func TestUnplayedHighRated_AllowedLibraries(t *testing.T) {
},
})
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
}
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestUnplayedHighRated_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -242,6 +359,8 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
t.Fatalf("expected content_rating = ANY filter, got:\n%s", query)
}
// args: minRating, userID, profileID, then content rating slice (one arg).
// Task 5.1 converted the rating ladder from IN ($N, $N+1, ...) to
// = ANY($N) with a single bound slice arg.
if len(args) != 4 {
t.Fatalf("expected exactly 4 args (minRating, userID, profileID, rating slice); got %d: %v", len(args), args)
}
@@ -251,6 +370,62 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
// ListForgottenFavorites SQL generation tests
// ---------------------------------------------------------------------------
// buildForgottenFavoritesQuery is the testable extraction of the SQL
// generation logic from ListForgottenFavorites.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= 7.0")
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t\t AND uwh.watched_at >= NOW() - ($"+itoa(argIdx+2)+" || ' days')::interval\n\t)")
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
func TestForgottenFavorites_BasicQuery(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
@@ -259,7 +434,6 @@ func TestForgottenFavorites_BasicQuery(t *testing.T) {
ProfileID: "profile-xyz",
})
assertQualifiedItemSelect(t, query)
if !strings.Contains(query, "mi.rating_imdb >= 7.0") {
t.Fatalf("expected 7.0 rating floor, got:\n%s", query)
}
@@ -303,14 +477,14 @@ func TestForgottenFavorites_NoLimitOmitsClause(t *testing.T) {
}
}
func TestForgottenFavorites_NoLibraryScope(t *testing.T) {
func TestForgottenFavorites_NoLibraryJoin(t *testing.T) {
query, _ := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
UserID: 1,
ProfileID: "p",
})
if strings.Contains(query, "media_item_libraries") {
t.Fatalf("expected no library scope when no library filter is set, got:\n%s", query)
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("expected no library join when no library filter is set, got:\n%s", query)
}
}
@@ -324,17 +498,15 @@ func TestForgottenFavorites_AllowedLibraries(t *testing.T) {
},
})
assertNoLibraryJoin(t, query)
if !strings.Contains(query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", query)
if !strings.Contains(query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join, got:\n%s", query)
}
if !strings.Contains(query, "mil_scope_in.media_folder_id = ANY($4)") {
t.Fatalf("expected allowed library ANY clause at $4, got:\n%s", query)
if !strings.Contains(query, "mil.media_folder_id IN ($4, $5)") {
t.Fatalf("expected allowed library IN clause at $4,$5, got:\n%s", query)
}
if len(args) != 4 {
t.Fatalf("expected 4 args, got %d: %v", len(args), args)
if len(args) != 5 {
t.Fatalf("expected 5 args, got %d: %v", len(args), args)
}
assertIntSliceArg(t, args, 3, []int{10, 11})
}
func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T) {
@@ -353,13 +525,14 @@ func TestForgottenFavorites_EmptyAllowedLibrariesReturnsEmptyQuery(t *testing.T)
func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 0,
LookbackDays: 0, // should default to 365
UserID: 1,
ProfileID: "p",
})
if !strings.Contains(query, "NOT EXISTS") {
t.Fatalf("expected NOT EXISTS subquery, got:\n%s", query)
}
// args[2] should be the defaulted lookback value (365).
if len(args) < 3 {
t.Fatalf("expected at least 3 args, got %d", len(args))
}
@@ -367,98 +540,3 @@ func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
t.Fatalf("expected lookback default 365, got %v", args[2])
}
}
func TestDiscoveryQueries_QualifyContentIDAndAvoidLibraryJoin(t *testing.T) {
restricted := AccessFilter{AllowedLibraryIDs: []int{1, 2}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: restricted}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: restricted}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: restricted}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertQualifiedItemSelect(t, tc.query)
assertNoLibraryJoin(t, tc.query)
if !strings.Contains(tc.query, "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_in") {
t.Fatalf("expected allowed library EXISTS predicate, got:\n%s", tc.query)
}
if strings.Contains(tc.query, "SELECT content_id,") {
t.Fatalf("SELECT list uses an unqualified content_id, got:\n%s", tc.query)
}
})
}
}
func TestDiscoveryQueries_DisabledLibrariesUseNotExists(t *testing.T) {
filter := AccessFilter{DisabledLibraryIDs: []int{9}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: filter}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: filter}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: filter}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assertNoLibraryJoin(t, tc.query)
assertDenyOnlyRequiresLibraryMembership(t, tc.query)
if !strings.Contains(tc.query, "NOT EXISTS (SELECT 1 FROM media_item_libraries mil_scope_out") {
t.Fatalf("expected disabled library NOT EXISTS predicate, got:\n%s", tc.query)
}
})
}
}
func assertQualifiedItemSelect(t *testing.T, query string) {
t.Helper()
if !strings.HasPrefix(query, "SELECT mi.content_id,") {
t.Fatalf("SELECT list is not qualified with the mi alias, got:\n%s", query)
}
}
func assertNoLibraryJoin(t *testing.T, query string) {
t.Helper()
if strings.Contains(query, "JOIN media_item_libraries") {
t.Fatalf("plain library JOIN can fan out discovery rows, got:\n%s", query)
}
}
func assertDenyOnlyRequiresLibraryMembership(t *testing.T, query string) {
t.Helper()
membershipPredicate := "EXISTS (SELECT 1 FROM media_item_libraries mil_scope_any WHERE mil_scope_any.content_id = mi.content_id)"
if !strings.Contains(query, membershipPredicate) {
t.Fatalf("deny-only library filters must require positive library membership, got:\n%s", query)
}
if strings.Contains(query, "mil_scope_any.media_folder_id") {
t.Fatalf("positive membership predicate should not bind a specific library, got:\n%s", query)
}
}
func assertIntSliceArg(t *testing.T, args []any, idx int, want []int) {
t.Helper()
if idx >= len(args) {
t.Fatalf("missing arg %d in %v", idx, args)
}
got, ok := args[idx].([]int)
if !ok {
t.Fatalf("arg %d has type %T, want []int", idx, args[idx])
}
if len(got) != len(want) {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("arg %d = %v, want %v", idx, got, want)
}
}
}
func mustQuery(query string, _ []any) string { return query }
-194
View File
@@ -1,194 +0,0 @@
package catalog
import (
"context"
"errors"
"fmt"
"sort"
"github.com/Silo-Server/silo-server/internal/secret"
)
// SettingsStore is the read/write surface over server_settings shared by the
// raw *ServerSettingsRepo and the *EncryptedSettingsRepo decorator. Consumers
// depend on this interface (rather than the concrete repo) so they
// transparently gain at-rest encryption for sensitive keys once the decorator
// is wired in.
type SettingsStore interface {
Get(ctx context.Context, key string) (string, error)
Set(ctx context.Context, key, value string) error
GetAll(ctx context.Context) (map[string]string, error)
}
// SensitiveSettingKeys is the single source of truth for which server_settings
// keys hold secrets. It drives BOTH at-rest encryption (the decorator encrypts
// these on write, decrypts them on read) AND admin-API redaction, so the two
// can never drift apart.
//
// It was audited from the config loader's real inputs
// (internal/config/db_loader.go), the admin redaction map, and the other known
// server_settings consumers — the Audiobookshelf-compat JWT secret
// (internal/audiobooks/config.go), the watch-sync OAuth client config
// (internal/watchsync/service.go), and the legacy single-instance arr keys —
// including the legacy aliases still read as fallbacks (s3.operational_*,
// recommendations.openai_api_key) and redis.sentinel_password, which the old
// redaction map omitted.
//
// Adding a key here is only safe when every reader of that key goes through a
// cipher-aware path (the decorator, or a repo that calls
// secret.Cipher.DecryptIfEncrypted with secret.SettingsAAD). A raw SQL reader
// would otherwise see ciphertext after the backfill.
var SensitiveSettingKeys = map[string]bool{
// Core auth signing secret — no longer the encryption root (SECRET_KEY is),
// and now itself encrypted at rest under SECRET_KEY.
"auth.jwt_secret": true,
// Audiobookshelf-compat HMAC signing key (generated + persisted as hex).
"audiobooks.abs.jwt_secret": true,
// S3 — public assets, private internal, user DB, plus the legacy
// "operational" aliases db_loader still reads as fallbacks (L155-192;
// migration 086 copied but did not delete them).
"s3.public_access_key": true,
"s3.public_secret_key": true,
"s3.public_token_secret": true,
"s3.private_access_key": true,
"s3.private_secret_key": true,
"s3.user_db_access_key": true,
"s3.user_db_secret_key": true,
"s3.operational_access_key": true,
"s3.operational_secret_key": true,
"s3.operational_token_secret": true,
// Redis — url may embed credentials (redis://:pass@host); sentinel_password
// is read at db_loader L334 and was MISSING from the old redaction map.
"redis.url": true,
"redis.sentinel_password": true,
// Metadata / list-provider API keys.
"tmdb.api_key": true,
"mdblist.api_key": true,
"introdb.api_key": true,
// Subtitle AI (on-demand translation) API key.
"subtitle_ai.api_key": true,
// Recommendations embedding auth (+ legacy openai alias at db_loader L409).
"recommendations.embedding_auth_token": true,
"recommendations.openai_api_key": true,
// Watch-sync OAuth client credentials. The client_id entries are low-value
// but kept as a harmless safe superset and to match existing redaction.
"watchsync.trakt.client_id": true,
"watchsync.trakt.client_secret": true,
"watchsync.simkl.client_id": true,
"watchsync.simkl.client_secret": true,
// Legacy single-instance arr keys (superseded by per-integration rows, but
// still referenced by older request_integrations rows until backfilled).
"requests.radarr.api_key": true,
"requests.sonarr.api_key": true,
}
// EncryptedSettingsRepo decorates a raw settings store, transparently
// encrypting the SensitiveSettingKeys on write and decrypting them on read.
// config.LoadFromDB and every other consumer keep seeing plaintext while the
// values rest as ciphertext in the database, GCM-bound to their key.
type EncryptedSettingsRepo struct {
// inner is the RAW store (no encryption). It must never be another
// EncryptedSettingsRepo, or writes would double-encrypt.
inner SettingsStore
cipher *secret.Cipher
}
// NewEncryptedSettingsRepo wraps a raw settings store with the cipher. Pass the
// concrete *ServerSettingsRepo (or an in-memory fake in tests) as inner.
func NewEncryptedSettingsRepo(inner SettingsStore, cipher *secret.Cipher) *EncryptedSettingsRepo {
return &EncryptedSettingsRepo{inner: inner, cipher: cipher}
}
var _ SettingsStore = (*EncryptedSettingsRepo)(nil)
// Set encrypts a sensitive, non-empty value (AAD-bound to its key) before
// delegating to the raw store; non-sensitive keys and empty values delegate
// unchanged. Encrypting an empty value is a no-op, so an absent secret stays an
// empty column rather than an envelope.
func (r *EncryptedSettingsRepo) Set(ctx context.Context, key, value string) error {
if SensitiveSettingKeys[key] && value != "" {
ct, err := r.cipher.Encrypt(value, secret.SettingsAAD(key))
if err != nil {
return fmt.Errorf("encrypt setting %q: %w", key, err)
}
value = ct
}
return r.inner.Set(ctx, key, value)
}
// Get reads a value and applies the read-path contract: legacy plaintext passes
// through, an enc:v1: value is decrypted, and a corrupt ciphertext errors.
func (r *EncryptedSettingsRepo) Get(ctx context.Context, key string) (string, error) {
value, err := r.inner.Get(ctx, key)
if err != nil {
return "", err
}
out, err := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if err != nil {
return "", fmt.Errorf("decrypt setting %q: %w", key, err)
}
return out, nil
}
// GetAll reads every setting and decrypts any enc:v1: value in place, so
// callers (notably config.LoadFromDB) receive a fully plaintext map.
func (r *EncryptedSettingsRepo) GetAll(ctx context.Context) (map[string]string, error) {
all, err := r.inner.GetAll(ctx)
if err != nil {
return nil, err
}
for key, value := range all {
out, derr := r.cipher.DecryptIfEncrypted(value, secret.SettingsAAD(key))
if derr != nil {
return nil, fmt.Errorf("decrypt setting %q: %w", key, derr)
}
all[key] = out
}
return all, nil
}
// BackfillSensitiveSettings encrypts any plaintext value currently stored under
// a sensitive key. It is idempotent (already-encrypted and empty values are
// skipped) and best-effort: per-key failures are collected and returned but do
// not abort the sweep, so one unreadable row cannot block startup. Returns the
// number of keys newly encrypted.
func (r *EncryptedSettingsRepo) BackfillSensitiveSettings(ctx context.Context) (int, error) {
keys := make([]string, 0, len(SensitiveSettingKeys))
for key := range SensitiveSettingKeys {
keys = append(keys, key)
}
sort.Strings(keys) // deterministic order for logs and tests
encrypted := 0
var errs []error
for _, key := range keys {
// Read the RAW inner value (not through this decorator) so we can tell
// plaintext from an already-encrypted envelope.
raw, err := r.inner.Get(ctx, key)
if err != nil {
errs = append(errs, fmt.Errorf("read %q: %w", key, err))
continue
}
if raw == "" || secret.IsEncrypted(raw) {
continue
}
ct, err := r.cipher.Encrypt(raw, secret.SettingsAAD(key))
if err != nil {
errs = append(errs, fmt.Errorf("encrypt %q: %w", key, err))
continue
}
if err := r.inner.Set(ctx, key, ct); err != nil {
errs = append(errs, fmt.Errorf("write %q: %w", key, err))
continue
}
encrypted++
}
return encrypted, errors.Join(errs...)
}
@@ -1,210 +0,0 @@
package catalog
import (
"context"
"crypto/rand"
"testing"
"github.com/Silo-Server/silo-server/internal/secret"
)
// memSettings is an in-memory raw SettingsStore for DB-free decorator tests.
type memSettings struct{ m map[string]string }
func newMemSettings() *memSettings { return &memSettings{m: map[string]string{}} }
func (s *memSettings) Get(_ context.Context, key string) (string, error) { return s.m[key], nil }
func (s *memSettings) Set(_ context.Context, key, value string) error {
s.m[key] = value
return nil
}
func (s *memSettings) GetAll(_ context.Context) (map[string]string, error) {
out := make(map[string]string, len(s.m))
for k, v := range s.m {
out[k] = v
}
return out, nil
}
func newCipher(t *testing.T) *secret.Cipher {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return c
}
func pickSensitiveKey() string {
for k := range SensitiveSettingKeys {
return k
}
return "auth.jwt_secret"
}
func TestEncryptedSettings_SensitiveRoundTrip(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "tmdb.api_key"
const plain = "tmdb-secret-key-value"
if err := dec.Set(ctx, key, plain); err != nil {
t.Fatalf("Set: %v", err)
}
// Raw store must hold ciphertext.
if got := raw.m[key]; !secret.IsEncrypted(got) {
t.Fatalf("raw store value = %q, want enc:v1: ciphertext", got)
}
// Decorator Get must return plaintext.
got, err := dec.Get(ctx, key)
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != plain {
t.Fatalf("Get = %q, want %q", got, plain)
}
}
func TestEncryptedSettings_NonSensitivePassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
const key = "server.log_level"
if SensitiveSettingKeys[key] {
t.Fatalf("precondition: %q must be non-sensitive", key)
}
if err := dec.Set(ctx, key, "debug"); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m[key]; got != "debug" {
t.Fatalf("non-sensitive raw value = %q, want plaintext 'debug'", got)
}
}
func TestEncryptedSettings_LegacyPlaintextPassthrough(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
// Simulate a pre-encryption row: a sensitive key holding raw plaintext.
raw.m["tmdb.api_key"] = "legacy-plaintext"
got, err := dec.Get(ctx, "tmdb.api_key")
if err != nil {
t.Fatalf("Get: %v", err)
}
if got != "legacy-plaintext" {
t.Fatalf("legacy plaintext Get = %q, want pass-through", got)
}
}
func TestEncryptedSettings_EmptySensitiveStaysEmpty(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", ""); err != nil {
t.Fatalf("Set: %v", err)
}
if got := raw.m["tmdb.api_key"]; got != "" {
t.Fatalf("empty sensitive value stored as %q, want empty (no envelope)", got)
}
}
func TestEncryptedSettings_GetAllDecrypts(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
if err := dec.Set(ctx, "tmdb.api_key", "k1"); err != nil {
t.Fatalf("Set: %v", err)
}
if err := dec.Set(ctx, "server.mode", "integrated"); err != nil {
t.Fatalf("Set: %v", err)
}
all, err := dec.GetAll(ctx)
if err != nil {
t.Fatalf("GetAll: %v", err)
}
if all["tmdb.api_key"] != "k1" {
t.Fatalf("GetAll sensitive = %q, want decrypted 'k1'", all["tmdb.api_key"])
}
if all["server.mode"] != "integrated" {
t.Fatalf("GetAll non-sensitive = %q", all["server.mode"])
}
}
// TestSensitiveSettingKeys_Audited locks the audited allowlist: a dropped key is
// a plaintext leak (and breaks redaction), so the critical secrets must stay
// present, while values that are NOT secrets must stay absent (encrypting them
// would corrupt config reads).
func TestSensitiveSettingKeys_Audited(t *testing.T) {
mustHave := []string{
"auth.jwt_secret",
"audiobooks.abs.jwt_secret",
"s3.public_secret_key",
"s3.operational_secret_key", // legacy alias still read as a fallback
"redis.sentinel_password", // was missing from the old redaction map
"recommendations.openai_api_key",
"tmdb.api_key",
"requests.radarr.api_key",
"requests.sonarr.api_key",
"watchsync.trakt.client_secret",
}
for _, k := range mustHave {
if !SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys missing critical secret key %q", k)
}
}
mustNotHave := []string{
"server.mode",
"server.log_level",
"database.max_connections",
"jellyfin_compat.server_id",
}
for _, k := range mustNotHave {
if SensitiveSettingKeys[k] {
t.Errorf("SensitiveSettingKeys must not contain non-secret key %q", k)
}
}
}
func TestEncryptedSettings_BackfillIdempotent(t *testing.T) {
ctx := context.Background()
raw := newMemSettings()
dec := NewEncryptedSettingsRepo(raw, newCipher(t))
key := pickSensitiveKey()
raw.m[key] = "plaintext-secret" // legacy row
raw.m["server.mode"] = "integrated" // non-sensitive, must be left alone
n, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill: %v", err)
}
if n != 1 {
t.Fatalf("backfill encrypted %d, want 1", n)
}
if !secret.IsEncrypted(raw.m[key]) {
t.Fatalf("after backfill raw[%q] = %q, want ciphertext", key, raw.m[key])
}
if raw.m["server.mode"] != "integrated" {
t.Fatalf("backfill mutated non-sensitive key: %q", raw.m["server.mode"])
}
// Decryptable back to the original.
if got, _ := dec.Get(ctx, key); got != "plaintext-secret" {
t.Fatalf("post-backfill Get = %q, want original", got)
}
// Second run is a no-op.
n2, err := dec.BackfillSensitiveSettings(ctx)
if err != nil {
t.Fatalf("backfill 2: %v", err)
}
if n2 != 0 {
t.Fatalf("second backfill encrypted %d, want 0 (no double-encrypt)", n2)
}
}
-13
View File
@@ -3,7 +3,6 @@ package catalog
import (
"context"
"errors"
"strings"
"testing"
"time"
@@ -137,15 +136,3 @@ func TestDeleteInBatchesReturnsError(t *testing.T) {
t.Fatalf("expected sentinel, got %v", err)
}
}
func TestLateOrphanCleanupUsesProvisionalSafetyPredicate(t *testing.T) {
conditions := normalizePredicateSQL(orphanedProvisionalMediaItemConditions)
for name, query := range map[string]string{
"collect": collectOrphanedProvisionalIDsByContentIDSQL,
"delete": deleteOrphanedProvisionalIDsByContentIDSQL,
} {
if !strings.Contains(normalizePredicateSQL(query), conditions) {
t.Fatalf("%s late orphan query does not use provisional safety predicate", name)
}
}
}
+22 -142
View File
@@ -563,6 +563,16 @@ func (r *FolderRepository) DeleteWithStats(
}
}
// Filter accumulated image dirs once, now that orphans are gone, against
// any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
}
stats.OrphanedImageDirs = filtered
}
// Phase 2: delete this folder's media_files (folder-tied, not item-tied).
if progress != nil {
progress(orphanTotal, orphanTotal, "Deleting media files")
@@ -582,29 +592,22 @@ func (r *FolderRepository) DeleteWithStats(
}
// Phase 3: delete remaining folder memberships (shared items kept; only the
// membership in this folder is removed). Deleting the memberships with
// RETURNING lets us catch skeleton items that a matcher created after the
// phase-1 orphan sweep but before the delete job reached this phase.
// membership in this folder is removed).
if progress != nil {
progress(orphanTotal, orphanTotal, "Removing library memberships")
}
lateOrphans, lateImageDirs, err := r.deleteFolderMemberships(ctx, id)
if err != nil {
return nil, fmt.Errorf("deleting media item links: %w", err)
}
stats.OrphanedItems += lateOrphans
for _, d := range lateImageDirs {
rawDirs[d] = struct{}{}
}
// Filter accumulated image dirs once, now that all item orphans are gone,
// against any surviving content. Empty deleting-set means "exclude nothing".
if len(rawDirs) > 0 {
filtered, err := filterUnreferencedImageDirs(ctx, r.pool, dirSetToSlice(rawDirs), []string{})
if err != nil {
return nil, err
if _, err := deleteInBatches(ctx, folderChildDeleteBatch, func(ctx context.Context) (int64, error) {
tag, e := r.pool.Exec(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)`, id, folderChildDeleteBatch)
if e != nil {
return 0, e
}
stats.OrphanedImageDirs = filtered
return tag.RowsAffected(), nil
}); err != nil {
return nil, fmt.Errorf("deleting media item links: %w", err)
}
// Phase 4: delete the now-lightweight folder row. Tolerate 0 rows so a
@@ -653,129 +656,6 @@ func (r *FolderRepository) collectOrphanBatch(ctx context.Context, folderID, lim
return ids, nil
}
// deleteFolderMemberships removes this folder's remaining item memberships and
// deletes any content that becomes orphaned as a result. This is intentionally
// separate from the phase-1 orphan sweep because in-flight matchers can create
// new skeleton memberships after phase 1 has already drained.
func (r *FolderRepository) deleteFolderMemberships(ctx context.Context, folderID int) (int, []string, error) {
var orphaned int
var imageDirs []string
for {
var contentIDs []string
if err := retryOnDeadlock(ctx, func() error {
ids, err := r.deleteFolderMembershipBatch(ctx, folderID, folderChildDeleteBatch)
if err != nil {
return err
}
contentIDs = ids
return nil
}); err != nil {
return orphaned, imageDirs, err
}
if len(contentIDs) > 0 {
deleted, dirs, err := r.deleteOrphanedItemsByContentID(ctx, contentIDs)
if err != nil {
return orphaned, imageDirs, err
}
orphaned += deleted
imageDirs = append(imageDirs, dirs...)
}
if len(contentIDs) < folderChildDeleteBatch {
return orphaned, imageDirs, nil
}
}
}
func (r *FolderRepository) deleteFolderMembershipBatch(ctx context.Context, folderID, limit int) ([]string, error) {
rows, err := r.pool.Query(ctx, `
DELETE FROM media_item_libraries
WHERE ctid IN (
SELECT ctid FROM media_item_libraries WHERE media_folder_id = $1 LIMIT $2
)
RETURNING content_id`, folderID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var contentIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning deleted membership content_id: %w", err)
}
contentIDs = append(contentIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return contentIDs, nil
}
const collectOrphanedProvisionalIDsByContentIDSQL = `
SELECT mi.content_id
FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
const deleteOrphanedProvisionalIDsByContentIDSQL = `
DELETE FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedProvisionalMediaItemConditions
func (r *FolderRepository) deleteOrphanedItemsByContentID(ctx context.Context, contentIDs []string) (int, []string, error) {
if len(contentIDs) == 0 {
return 0, nil, nil
}
orphanIDs, err := r.collectOrphanedProvisionalIDsByContentID(ctx, contentIDs)
if err != nil {
return 0, nil, err
}
if len(orphanIDs) == 0 {
return 0, nil, nil
}
imageDirs, err := collectImageDirs(ctx, r.pool, orphanIDs)
if err != nil {
return 0, nil, err
}
var deleted int64
if err := retryOnDeadlock(ctx, func() error {
tag, err := r.pool.Exec(ctx, deleteOrphanedProvisionalIDsByContentIDSQL, orphanIDs)
if err != nil {
return err
}
deleted = tag.RowsAffected()
return nil
}); err != nil {
return 0, nil, err
}
return int(deleted), imageDirs, nil
}
func (r *FolderRepository) collectOrphanedProvisionalIDsByContentID(ctx context.Context, contentIDs []string) ([]string, error) {
rows, err := r.pool.Query(ctx, collectOrphanedProvisionalIDsByContentIDSQL, contentIDs)
if err != nil {
return nil, err
}
defer rows.Close()
var orphanIDs []string
for rows.Next() {
var contentID string
if err := rows.Scan(&contentID); err != nil {
return nil, fmt.Errorf("scanning orphan content_id: %w", err)
}
orphanIDs = append(orphanIDs, contentID)
}
if err := rows.Err(); err != nil {
return nil, err
}
return orphanIDs, nil
}
// dirSetToSlice returns the keys of set as a slice (nil if empty).
func dirSetToSlice(set map[string]struct{}) []string {
if len(set) == 0 {
+45 -96
View File
@@ -35,26 +35,6 @@ func NewItemRepository(pool *pgxpool.Pool) *ItemRepository {
return &ItemRepository{pool: pool}
}
// GetPosterPath returns the current poster path for a media item. Missing or
// NULL poster values are returned as an empty string.
func (r *ItemRepository) GetPosterPath(ctx context.Context, contentID string) (string, error) {
if r == nil || r.pool == nil {
return "", ErrItemNotFound
}
var posterPath string
if err := r.pool.QueryRow(ctx, `
SELECT COALESCE(poster_path, '')
FROM media_items
WHERE content_id = $1
`, contentID).Scan(&posterPath); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return "", ErrItemNotFound
}
return "", fmt.Errorf("getting media item poster path: %w", err)
}
return posterPath, nil
}
// overviewMatchFloor is the minimum ts_rank_cd score an overview-only match
// must achieve to be returned when no title FTS match exists in the candidate
// set. The overview tsvector is built without setweight(), so PostgreSQL's
@@ -65,87 +45,60 @@ func (r *ItemRepository) GetPosterPath(ctx context.Context, contentID string) (s
// incidental one-mention hits that flooded results before.
const overviewMatchFloor = 0.15
// itemColumnNames lists, in scan order, every column selected by media_items
// item queries. Shared by itemColumns, qualifiedItemColumns,
// qualifiedListItemColumns, and qualifiedItemColumnRefs so the select lists
// can never drift from each other or from scanItem.
var itemColumnNames = []string{
"content_id", "type", "title", "sort_title", "default_metadata_language", "original_title", "year", "genres",
"content_rating", "runtime", "overview", "tagline",
"rating_imdb", "rating_tmdb", "rating_rt_critic", "rating_rt_audience",
"imdb_id", "tmdb_id", "tvdb_id",
"poster_path", "poster_thumbhash", "backdrop_path", "backdrop_thumbhash", "logo_path",
"metadata_s3_path", "metadata_etag", "season_count",
"studios", "networks", "countries", "keywords", "original_language", "release_date::text", "first_air_date", "last_air_date", "air_time", "air_timezone",
"show_status",
"matched_at", "last_refreshed", "refresh_failures",
"episode_metadata_incomplete", "episode_metadata_last_checked_at", "locked_fields", "status", "created_at", "updated_at",
}
// nullableStringItemColumns are media_items columns that may hold NULL but
// scan into plain (non-pointer) string fields on models.MediaItem, so select
// lists coalesce them to ”.
var nullableStringItemColumns = map[string]bool{
"poster_path": true,
"poster_thumbhash": true,
"backdrop_path": true,
"backdrop_thumbhash": true,
"logo_path": true,
"metadata_s3_path": true,
"metadata_etag": true,
}
// itemColumnExpr renders one select-list entry for col, qualified with alias
// when non-empty. Nullable string columns are coalesced to ” and aliased
// back to their own name so queries that wrap the select list in a CTE or
// subquery can still reference the column by name.
func itemColumnExpr(alias, col string) string {
qualified := col
if alias != "" {
qualified = alias + "." + col
}
if nullableStringItemColumns[col] {
return fmt.Sprintf("COALESCE(%s, '') AS %s", qualified, col)
}
return qualified
}
func joinItemColumns(alias string) string {
exprs := make([]string, len(itemColumnNames))
for i, col := range itemColumnNames {
exprs[i] = itemColumnExpr(alias, col)
}
return strings.Join(exprs, ", ")
}
// itemColumns is the list of columns returned by all SELECT queries on media_items.
var itemColumns = joinItemColumns("")
const itemColumns = `content_id, type, title, sort_title, default_metadata_language, original_title, year, genres,
content_rating, runtime, overview, tagline,
rating_imdb, rating_tmdb, rating_rt_critic, rating_rt_audience,
imdb_id, tmdb_id, tvdb_id,
poster_path, poster_thumbhash, backdrop_path, backdrop_thumbhash, logo_path,
metadata_s3_path, metadata_etag, season_count,
studios, networks, countries, keywords, original_language, release_date::text, first_air_date, last_air_date, air_time, air_timezone,
show_status,
matched_at, last_refreshed, refresh_failures,
episode_metadata_incomplete, episode_metadata_last_checked_at, locked_fields, status, created_at, updated_at`
func qualifiedItemColumns(alias string) string {
return joinItemColumns(alias)
}
// qualifiedItemColumnRefs renders plain alias-qualified column references
// without COALESCE or AS aliases, for contexts like GROUP BY where output
// aliases are invalid.
func qualifiedItemColumnRefs(alias string) string {
refs := make([]string, len(itemColumnNames))
for i, col := range itemColumnNames {
refs[i] = alias + "." + col
cols := []string{
"content_id", "type", "title", "sort_title", "default_metadata_language", "original_title", "year", "genres",
"content_rating", "runtime", "overview", "tagline",
"rating_imdb", "rating_tmdb", "rating_rt_critic", "rating_rt_audience",
"imdb_id", "tmdb_id", "tvdb_id",
"poster_path", "poster_thumbhash", "backdrop_path", "backdrop_thumbhash", "logo_path",
"metadata_s3_path", "metadata_etag", "season_count",
"studios", "networks", "countries", "keywords", "original_language", "release_date::text", "first_air_date", "last_air_date", "air_time", "air_timezone",
"show_status",
"matched_at", "last_refreshed", "refresh_failures",
"episode_metadata_incomplete", "episode_metadata_last_checked_at", "locked_fields", "status", "created_at", "updated_at",
}
return strings.Join(refs, ", ")
prefixed := make([]string, len(cols))
for i, col := range cols {
prefixed[i] = alias + "." + col
}
return strings.Join(prefixed, ", ")
}
func qualifiedListItemColumns(alias string) string {
exprs := make([]string, len(itemColumnNames))
for i, col := range itemColumnNames {
cols := []string{
"content_id", "type", "title", "sort_title", "default_metadata_language", "original_title", "year", "genres",
"content_rating", "runtime", "overview", "tagline",
"rating_imdb", "rating_tmdb", "rating_rt_critic", "rating_rt_audience",
"imdb_id", "tmdb_id", "tvdb_id",
"poster_path", "poster_thumbhash", "backdrop_path", "backdrop_thumbhash", "logo_path",
"metadata_s3_path", "metadata_etag", "season_count",
"studios", "networks", "countries", "keywords", "original_language", "release_date::text", "first_air_date", "last_air_date", "air_time", "air_timezone",
"show_status",
"matched_at", "last_refreshed", "refresh_failures",
"episode_metadata_incomplete", "episode_metadata_last_checked_at", "locked_fields", "status", "created_at", "updated_at",
}
prefixed := make([]string, len(cols))
for i, col := range cols {
if col == "last_air_date" {
exprs[i] = effectiveLastAirDateExpr(alias) + " AS last_air_date"
prefixed[i] = effectiveLastAirDateExpr(alias)
continue
}
exprs[i] = itemColumnExpr(alias, col)
prefixed[i] = alias + "." + col
}
return strings.Join(exprs, ", ")
return strings.Join(prefixed, ", ")
}
// scanItem scans a single row into a *models.MediaItem.
@@ -1003,12 +956,8 @@ func (r *ItemRepository) buildSearchSQL(query string, itemTypes []string, limit,
)
// Use qualified column names inside the CTE to avoid ambiguity when
// the FROM clause includes a JOIN to media_item_libraries. The select
// list aliases coalesced columns back to their own names (poster_path
// etc.) so the outer query can re-reference them; GROUP BY needs the
// raw references because output aliases are invalid there.
// the FROM clause includes a JOIN to media_item_libraries.
qualifiedCols := qualifiedItemColumns("mi")
groupByCols := qualifiedItemColumnRefs("mi")
scoredCTE := fmt.Sprintf(`
WITH scored AS (
SELECT
@@ -1035,7 +984,7 @@ func (r *ItemRepository) buildSearchSQL(query string, itemTypes []string, limit,
%s
GROUP BY %s
)
`, qualifiedCols, exactTitleMatch, contiguousTitleMatch, yearIdx, yearIdx, titleVector, titleQuery, overviewVector, phraseIdx, titleVector, phraseIdx, fromClause, whereClause, groupByCols)
`, qualifiedCols, exactTitleMatch, contiguousTitleMatch, yearIdx, yearIdx, titleVector, titleQuery, overviewVector, phraseIdx, titleVector, phraseIdx, fromClause, whereClause, qualifiedCols)
// COUNT(*) OVER () runs after the GROUP BY in the scored CTE collapses
// duplicates from the library JOIN, so the window count preserves the
-57
View File
@@ -2,7 +2,6 @@ package catalog
import (
"fmt"
"sort"
"strings"
"testing"
)
@@ -368,59 +367,3 @@ func TestItemRepo_Search_NormalizesTsqueryInput(t *testing.T) {
t.Fatalf("overview arm should NOT wrap $1 in normalize_search_text; got:\n%s", sql)
}
}
// TestItemRepo_Search_ScoredCTEExposesItemColumnNames guards the contract
// between the scored CTE and the outer SELECT in buildSearchSQL. The CTE
// projects qualifiedItemColumns("mi") and the outer query re-selects those
// columns by name via itemColumns, so every entry must expose its own column
// name as the output name. Postgres names an unaliased expression like
// COALESCE(mi.poster_path, ”) "coalesce", which breaks the outer reference
// with SQLSTATE 42703 (column "poster_path" does not exist) — exactly how
// search returned 500s when the nullable-string COALESCE wrappers first
// landed without AS aliases.
func TestItemRepo_Search_ScoredCTEExposesItemColumnNames(t *testing.T) {
exposed := map[string]bool{}
for _, part := range splitTopLevelSQLCommas(qualifiedItemColumns("mi")) {
exposed[sqlOutputColumnName(part)] = true
}
var missing []string
for _, col := range itemColumnNames {
name := trailingSQLIdent(col)
if !exposed[name] {
missing = append(missing, name)
}
}
if len(missing) > 0 {
sort.Strings(missing)
t.Fatalf("qualifiedItemColumns(\"mi\") does not expose output columns %v; "+
"alias each expression back to its column name (e.g. COALESCE(mi.x, '') AS x) "+
"or the search CTE's outer SELECT fails with SQLSTATE 42703", missing)
}
}
// TestItemRepo_Search_GroupByHasNoOutputAliases asserts the scored CTE's
// GROUP BY uses raw column references: AS aliases are valid in a select list
// but are a syntax error inside GROUP BY, so the CTE must not reuse the
// aliased projection there.
func TestItemRepo_Search_GroupByHasNoOutputAliases(t *testing.T) {
repo := &ItemRepository{}
dataSQL, countSQL, _ := repo.buildSearchSQL("avatar", []string{"movie"}, 20, 0, AccessFilter{})
for _, sql := range []string{dataSQL, countSQL} {
idx := strings.Index(sql, "GROUP BY")
if idx < 0 {
t.Fatalf("expected GROUP BY in scored CTE; got:\n%s", sql)
}
clause := sql[idx:]
if end := strings.Index(clause, ")"); end >= 0 {
clause = clause[:end]
}
if strings.Contains(clause, " AS ") {
t.Fatalf("GROUP BY must not contain output aliases; got:\n%s", clause)
}
if strings.Contains(clause, "COALESCE") {
t.Fatalf("GROUP BY should group by raw columns, not COALESCE expressions; got:\n%s", clause)
}
}
}
-117
View File
@@ -1,117 +0,0 @@
package catalog
import (
"net/url"
"reflect"
"strings"
"testing"
)
// TestMediaScopeItemTypes pins the expansion of group scopes: "video" covers
// the video-side media_items types so search/browse can offer a
// "Movies & Series vs Audiobooks" split without enumerating types per caller.
func TestMediaScopeItemTypes(t *testing.T) {
cases := []struct {
scope string
want []string
}{
{"", nil},
{"movie", []string{"movie"}},
{"audiobook", []string{"audiobook"}},
{"video", []string{"movie", "series"}},
{" Video ", []string{"movie", "series"}},
}
for _, tc := range cases {
if got := MediaScopeItemTypes(tc.scope); !reflect.DeepEqual(got, tc.want) {
t.Errorf("MediaScopeItemTypes(%q) = %v, want %v", tc.scope, got, tc.want)
}
}
}
func TestMediaScopeMatchesItemType(t *testing.T) {
cases := []struct {
scope string
itemType string
want bool
}{
{"", "audiobook", true},
{"video", "movie", true},
{"video", "series", true},
{"video", "audiobook", false},
{"audiobook", "audiobook", true},
{"movie", "series", false},
}
for _, tc := range cases {
if got := MediaScopeMatchesItemType(tc.scope, tc.itemType); got != tc.want {
t.Errorf("MediaScopeMatchesItemType(%q, %q) = %v, want %v", tc.scope, tc.itemType, got, tc.want)
}
}
}
// TestParseCatalogRequest_VideoMediaScope asserts ?type=video parses into the
// query definition's media scope, and that catalogSearchAccess expands it to
// the video item types for the direct search path.
func TestParseCatalogRequest_VideoMediaScope(t *testing.T) {
req, err := ParseCatalogRequest(url.Values{
"source": {"query"},
"q": {"the rookie"},
"type": {"video"},
})
if err != nil {
t.Fatalf("ParseCatalogRequest: %v", err)
}
if req.Query.MediaScope != "video" {
t.Fatalf("expected media scope video, got %q", req.Query.MediaScope)
}
_, itemTypes, earlyEmpty := catalogSearchAccess(req, AccessFilter{})
if earlyEmpty {
t.Fatal("unexpected early empty")
}
if !reflect.DeepEqual(itemTypes, []string{"movie", "series"}) {
t.Fatalf("expected video scope to expand to movie+series, got %v", itemTypes)
}
}
// TestPreviewPage_VideoScopeUsesTypeAny asserts the preview/query-executor
// path renders a multi-type condition for the video group scope.
func TestPreviewPage_VideoScopeUsesTypeAny(t *testing.T) {
sql, args, err := (&QueryExecutor{}).buildPreviewPageSQL(
QueryDefinition{
MediaScope: "video",
Sort: QuerySort{Field: "title", Order: "asc"},
},
AccessFilter{},
20,
0,
true,
)
if err != nil {
t.Fatalf("buildPreviewPageSQL error: %v", err)
}
if !strings.Contains(sql, "mi.type = ANY(") {
t.Fatalf("expected mi.type = ANY(...) for video scope, got %s", sql)
}
found := false
for _, arg := range args {
if types, ok := arg.([]string); ok && reflect.DeepEqual(types, []string{"movie", "series"}) {
found = true
}
}
if !found {
t.Fatalf("expected movie+series type arg, got %v", args)
}
}
// TestQueryDefinitionValidate_VideoScope asserts "video" passes definition
// validation alongside the single-type scopes.
func TestQueryDefinitionValidate_VideoScope(t *testing.T) {
def := QueryDefinition{MediaScope: "video"}
if err := def.Validate(); err != nil {
t.Fatalf("expected video media scope to validate, got %v", err)
}
bad := QueryDefinition{MediaScope: "podcast"}
if err := bad.Validate(); err == nil {
t.Fatal("expected invalid media scope to fail validation")
}
}
-200
View File
@@ -1,200 +0,0 @@
package catalog
import (
"context"
"fmt"
"github.com/jackc/pgx/v5/pgxpool"
)
const defaultOrphanedProvisionalCleanupBatchSize = 1000
const orphanedMediaItemSafetyConditions = `NOT EXISTS (
SELECT 1 FROM public.media_item_libraries mil
WHERE mil.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.media_files mf
WHERE mf.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.episodes e
WHERE e.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.seasons s
WHERE s.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.library_collection_items lci
WHERE lci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_bookmarks ab
WHERE ab.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_playback_sessions aps
WHERE aps.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.abs_rss_feeds arf
WHERE arf.library_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.downloads d
WHERE d.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.playback_history_admin pha
WHERE pha.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_bindings psib
WHERE psib.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.plex_sync_item_state psis
WHERE psis.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.podcast_feeds pf
WHERE pf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_favorites uf
WHERE uf.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_downloads ud
WHERE ud.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_history_hidden_items uhhi
WHERE uhhi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid
WHERE uhid.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_home_item_dismissals uhid_series
WHERE uhid_series.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_audio_preferences uap
WHERE uap.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_personal_collection_items upci
WHERE upci.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_ratings ur
WHERE ur.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_series_playback_preferences uspp
WHERE uspp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_subtitle_preferences usp
WHERE usp.series_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_history uwh
WHERE uwh.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watch_progress uwp
WHERE uwp.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.user_watchlist uwl
WHERE uwl.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_favorite_items wpfi
WHERE wpfi.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_history_exports wphe
WHERE wphe.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_provider_scrobble_sessions wpss
WHERE wpss.media_item_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_rooms wtr
WHERE wtr.selected_content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.watch_together_suggestions wts
WHERE wts.content_id = mi.content_id
)
AND NOT EXISTS (
SELECT 1 FROM public.webhook_sync_item_state wsis
WHERE wsis.media_item_id = mi.content_id
)`
const orphanedProvisionalMediaItemConditions = `mi.status IN ('pending', 'unmatched', 'ambiguous')
AND ` + orphanedMediaItemSafetyConditions
const orphanedProvisionalMediaItemPredicate = `
WHERE ` + orphanedProvisionalMediaItemConditions
type OrphanedProvisionalCleanupStats struct {
Candidates int
Deleted int
}
type OrphanedProvisionalCleaner struct {
pool *pgxpool.Pool
}
func NewOrphanedProvisionalCleaner(pool *pgxpool.Pool) *OrphanedProvisionalCleaner {
return &OrphanedProvisionalCleaner{pool: pool}
}
func (c *OrphanedProvisionalCleaner) Cleanup(ctx context.Context, batchSize int) (OrphanedProvisionalCleanupStats, error) {
var stats OrphanedProvisionalCleanupStats
if c == nil || c.pool == nil {
return stats, fmt.Errorf("orphaned provisional cleanup is not configured")
}
if batchSize <= 0 {
batchSize = defaultOrphanedProvisionalCleanupBatchSize
}
if err := c.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM public.media_items mi `+orphanedProvisionalMediaItemPredicate,
).Scan(&stats.Candidates); err != nil {
return stats, fmt.Errorf("counting orphaned provisional media items: %w", err)
}
for {
tag, err := c.pool.Exec(ctx, `
WITH candidates AS (
SELECT mi.content_id
FROM public.media_items mi
`+orphanedProvisionalMediaItemPredicate+`
ORDER BY mi.content_id ASC
LIMIT $1
)
DELETE FROM public.media_items mi
USING candidates c
WHERE mi.content_id = c.content_id
`, batchSize)
if err != nil {
return stats, fmt.Errorf("deleting orphaned provisional media items: %w", err)
}
deleted := int(tag.RowsAffected())
stats.Deleted += deleted
if deleted < batchSize {
break
}
}
return stats, nil
}
-41
View File
@@ -1,41 +0,0 @@
package catalog
import (
"strings"
"testing"
)
func normalizePredicateSQL(s string) string {
return strings.Join(strings.Fields(s), " ")
}
func TestOrphanedProvisionalPredicatePreservesDurableMediaItemReferences(t *testing.T) {
predicate := normalizePredicateSQL(orphanedProvisionalMediaItemPredicate)
for _, want := range []string{
"public.abs_bookmarks ab WHERE ab.library_item_id = mi.content_id",
"public.abs_playback_sessions aps WHERE aps.content_id = mi.content_id",
"public.abs_rss_feeds arf WHERE arf.library_item_id = mi.content_id",
"public.episodes e WHERE e.series_id = mi.content_id",
"public.podcast_feeds pf WHERE pf.media_item_id = mi.content_id",
"public.seasons s WHERE s.series_id = mi.content_id",
"public.user_audio_preferences uap WHERE uap.series_id = mi.content_id",
"public.user_home_item_dismissals uhid WHERE uhid.media_item_id = mi.content_id",
"public.user_home_item_dismissals uhid_series WHERE uhid_series.series_id = mi.content_id",
"public.user_personal_collection_items upci WHERE upci.media_item_id = mi.content_id",
"public.user_series_playback_preferences uspp WHERE uspp.series_id = mi.content_id",
"public.user_subtitle_preferences usp WHERE usp.series_id = mi.content_id",
} {
if !strings.Contains(predicate, normalizePredicateSQL(want)) {
t.Fatalf("cleanup predicate missing durable reference guard %q", want)
}
}
for _, droppedTable := range []string{
"abs_collection_items",
"abs_playlist_items",
"abs_playlists",
} {
if strings.Contains(predicate, droppedTable) {
t.Fatalf("cleanup predicate must not reference dropped legacy table %q", droppedTable)
}
}
}
+5 -49
View File
@@ -104,53 +104,6 @@ type QueryDefinition struct {
Limit *int `json:"limit,omitempty"`
}
// MediaScopeVideo is the group scope covering all video-side item types. It
// is accepted anywhere a single-type media scope is and expands to the
// underlying media_items.type values via MediaScopeItemTypes.
const MediaScopeVideo = "video"
// IsValidMediaScope reports whether scope (already normalized to lowercase)
// is an accepted media_scope value. Empty means unscoped and is valid.
func IsValidMediaScope(scope string) bool {
switch scope {
case "", "movie", "series", "episode", "audiobook", MediaScopeVideo:
return true
default:
return false
}
}
// MediaScopeItemTypes expands a media scope into the media_items.type values
// it covers. Single-type scopes map to themselves; the empty scope returns
// nil (unscoped).
func MediaScopeItemTypes(scope string) []string {
scope = strings.ToLower(strings.TrimSpace(scope))
switch scope {
case "":
return nil
case MediaScopeVideo:
return []string{"movie", "series"}
default:
return []string{scope}
}
}
// MediaScopeMatchesItemType reports whether an item type falls inside scope.
// An empty scope matches every type.
func MediaScopeMatchesItemType(scope, itemType string) bool {
types := MediaScopeItemTypes(scope)
if len(types) == 0 {
return true
}
itemType = strings.ToLower(strings.TrimSpace(itemType))
for _, t := range types {
if t == itemType {
return true
}
}
return false
}
const (
DefaultSmartCollectionItemLimit = 100
MaxSmartCollectionItemLimit = 500
@@ -229,8 +182,11 @@ func (q QueryDefinition) ValidateWithOptions(allowPersonalizedSorts, allowPerson
}
}
if !IsValidMediaScope(normalized.MediaScope) {
return fmt.Errorf("media_scope must be 'movie', 'series', 'episode', 'audiobook', or 'video'")
if normalized.MediaScope != "" &&
normalized.MediaScope != "movie" &&
normalized.MediaScope != "series" &&
normalized.MediaScope != "episode" {
return fmt.Errorf("media_scope must be 'movie', 'series', or 'episode'")
}
if normalized.Match != "all" && normalized.Match != "any" {
-12
View File
@@ -60,18 +60,6 @@ func TestValidate_EpisodeMediaScope(t *testing.T) {
}
}
func TestValidate_AudiobookMediaScope(t *testing.T) {
qd := QueryDefinition{
MediaScope: "audiobook",
Match: "all",
Groups: []QueryGroup{},
Sort: QuerySort{Field: "title", Order: "asc"},
}
if err := qd.Validate(); err != nil {
t.Fatalf("expected audiobook media scope to be valid, got %v", err)
}
}
func TestValidateWithSortScope_RejectsPersonalizedSortWithoutProfileScope(t *testing.T) {
qd := QueryDefinition{
Match: "all",
+2 -8
View File
@@ -282,14 +282,8 @@ func (e *QueryExecutor) buildPreviewPagePlan(
conditions = append(conditions, filterWhere)
}
if def.MediaScope != "" && !isEpisodeCatalogScope(def.MediaScope) {
scopeTypes := MediaScopeItemTypes(def.MediaScope)
if len(scopeTypes) == 1 {
conditions = append(conditions, fmt.Sprintf("mi.type = $%d", argIdx))
args = append(args, scopeTypes[0])
} else {
conditions = append(conditions, fmt.Sprintf("mi.type = ANY($%d)", argIdx))
args = append(args, scopeTypes)
}
conditions = append(conditions, fmt.Sprintf("mi.type = $%d", argIdx))
args = append(args, def.MediaScope)
argIdx++
}
libScopeWhere, libScopeArgs, hasLibraryScope := "", []any(nil), false
+1 -1
View File
@@ -32,7 +32,7 @@ func ExtractFrame(ctx context.Context, opts FrameExtractOptions) ([]byte, string
runExtract = runFFmpegFrameExtract
}
resolvedAccel := playback.ResolveHWAccelWithFFmpeg(opts.HWAccel, ffmpegPath)
resolvedAccel := playback.ResolveHWAccel(opts.HWAccel)
resolvedDevice := opts.HWDevice
if resolvedDevice == "" && (resolvedAccel == "qsv" || resolvedAccel == "vaapi") {
resolvedDevice = playback.PickRenderDevice("")
+1 -1
View File
@@ -652,7 +652,7 @@ func (s *Service) extractFrameLocal(ctx context.Context, inputPath string, seekS
func (s *Service) resolveHWConfig() (string, string) {
s.hwResolveOnce.Do(func() {
s.resolvedHWAccel = playback.ResolveHWAccelWithFFmpeg(s.hwAccel, s.ffmpegPath)
s.resolvedHWAccel = playback.ResolveHWAccel(s.hwAccel)
s.resolvedHWDevice = s.hwDevice
if s.resolvedHWDevice == "" && (s.resolvedHWAccel == "qsv" || s.resolvedHWAccel == "vaapi") {
s.resolvedHWDevice = playback.PickRenderDevice("")
-19
View File
@@ -7,11 +7,6 @@ import (
"github.com/joho/godotenv"
)
// minSecretKeyLen is the minimum acceptable SECRET_KEY length (in characters).
// It mirrors secret.MinMasterKeyLen; the package is not imported here to keep
// the bootstrap loader dependency-free, so the two constants must stay in sync.
const minSecretKeyLen = 32
// BootstrapConfig holds the minimal config needed before database connection.
type BootstrapConfig struct {
DatabaseURL string
@@ -19,10 +14,6 @@ type BootstrapConfig struct {
Listen string
JFListen string
Mode string
// SecretKey is the master key (raw SECRET_KEY env value) from which the
// at-rest credential cipher derives its data key. It lives outside Postgres
// so encrypted secrets survive a full database compromise/dump.
SecretKey []byte
}
// LoadBootstrap loads bootstrap configuration from a .env file (if it exists)
@@ -37,15 +28,6 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
return nil, fmt.Errorf("DATABASE_URL is required (set in .env or environment)")
}
// SECRET_KEY is the at-rest encryption master key. It is required: the server
// must never fall back to a zero key or a key derived from another value, or
// encrypted credentials would not survive a database dump. godotenv has
// already loaded .env above, so dev sets it there.
secretKey := os.Getenv("SECRET_KEY")
if len(secretKey) < minSecretKeyLen {
return nil, fmt.Errorf("SECRET_KEY is required (>=%d chars); generate one with: openssl rand -base64 48", minSecretKeyLen)
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
@@ -69,6 +51,5 @@ func LoadBootstrap(envFile string) (*BootstrapConfig, error) {
Listen: ":" + port,
JFListen: ":" + jfPort,
Mode: mode,
SecretKey: []byte(secretKey),
}, nil
}
+4 -3
View File
@@ -198,12 +198,13 @@ type authConfigRaw struct {
RefreshTokenExpiry string `yaml:"refresh_token_expiry"`
}
// AudiobookshelfCompatConfig holds the dedicated ABS-compat listener setting.
// The listener binds its own port (:13378, ABS's
// AudiobookshelfCompatConfig holds the dedicated ABS-compat listener
// settings. The listener binds its own port (default :13378, ABS's
// conventional port) and serves the full ABS protocol — login, libraries,
// items, sessions — without colliding with silo's SPA on the main listener.
type AudiobookshelfCompatConfig struct {
Listen string `yaml:"-"`
Listen string `yaml:"listen"`
PublicURL string `yaml:"public_url"`
}
// JellyfinCompatConfig holds compatibility proxy settings.
+7 -5
View File
@@ -355,14 +355,16 @@ func LoadFromDB(m map[string]string) (*Config, error) {
}
cfg.Auth.RefreshTokenExpiry = refreshTokenExpiry
// AudiobookshelfCompat — dedicated listener for ABS client apps. Docker
// deployments publish :13378, so listen by default unless disabled.
absCompatEnabled, err := boolOr(m, "audiobookshelf_compat.enabled", true)
// AudiobookshelfCompat — dedicated listener for ABS client apps. Keep the
// listener off unless the audiobooks feature flag is explicitly enabled.
audiobooksEnabled, err := boolOr(m, "audiobooks.enabled", false)
if err != nil {
return nil, err
}
if absCompatEnabled {
cfg.AudiobookshelfCompat.Listen = ":13378"
if audiobooksEnabled {
// Default :13378 mirrors the real Audiobookshelf server convention.
cfg.AudiobookshelfCompat.Listen = stringOr(m, "audiobookshelf_compat.listen", ":13378")
cfg.AudiobookshelfCompat.PublicURL = stringOr(m, "audiobookshelf_compat.public_url", "http://127.0.0.1:13378")
}
// JellyfinCompat
+3 -11
View File
@@ -34,16 +34,8 @@ func TestLoadFromDBMetadataPresignExpiryRejectsInvalidDuration(t *testing.T) {
}
}
func TestLoadFromDBAudiobookshelfCompatFlagGatesCompatListener(t *testing.T) {
cfg, err := LoadFromDB(map[string]string{})
if err != nil {
t.Fatalf("LoadFromDB() returned error: %v", err)
}
if cfg.AudiobookshelfCompat.Listen != ":13378" {
t.Fatalf("default audiobooks listener = %q, want default :13378", cfg.AudiobookshelfCompat.Listen)
}
cfg, err = LoadFromDB(map[string]string{"audiobookshelf_compat.enabled": "false"})
func TestLoadFromDBAudiobooksFeatureFlagGatesCompatListener(t *testing.T) {
cfg, err := LoadFromDB(map[string]string{"audiobooks.enabled": "false"})
if err != nil {
t.Fatalf("LoadFromDB() returned error: %v", err)
}
@@ -51,7 +43,7 @@ func TestLoadFromDBAudiobookshelfCompatFlagGatesCompatListener(t *testing.T) {
t.Fatalf("disabled audiobooks listener = %q, want empty", cfg.AudiobookshelfCompat.Listen)
}
cfg, err = LoadFromDB(map[string]string{"audiobookshelf_compat.enabled": "true"})
cfg, err = LoadFromDB(map[string]string{"audiobooks.enabled": "true"})
if err != nil {
t.Fatalf("LoadFromDB() returned error: %v", err)
}
+20 -331
View File
@@ -6,14 +6,11 @@ import (
"errors"
"fmt"
"sort"
"strconv"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/secret"
)
var (
@@ -29,135 +26,11 @@ var (
)
type Repository struct {
pool *pgxpool.Pool
cipher *secret.Cipher
pool *pgxpool.Pool
}
func NewRepository(pool *pgxpool.Pool, cipher *secret.Cipher) *Repository {
return &Repository{pool: pool, cipher: cipher}
}
// AAD builders binding each history-import secret column to its row id.
func sourceAdminTokenAAD(sourceID int) string {
return secret.RowAAD("history_import_sources", "admin_token", strconv.Itoa(sourceID))
}
func connectServerAccessKeyAAD(sessionID string, index int, server ConnectServer) string {
return secret.RowAAD("history_import_connect_sessions", "servers_json.access_key", sessionID+":"+strconv.Itoa(index)+":"+server.ID)
}
func connectSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_connect_sessions", "connect_access_token", sessionID)
}
func plexServerAccessTokenAAD(sessionID string, index int, server PlexServer) string {
return secret.RowAAD("history_import_plex_sessions", "servers_json.access_token", sessionID+":"+strconv.Itoa(index)+":"+server.ClientIdentifier)
}
func plexSessionTokenAAD(sessionID string) string {
return secret.RowAAD("history_import_plex_sessions", "auth_token", sessionID)
}
func (r *Repository) encryptSourceAdminToken(sourceID int, token string) (string, error) {
return r.cipher.Encrypt(token, sourceAdminTokenAAD(sourceID))
}
func (r *Repository) encryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
if out[i].AccessKey == "" {
continue
}
accessKey, err := r.cipher.Encrypt(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) encryptPlaintextConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, bool, error) {
out := append([]ConnectServer(nil), servers...)
changed := false
for i := range out {
accessKey, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt connect server access key: %w", err)
}
if didChange {
out[i].AccessKey = accessKey
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptConnectServers(sessionID string, servers []ConnectServer) ([]ConnectServer, error) {
out := append([]ConnectServer(nil), servers...)
for i := range out {
accessKey, err := r.cipher.DecryptIfEncrypted(out[i].AccessKey, connectServerAccessKeyAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt connect server access key: %w", err)
}
out[i].AccessKey = accessKey
}
return out, nil
}
func (r *Repository) marshalConnectServersForWrite(sessionID string, servers []ConnectServer) ([]byte, error) {
encrypted, err := r.encryptConnectServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
}
func (r *Repository) encryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
if out[i].AccessToken == "" {
continue
}
accessToken, err := r.cipher.Encrypt(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("encrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) encryptPlaintextPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, bool, error) {
out := append([]PlexServer(nil), servers...)
changed := false
for i := range out {
accessToken, didChange, err := r.cipher.EncryptIfPlaintext(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, false, fmt.Errorf("encrypt plex server access token: %w", err)
}
if didChange {
out[i].AccessToken = accessToken
changed = true
}
}
return out, changed, nil
}
func (r *Repository) decryptPlexServers(sessionID string, servers []PlexServer) ([]PlexServer, error) {
out := append([]PlexServer(nil), servers...)
for i := range out {
accessToken, err := r.cipher.DecryptIfEncrypted(out[i].AccessToken, plexServerAccessTokenAAD(sessionID, i, out[i]))
if err != nil {
return nil, fmt.Errorf("decrypt plex server access token: %w", err)
}
out[i].AccessToken = accessToken
}
return out, nil
}
func (r *Repository) marshalPlexServersForWrite(sessionID string, servers []PlexServer) ([]byte, error) {
encrypted, err := r.encryptPlexServers(sessionID, servers)
if err != nil {
return nil, err
}
return json.Marshal(encrypted)
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
}
func (r *Repository) ListEnabledSources(ctx context.Context) ([]Source, error) {
@@ -207,44 +80,20 @@ func (r *Repository) GetSourceByID(ctx context.Context, id int) (*Source, error)
}
func (r *Repository) CreateSource(ctx context.Context, input CreateSourceInput) (*Source, error) {
tx, err := r.pool.Begin(ctx)
if err != nil {
return nil, fmt.Errorf("begin creating history import source: %w", err)
}
defer tx.Rollback(ctx)
row := tx.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6)
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_sources (name, source_type, base_url, system_id, enabled, sort_order,
admin_token)
VALUES ($1, $2, $3, NULLIF($4, ''), $5, $6, NULLIF($7, ''))
RETURNING id, name, source_type, base_url, COALESCE(system_id, ''), enabled, sort_order,
(admin_token IS NOT NULL) AS has_admin_token,
created_at, updated_at`,
input.Name, input.SourceType, input.BaseURL, input.SystemID, input.Enabled, input.SortOrder,
input.AdminToken,
)
source, err := scanSource(row)
if err != nil {
return nil, fmt.Errorf("creating history import source: %w", err)
}
if input.AdminToken != "" {
encryptedToken, err := r.encryptSourceAdminToken(source.ID, input.AdminToken)
if err != nil {
return nil, fmt.Errorf("encrypt admin token for source %d: %w", source.ID, err)
}
result, err := tx.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, source.ID, encryptedToken)
if err != nil {
return nil, fmt.Errorf("setting admin token for source %d: %w", source.ID, err)
}
if result.RowsAffected() == 0 {
return nil, ErrSourceNotFound
}
source.HasAdminToken = true
}
if err := tx.Commit(ctx); err != nil {
return nil, fmt.Errorf("commit creating history import source: %w", err)
}
return source, nil
}
@@ -286,22 +135,18 @@ func (r *Repository) DeleteSource(ctx context.Context, id int) error {
}
func (r *Repository) CreateConnectSession(ctx context.Context, session ConnectSession) (*ConnectSession, error) {
serversJSON, err := r.marshalConnectServersForWrite(session.ID, session.Servers)
serversJSON, err := json.Marshal(session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling connect servers: %w", err)
}
accessToken, err := r.cipher.Encrypt(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt connect access token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_connect_sessions (
id, user_id, connect_user_id, connect_access_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.ConnectUserID, accessToken, serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.ConnectUserID, session.ConnectAccessToken, serversJSON, session.ExpiresAt,
)
stored, err := r.scanConnectSession(row)
stored, err := scanConnectSession(row)
if err != nil {
return nil, fmt.Errorf("creating connect session: %w", err)
}
@@ -313,7 +158,7 @@ func (r *Repository) GetConnectSession(ctx context.Context, userID int, sessionI
SELECT id, user_id, connect_user_id, connect_access_token, servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_connect_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := r.scanConnectSession(row)
session, err := scanConnectSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrConnectSessionNotFound
}
@@ -354,22 +199,18 @@ func (r *Repository) DeleteExpiredConnectSessions(ctx context.Context) error {
}
func (r *Repository) CreatePlexSession(ctx context.Context, session PlexSession) (*PlexSession, error) {
serversJSON, err := r.marshalPlexServersForWrite(session.ID, session.Servers)
serversJSON, err := json.Marshal(session.Servers)
if err != nil {
return nil, fmt.Errorf("marshaling plex servers: %w", err)
}
authToken, err := r.cipher.Encrypt(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("encrypt plex auth token: %w", err)
}
row := r.pool.QueryRow(ctx, `
INSERT INTO history_import_plex_sessions (
id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at
) VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, user_id, pin_id, pin_code, auth_token, servers_json, expires_at, consumed_at, created_at, updated_at`,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(authToken), serversJSON, session.ExpiresAt,
session.ID, session.UserID, session.PinID, session.PinCode, nilIfEmpty(session.AuthToken), serversJSON, session.ExpiresAt,
)
stored, err := r.scanPlexSession(row)
stored, err := scanPlexSession(row)
if err != nil {
return nil, fmt.Errorf("creating plex session: %w", err)
}
@@ -381,7 +222,7 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
SELECT id, user_id, pin_id, pin_code, COALESCE(auth_token, ''), servers_json, expires_at, consumed_at, created_at, updated_at
FROM history_import_plex_sessions
WHERE id = $1 AND user_id = $2`, sessionID, userID)
session, err := r.scanPlexSession(row)
session, err := scanPlexSession(row)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrPlexSessionNotFound
}
@@ -398,18 +239,14 @@ func (r *Repository) GetPlexSession(ctx context.Context, userID int, sessionID s
}
func (r *Repository) UpdatePlexSessionAuth(ctx context.Context, sessionID, authToken string, servers []PlexServer) error {
serversJSON, err := r.marshalPlexServersForWrite(sessionID, servers)
serversJSON, err := json.Marshal(servers)
if err != nil {
return fmt.Errorf("marshaling plex servers: %w", err)
}
encryptedToken, err := r.cipher.Encrypt(authToken, plexSessionTokenAAD(sessionID))
if err != nil {
return fmt.Errorf("encrypt plex auth token: %w", err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET auth_token = $2, servers_json = $3, updated_at = NOW()
WHERE id = $1`, sessionID, encryptedToken, serversJSON)
WHERE id = $1`, sessionID, authToken, serversJSON)
if err != nil {
return fmt.Errorf("updating plex session auth %s: %w", sessionID, err)
}
@@ -443,136 +280,6 @@ func (r *Repository) DeleteExpiredPlexSessions(ctx context.Context) error {
return nil
}
// BackfillSessionServerSecrets encrypts credentials nested inside history-import
// session server JSON. The top-level session token columns are handled by the
// generic column backfill, but Emby Connect access keys and Plex per-server
// access tokens live inside servers_json and need a typed JSON pass.
func (r *Repository) BackfillSessionServerSecrets(ctx context.Context) (int, error) {
connectN, connectErr := r.backfillConnectSessionServerSecrets(ctx)
plexN, plexErr := r.backfillPlexSessionServerSecrets(ctx)
return connectN + plexN, errors.Join(connectErr, plexErr)
}
func (r *Repository) backfillConnectSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_connect_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select connect sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan connect session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate connect sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []ConnectServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode connect session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextConnectServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt connect session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode connect session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_connect_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update connect session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) backfillPlexSessionServerSecrets(ctx context.Context) (int, error) {
rows, err := r.pool.Query(ctx, `
SELECT id, servers_json::text
FROM history_import_plex_sessions
WHERE servers_json <> '[]'::jsonb`)
if err != nil {
return 0, fmt.Errorf("select plex sessions: %w", err)
}
type rec struct{ id, raw string }
var recs []rec
for rows.Next() {
var r rec
if err := rows.Scan(&r.id, &r.raw); err != nil {
rows.Close()
return 0, fmt.Errorf("scan plex session: %w", err)
}
recs = append(recs, r)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, fmt.Errorf("iterate plex sessions: %w", err)
}
count := 0
var errs []error
for _, rec := range recs {
var servers []PlexServer
if err := json.Unmarshal([]byte(rec.raw), &servers); err != nil {
errs = append(errs, fmt.Errorf("decode plex session %s servers: %w", rec.id, err))
continue
}
encrypted, changed, err := r.encryptPlaintextPlexServers(rec.id, servers)
if err != nil {
errs = append(errs, fmt.Errorf("encrypt plex session %s servers: %w", rec.id, err))
continue
}
if !changed {
continue
}
body, err := json.Marshal(encrypted)
if err != nil {
errs = append(errs, fmt.Errorf("encode plex session %s servers: %w", rec.id, err))
continue
}
tag, err := r.pool.Exec(ctx, `
UPDATE history_import_plex_sessions
SET servers_json = $2::jsonb, updated_at = NOW()
WHERE id = $1 AND servers_json = $3::jsonb`,
rec.id, string(body), rec.raw)
if err != nil {
errs = append(errs, fmt.Errorf("update plex session %s servers: %w", rec.id, err))
continue
}
if tag.RowsAffected() > 0 {
count++
}
}
return count, errors.Join(errs...)
}
func (r *Repository) ProfileExistsForUser(ctx context.Context, userID int, profileID string) (bool, error) {
var exists bool
if err := r.pool.QueryRow(ctx,
@@ -1063,7 +770,7 @@ func scanSources(rows pgx.Rows) ([]Source, error) {
return sources, rows.Err()
}
func (r *Repository) scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
func scanConnectSession(scanner interface{ Scan(dest ...any) error }) (*ConnectSession, error) {
var session ConnectSession
var serversJSON []byte
if err := scanner.Scan(
@@ -1072,19 +779,10 @@ func (r *Repository) scanConnectSession(scanner interface{ Scan(dest ...any) err
); err != nil {
return nil, err
}
token, err := r.cipher.DecryptIfEncrypted(session.ConnectAccessToken, connectSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt connect access token: %w", err)
}
session.ConnectAccessToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling connect session servers: %w", err)
}
session.Servers, err = r.decryptConnectServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
@@ -1223,7 +921,7 @@ func nonNilUnmatchedSamples(values []UnmatchedSample) []UnmatchedSample {
return values
}
func (r *Repository) scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
func scanPlexSession(scanner interface{ Scan(dest ...any) error }) (*PlexSession, error) {
var session PlexSession
var authToken *string
var serversJSON []byte
@@ -1237,19 +935,10 @@ func (r *Repository) scanPlexSession(scanner interface{ Scan(dest ...any) error
if authToken != nil {
session.AuthToken = *authToken
}
token, err := r.cipher.DecryptIfEncrypted(session.AuthToken, plexSessionTokenAAD(session.ID))
if err != nil {
return nil, fmt.Errorf("decrypt plex auth token: %w", err)
}
session.AuthToken = token
if len(serversJSON) > 0 {
if err := json.Unmarshal(serversJSON, &session.Servers); err != nil {
return nil, fmt.Errorf("unmarshaling plex session servers: %w", err)
}
session.Servers, err = r.decryptPlexServers(session.ID, session.Servers)
if err != nil {
return nil, err
}
}
return &session, nil
}
+1 -9
View File
@@ -21,14 +21,10 @@ var (
// SetSourceAdminToken stores an admin token for the given source.
func (r *Repository) SetSourceAdminToken(ctx context.Context, sourceID int, token string) error {
encryptedToken, err := r.encryptSourceAdminToken(sourceID, token)
if err != nil {
return fmt.Errorf("encrypt admin token for source %d: %w", sourceID, err)
}
result, err := r.pool.Exec(ctx, `
UPDATE history_import_sources
SET admin_token = $2, updated_at = NOW()
WHERE id = $1`, sourceID, encryptedToken)
WHERE id = $1`, sourceID, token)
if err != nil {
return fmt.Errorf("setting admin token for source %d: %w", sourceID, err)
}
@@ -77,10 +73,6 @@ func (r *Repository) GetSourceWithAdminToken(ctx context.Context, sourceID int)
if err != nil {
return nil, "", fmt.Errorf("getting source %d with admin token: %w", sourceID, err)
}
adminToken, err = r.cipher.DecryptIfEncrypted(adminToken, sourceAdminTokenAAD(sourceID))
if err != nil {
return nil, "", fmt.Errorf("decrypt admin token for source %d: %w", sourceID, err)
}
return &s, adminToken, nil
}
+6 -194
View File
@@ -1,34 +1,15 @@
package historyimport
import (
"bytes"
"crypto/rand"
"encoding/json"
"fmt"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/secret"
)
// newTestRepo builds a Repository with a real cipher for scan-method tests.
func newTestRepo(t *testing.T) *Repository {
t.Helper()
key := make([]byte, 48)
if _, err := rand.Read(key); err != nil {
t.Fatalf("rand: %v", err)
}
c, err := secret.New(key)
if err != nil {
t.Fatalf("secret.New: %v", err)
}
return &Repository{cipher: c}
}
type plexSessionScannerStub struct {
authToken *string
servers []PlexServer
serversRaw []byte
authToken *string
servers []PlexServer
}
func (s plexSessionScannerStub) Scan(dest ...any) error {
@@ -71,13 +52,9 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
return fmt.Errorf("dest[4] type = %T, want *string or **string", dest[4])
}
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
serversJSON, err := json.Marshal(s.servers)
if err != nil {
return err
}
serversDest, ok := dest[5].(*[]byte)
if !ok {
@@ -115,7 +92,7 @@ func (s plexSessionScannerStub) Scan(dest ...any) error {
func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
t.Parallel()
session, err := newTestRepo(t).scanPlexSession(plexSessionScannerStub{
session, err := scanPlexSession(plexSessionScannerStub{
authToken: nil,
servers: []PlexServer{{
Name: "Plex Server",
@@ -141,169 +118,4 @@ func TestScanPlexSession_AllowsNullAuthToken(t *testing.T) {
if session.Servers[0].ClientIdentifier != "server-1" {
t.Fatalf("session.Servers[0].ClientIdentifier = %q, want server-1", session.Servers[0].ClientIdentifier)
}
if session.Servers[0].AccessToken != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", session.Servers[0].AccessToken)
}
}
type connectSessionScannerStub struct {
accessToken string
servers []ConnectServer
serversRaw []byte
}
func (s connectSessionScannerStub) Scan(dest ...any) error {
if len(dest) != 9 {
return fmt.Errorf("unexpected scan dest count: %d", len(dest))
}
id, ok := dest[0].(*string)
if !ok {
return fmt.Errorf("dest[0] type = %T, want *string", dest[0])
}
*id = "connect-session-1"
userID, ok := dest[1].(*int)
if !ok {
return fmt.Errorf("dest[1] type = %T, want *int", dest[1])
}
*userID = 42
connectUserID, ok := dest[2].(*string)
if !ok {
return fmt.Errorf("dest[2] type = %T, want *string", dest[2])
}
*connectUserID = "connect-user-1"
accessToken, ok := dest[3].(*string)
if !ok {
return fmt.Errorf("dest[3] type = %T, want *string", dest[3])
}
*accessToken = s.accessToken
serversJSON := s.serversRaw
if serversJSON == nil {
var err error
serversJSON, err = json.Marshal(s.servers)
if err != nil {
return err
}
}
serversDest, ok := dest[4].(*[]byte)
if !ok {
return fmt.Errorf("dest[4] type = %T, want *[]byte", dest[4])
}
*serversDest = serversJSON
expiresAt, ok := dest[5].(*time.Time)
if !ok {
return fmt.Errorf("dest[5] type = %T, want *time.Time", dest[5])
}
*expiresAt = time.Date(2026, time.April, 1, 13, 0, 0, 0, time.UTC)
consumedAt, ok := dest[6].(**time.Time)
if !ok {
return fmt.Errorf("dest[6] type = %T, want **time.Time", dest[6])
}
*consumedAt = nil
createdAt, ok := dest[7].(*time.Time)
if !ok {
return fmt.Errorf("dest[7] type = %T, want *time.Time", dest[7])
}
*createdAt = time.Date(2026, time.April, 1, 12, 0, 0, 0, time.UTC)
updatedAt, ok := dest[8].(*time.Time)
if !ok {
return fmt.Errorf("dest[8] type = %T, want *time.Time", dest[8])
}
*updatedAt = time.Date(2026, time.April, 1, 12, 30, 0, 0, time.UTC)
return nil
}
func TestConnectSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []ConnectServer{{
ID: "emby-server-1",
Name: "Emby Server",
AccessKey: "server-access-key",
URL: "https://emby.example",
HasRemoteURL: true,
}}
raw, err := repo.marshalConnectServersForWrite("connect-session-1", servers)
if err != nil {
t.Fatalf("marshalConnectServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-access-key")) {
t.Fatalf("stored connect servers JSON leaked access key: %s", raw)
}
session, err := repo.scanConnectSession(connectSessionScannerStub{
accessToken: "connect-token",
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanConnectSession: %v", err)
}
if got := session.Servers[0].AccessKey; got != "server-access-key" {
t.Fatalf("session.Servers[0].AccessKey = %q, want server-access-key", got)
}
}
func TestPlexSessionServersEncryptedAtRestAndDecryptedOnRead(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
servers := []PlexServer{{
Name: "Plex Server",
ClientIdentifier: "server-1",
AccessToken: "server-token",
RemoteURL: "https://plex.example",
Owned: true,
HasRemoteURL: true,
}}
raw, err := repo.marshalPlexServersForWrite("plex-session-1", servers)
if err != nil {
t.Fatalf("marshalPlexServersForWrite: %v", err)
}
if bytes.Contains(raw, []byte("server-token")) {
t.Fatalf("stored plex servers JSON leaked access token: %s", raw)
}
session, err := repo.scanPlexSession(plexSessionScannerStub{
authToken: nil,
serversRaw: raw,
})
if err != nil {
t.Fatalf("scanPlexSession: %v", err)
}
if got := session.Servers[0].AccessToken; got != "server-token" {
t.Fatalf("session.Servers[0].AccessToken = %q, want server-token", got)
}
}
func TestSourceAdminTokenEncryptionUsesSourceAAD(t *testing.T) {
t.Parallel()
repo := newTestRepo(t)
ciphertext, err := repo.encryptSourceAdminToken(42, "admin-token")
if err != nil {
t.Fatalf("encryptSourceAdminToken: %v", err)
}
if ciphertext == "admin-token" || bytes.Contains([]byte(ciphertext), []byte("admin-token")) {
t.Fatalf("encrypted admin token leaked plaintext: %q", ciphertext)
}
plaintext, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(42))
if err != nil {
t.Fatalf("decrypt with matching source AAD: %v", err)
}
if plaintext != "admin-token" {
t.Fatalf("decrypted admin token = %q, want admin-token", plaintext)
}
if _, err := repo.cipher.Decrypt(ciphertext, sourceAdminTokenAAD(43)); err == nil {
t.Fatalf("decrypt with different source AAD succeeded, want authentication failure")
}
}
-54
View File
@@ -1,54 +0,0 @@
package jellycompat
import (
"context"
"log/slog"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/catalog"
)
// ScopeResolver resolves a viewer request into an effective access scope.
// Implemented by *access.Resolver.
type ScopeResolver interface {
Resolve(ctx context.Context, input access.ResolveInput) (access.Scope, error)
}
// NewScopeAccessFilter returns an AccessFilterResolver backed by the shared
// access scope resolver, so account-level library restrictions
// (users.library_ids), profile library restrictions, user-disabled
// libraries, content-rating ceilings, and playback-quality ceilings all
// apply to the Jellyfin-compat API exactly as they do to the native API.
//
// PIN verification is skipped because compat login already verified the
// profile PIN (password#pin convention). Resolution failures fail closed:
// the viewer gets an empty library allowlist rather than full access.
func NewScopeAccessFilter(resolver ScopeResolver) AccessFilterResolver {
return func(ctx context.Context, userID int, profileID string) catalog.AccessFilter {
scope, err := resolver.Resolve(ctx, access.ResolveInput{
UserID: userID,
ProfileID: profileID,
SkipPINVerification: true,
})
if err != nil {
slog.Warn("jellycompat: access scope resolution failed; denying library access",
"user_id", userID,
"profile_id", profileID,
"error", err,
)
return catalog.AccessFilter{
AllowedLibraryIDs: []int{},
UserID: userID,
ProfileID: profileID,
}
}
return catalog.AccessFilter{
AllowedLibraryIDs: scope.AllowedLibraryIDs,
DisabledLibraryIDs: scope.DisabledLibraryIDs,
MaxContentRating: scope.MaxContentRating,
MaxPlaybackQuality: scope.MaxPlaybackQuality,
UserID: userID,
ProfileID: profileID,
}
}
}
-177
View File
@@ -1,177 +0,0 @@
package jellycompat
import (
"context"
"errors"
"reflect"
"testing"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/models"
)
type stubScopeResolver struct {
scope access.Scope
err error
input access.ResolveInput
}
func (s *stubScopeResolver) Resolve(_ context.Context, input access.ResolveInput) (access.Scope, error) {
s.input = input
return s.scope, s.err
}
func TestScopeAccessFilterMapsScope(t *testing.T) {
resolver := &stubScopeResolver{
scope: access.Scope{
UserID: 7,
ProfileID: "profile-1",
AllowedLibraryIDs: []int{2, 19},
LibrariesRestricted: true,
MaxContentRating: "PG-13",
MaxPlaybackQuality: "1080p",
},
}
filter := NewScopeAccessFilter(resolver)(context.Background(), 7, "profile-1")
if !resolver.input.SkipPINVerification {
t.Fatal("expected SkipPINVerification=true (PIN is verified at compat login)")
}
if resolver.input.UserID != 7 || resolver.input.ProfileID != "profile-1" {
t.Fatalf("unexpected resolve input: %+v", resolver.input)
}
if !reflect.DeepEqual(filter.AllowedLibraryIDs, []int{2, 19}) {
t.Fatalf("AllowedLibraryIDs = %v, want [2 19]", filter.AllowedLibraryIDs)
}
if filter.MaxContentRating != "PG-13" {
t.Fatalf("MaxContentRating = %q, want PG-13", filter.MaxContentRating)
}
if filter.MaxPlaybackQuality != "1080p" {
t.Fatalf("MaxPlaybackQuality = %q, want 1080p", filter.MaxPlaybackQuality)
}
if filter.UserID != 7 || filter.ProfileID != "profile-1" {
t.Fatalf("identity not propagated: %+v", filter)
}
}
func TestScopeAccessFilterUnrestrictedScope(t *testing.T) {
resolver := &stubScopeResolver{
scope: access.Scope{
UserID: 1,
DisabledLibraryIDs: []int{7},
},
}
filter := NewScopeAccessFilter(resolver)(context.Background(), 1, "profile-1")
if filter.AllowedLibraryIDs != nil {
t.Fatalf("AllowedLibraryIDs = %v, want nil (unrestricted)", filter.AllowedLibraryIDs)
}
if !reflect.DeepEqual(filter.DisabledLibraryIDs, []int{7}) {
t.Fatalf("DisabledLibraryIDs = %v, want [7]", filter.DisabledLibraryIDs)
}
}
func TestScopeAccessFilterFailsClosed(t *testing.T) {
resolver := &stubScopeResolver{err: errors.New("boom")}
filter := NewScopeAccessFilter(resolver)(context.Background(), 7, "profile-1")
if filter.AllowedLibraryIDs == nil || len(filter.AllowedLibraryIDs) != 0 {
t.Fatalf("AllowedLibraryIDs = %v, want empty non-nil allowlist (deny all)", filter.AllowedLibraryIDs)
}
}
type stubFolderSource struct {
enabled []*models.MediaFolder
byIDs []*models.MediaFolder
listByIDsArgs []int
}
func (s *stubFolderSource) GetEnabled(context.Context) ([]*models.MediaFolder, error) {
return s.enabled, nil
}
func (s *stubFolderSource) ListByIDs(_ context.Context, ids []int) ([]*models.MediaFolder, error) {
s.listByIDsArgs = ids
return s.byIDs, nil
}
func TestListUserLibrariesRestrictedAllowlist(t *testing.T) {
folders := &stubFolderSource{
byIDs: []*models.MediaFolder{
{ID: 2, Name: "TV Shows", Type: "series"},
{ID: 19, Name: "Movies", Type: "movies"},
},
}
svc := &directContentService{
folderRepo: folders,
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
AllowedLibraryIDs: []int{2, 19},
LibrariesRestricted: true,
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 7, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if !reflect.DeepEqual(folders.listByIDsArgs, []int{2, 19}) {
t.Fatalf("ListByIDs called with %v, want [2 19]", folders.listByIDsArgs)
}
if len(libraries) != 2 {
t.Fatalf("got %d libraries, want 2", len(libraries))
}
}
func TestListUserLibrariesFiltersDisabled(t *testing.T) {
svc := &directContentService{
folderRepo: &stubFolderSource{
enabled: []*models.MediaFolder{
{ID: 2, Name: "TV Shows", Type: "series"},
{ID: 7, Name: "XXX", Type: "movies"},
{ID: 19, Name: "Movies", Type: "movies"},
},
},
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
DisabledLibraryIDs: []int{7},
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 1, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if len(libraries) != 2 {
t.Fatalf("got %d libraries, want 2 (disabled library not filtered)", len(libraries))
}
for _, lib := range libraries {
if lib.ID == 7 {
t.Fatal("user-disabled library 7 still present in views list")
}
}
}
func TestListUserLibrariesEmptyAllowlistDeniesAll(t *testing.T) {
svc := &directContentService{
accessFilter: NewScopeAccessFilter(&stubScopeResolver{
scope: access.Scope{
AllowedLibraryIDs: []int{},
LibrariesRestricted: true,
},
}),
}
libraries, err := svc.ListUserLibraries(context.Background(), &Session{StreamAppUserID: 7, ProfileID: "profile-1"})
if err != nil {
t.Fatalf("ListUserLibraries: %v", err)
}
if len(libraries) != 0 {
t.Fatalf("got %d libraries, want 0 for empty allowlist", len(libraries))
}
}

Some files were not shown because too many files have changed in this diff Show More