haeaeae/silo-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)

Browse Source 
* feat(metadata): register builtin NFO provider and broaden parsing

Phases A and B of the #216 local-NFO work, implemented test-first.

Registration & hint-first identity (Phase A):
- Migration seeds a reserved kind='builtin' silo.builtin installation
  and an 'nfo' metadata capability (default_enabled=false, priority 1
  for movie/series) with a partial unique index and documented Down.
- In-process builtin provider registry (internal/metadata/builtin.go);
  buildProviders returns the registered provider for builtin rows.
- Guard rails keep the reserved row out of every plugin surface (user
  plugin-settings, installations list, image resolvers, preload,
  auto-update, store Delete, mutation handlers -> 409); silo.builtin is
  a reserved manifest id.
- Startup sync materializes legacy content_level='' chains per level,
  then appends builtin capabilities disabled via
  AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy
  priority now respects default_enabled=false.
- NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider
  with per-mode conflict policy (stored IDs win on scheduled refresh,
  NFO wins on manual refresh, Identify skips NFO); ID-less candidates
  are excluded from provider-priority tie-breaks and nfo never counts
  as corroboration.
- Web chain-editor empty-state gate is now server-derived so builtin
  providers are reachable on plugin-less servers.

Parser breadth & sidecar hardening (Phase B):
- Parser covers the practical Kodi/Jellyfin field set for <movie> and
  <tvshow>: original title, tagline, runtime, dates, content rating,
  genres/studios/countries/tags, multi-source ratings with scale
  normalization, cast with roles/order, director/credits. Empty
  collections stay nil so merge early-returns apply.
- findNFO parses candidates and falls through on read/parse failure or
  root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo;
  GetMetadata gains the same ContentType guard Search has.
- New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate
  in merge (Go) and the edit-metadata dialog (web), closing the gap
  where a manual refresh re-applied NFO dates over admin corrections.
- Merge-contract tests pin NFO fill semantics, genres whole-list
  first-provider-wins, and NFO edits propagating on manual refresh only.
- Docs: new admin wiki page (supported fields, merge semantics,
  naming-supplies-structure contract), index bullet, sidecar wording
  revision, v1-scope feature-detection note.

Zero behavior change while the provider is disabled (default); pinned
by CI-mode and DB-gated test suites.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* feat(metadata): ingest local sidecar artwork and read series-depth NFO

Phases C and D of the #216 local-NFO work, implemented test-first, plus
the mixed-library use-case pins. Together these deliver the headline
case: a series absent from every remote database (e.g. a fitness
library) scans into a fully presented show -> named seasons -> titled
episodes tree from NFO files and sidecar art alone.

Local sidecar artwork through the S3 image cache (Phase C):
- The NFO provider implements ImageProvider: poster/backdrop/logo
  sidecar discovery with a fixed precedence map, symlink/non-regular
  rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic
  filenames apply only via the sidecar search paths, so a shared
  folder.jpg in a flat multi-movie directory applies to none.
- file:// becomes a live local source scheme: routed into *_source_path
  (never *_path), accepted by every image enqueue gate, attributed as
  provider "local", excluded from cached-path detection.
- The image-cache processor caches local files with lexical-on-logical
  confinement to the library roots, open-handle reads with re-checks,
  the same variant widths as remote art, and stable (7-day) failure
  classification. Keys land under
  local/{contentType}/{contentID}/{hash8}/{imageType}; superseded
  prefixes are cleaned on re-cache and item deletion.
- applyIfBetter gains a local exemption so rating-0 local art can fill
  matched items without being stickily displaced; ImageRequest carries
  additive sidecar path context.

Series depth (Phase D):
- SeasonsRequest/EpisodesRequest carry additive local path context
  (series roots, per-season directories, per-episode file paths),
  derived from naming at match time and reconstructed on refresh.
- season.nfo supplies season name/plot; NFO season numbers are advisory
  (directory-derived number wins with a Warn - naming owns structure).
  <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles
  episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy
  wins over NFO numbers.
- Episode NFOs work without a season.nfo (provider seasons unioned with
  on-disk seasons); SynthesizeFallbackEpisodes always runs after persist
  so NFO-less episodes keep synthesized rows. Season/episode file:// art
  rides the Phase C pipeline unchanged.
- Migration adds season:1/episode:1 to the builtin NFO capability's
  default_priority (still default_enabled=false).

Mixed sports-library use case (tests only, no product change):
- Pins the classification contract for one library holding movie-shaped
  and show-shaped content (WWE PPV events as movies next to a "WWE
  SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming
  decides movie-vs-series per file before any provider runs; the NFO
  supplies metadata/identity but never flips type (ContentType guard);
  the per-root Type override is the correction path.
- NFO-driven type classification at scan time is recorded as an explicit
  deferred open question.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* docs(metadata): document local NFO metadata architecture

Add a single as-built architecture page
(docs/architecture/local-nfo-metadata.md) for the #216 local-NFO
feature: the builtin registration model, hint-first identity semantics,
the file:// -> S3 artwork pipeline and its deployment constraint, series
depth, the mixed-library classification contract, and known limitations.

This replaces the working implementation plan, the per-phase specs, and
the narrow sidecar-artwork note, which were planning drafts and are left
untracked; admin-facing behavior remains in the wiki.

Part of #216

AI-use disclosure: planned, drafted, and consolidated with Claude Code
(Fable 5) using multi-agent exploration and adversarial review.

* fix(metadata): address PR review findings on NFO builtin provider

Fold in the valid, low-risk fixes surfaced by automated review on #390:

- imagecache: extract validateCacheRequest so CacheBytes (the local
  sidecar season/episode path) enforces the same episode-requires-season
  guard as Cache, preventing distinct episodes' art from colliding under
  one S3 key.
- image_cache_processor: close the sidecar symlink-swap window by
  rejecting the opened handle unless os.SameFile matches the Lstat'd
  file, so a leaf swapped to a symlink can't pull an out-of-root target
  into the public cache.
- plugins: guard the reserved builtin installation row in the store's
  Update, matching Delete, so its version/enabled/capabilities can never
  be rewritten even if a mutation slips past the HTTP layer.
- cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck
  DB round-trip fails fast at startup instead of hanging.
- metadata: panic instead of silently no-op'ing on an invalid
  RegisterBuiltinProvider call (init-time programmer error).
- docs: correct the media-folder-and-naming NFO paragraph to state
  season/episode NFOs and sidecar artwork are actively read.

---------

Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
This commit is contained in:
CoffeeKnyte
2026-07-16 17:55:36 -04:00
committed by GitHub
co-authored by Quick104
parent 1664c60425
commit 91e1164090
60 changed files with 7253 additions and 345 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cmd/silo/main.go
+37
View File
@@ -67,6 +67,11 @@ import (
"github.com/Silo-Server/silo-server/internal/markers"
"github.com/Silo-Server/silo-server/internal/mdblist"
"github.com/Silo-Server/silo-server/internal/metadata"
// Built-in metadata providers self-register into the metadata package's
// builtin registry on import; buildProviders resolves their seeded chain
// entries in-process (no gRPC).
_ "github.com/Silo-Server/silo-server/internal/metadata/nfo"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/nodeconfig"
"github.com/Silo-Server/silo-server/internal/nodepool"
@@ -999,6 +1004,11 @@ func main() {
}
out := make([]pluginhost.InstalledPluginRecord, 0, len(installations))
for _, installation := range installations {
// The reserved builtin row is not a plugin; keep it out
// of the host's installed-plugin listing.
if installation.IsBuiltin() {
continue
}
capabilities, err := installationStore.ListCapabilities(ctx, installation.ID)
if err != nil {
return nil, err
@@ -1148,6 +1158,17 @@ func main() {
var mangaEnricher *manga.Enricher
if needsWorkers && deps.DB != nil && deps.FileRepo != nil {
chainRepo := metadata.NewChainRepository(deps.DB)
// Make every existing library chain aware of the built-in providers
// before serving: materialize legacy content_level='' chains per level,
// then append registered builtins disabled (idempotent; also the repair
// path after a stale chain-editor save drops a builtin row). Runs before
// the metadata service exists, so no chain cache to invalidate here.
syncCtx, syncCancel := context.WithTimeout(appCtx, 30*time.Second)
syncErr := metadata.SyncBuiltinProviderChains(syncCtx, chainRepo)
syncCancel()
if syncErr != nil {
log.Fatalf("sync builtin provider chains: %v", syncErr)
}
skippedRootRepo = metadata.NewSkippedRootRepository(deps.DB)
itemRepo = catalog.NewItemRepository(deps.DB).WithActiveSearchProvider(activeCatalogSearchProvider)
episodeRepo = catalog.NewEpisodeRepository(deps.DB)
@@ -1272,6 +1293,11 @@ func main() {
People: personRepo,
},
)
// Local file:// artwork (NFO sidecars): confine reads to the owning
// library's roots and sweep stale hashed local/ prefixes on re-cache.
// The processor host must mount the libraries, like the metadata worker.
metadataImageCacheProcessor.SetLibraryRootResolver(deps.FolderRepo)
metadataImageCacheProcessor.SetImagePrefixDeleter(deps.S3Public)
metadataService.SetAutoCacheImages(cfg.Metadata.CacheImages)
metadataImageCacheProcessor.SetEnabled(cfg.Metadata.CacheImages)
configWatcher.OnChange(func(_, updated *config.Config) {
@@ -2794,6 +2820,12 @@ func reloadPluginImageResolvers(
if installation == nil {
continue
}
// Builtin installations resolve in-process metadata providers only;
// registering them here would claim their capability id as a gRPC
// image-resolver scheme with no binary behind it.
if installation.IsBuiltin() {
continue
}
capabilities, err := store.ListCapabilities(ctx, installation.ID)
if err != nil {
return fmt.Errorf("list image resolver capabilities for installation %d: %w", installation.ID, err)
@@ -2980,6 +3012,11 @@ func reloadMarkerPluginProviders(
if installation == nil {
continue
}
// Builtin installations expose no marker providers; defense in depth
// alongside the capability-type filter below.
if installation.IsBuiltin() {
continue
}
capabilities, err := store.ListCapabilities(ctx, installation.ID)
if err != nil {
return fmt.Errorf("list marker provider capabilities for installation %d: %w", installation.ID, err)
docs/architecture/local-nfo-metadata.md
+126
View File
@@ -0,0 +1,126 @@
# Local NFO metadata and sidecar artwork (as built)
Implements #216: Kodi/Jellyfin-style `.nfo` files and sidecar images
(`poster.jpg`, `fanart.jpg`, …) next to movies and shows are read by a builtin
metadata provider, so curated local libraries work with partial or zero
TMDB/TVDB coverage. Admin-facing usage lives in
`docs/wiki/admin/nfo-local-metadata.md`; this page records the architecture for
maintainers.
## Registration: a builtin provider in the plugin chain
Provider chains are keyed on `plugin_installations` rows, so builtins are
registered **as data**: a reserved installation (`plugin_id='silo.builtin'`,
`kind='builtin'`, sentinel `install_path`, `update_policy='manual'`) plus a
`metadata_provider.v1` capability `nfo` with `default_priority` 1 for
movie/series/season/episode and `default_enabled=false`
(`migrations/sql/20260712100125_builtin_host_providers.sql`,
`...124741_nfo_capability_season_episode_levels.sql`).
- `internal/metadata/builtin.go` is the in-process registry; providers
self-register (`internal/metadata/nfo/register.go`) and `buildProviders`
(`internal/metadata/chain.go`) returns the in-process provider for
`kind='builtin'` rows instead of constructing a gRPC plugin.
- Guard rails keep the reserved row out of every plugin surface (user plugin
settings, installations list, image resolvers, preload, auto-update, store
delete, mutation handlers); `silo.builtin` is a reserved manifest id. Generic
installation reads must **not** filter builtins — the chain enabled-check
depends on them.
- On startup, `SyncBuiltinProviderChains` (`internal/metadata/builtin_sync.go`)
first materializes legacy `content_level=''` chains per level, then appends
builtin capabilities to all existing chains, disabled. New libraries pick the
capability up through normal chain seeding. The chain-less fallback respects
`default_enabled=false`.
- **Everything is inert until an admin enables "NFO Files" on a library.**
## Identity: hint-first, never candidate-first
NFO `<uniqueid>` values (tmdb/imdb/tvdb) are **trusted identity hints** seeded
before remote search (`IdentityHintProvider`, applied in
`internal/metadata/service.go`), unlocking full remote enrichment even when
search-by-title fails. Conflict policy by mode:
- initial match / scheduled refresh: NFO hints beat folder-name hints; stored
durable IDs beat NFO hints (no background identity flips);
- manual refresh: NFO hints beat stored IDs — the recovery path for a corrected
NFO. The item's `content_id` re-anchors too (not just its stored provider
IDs): a provider-anchored item whose fixed `<uniqueid>` now derives a
different anchor is renamed to it, or merged onto the existing item already
holding that id;
- manual Identify: the user's choice wins; the NFO provider is skipped for the
whole operation.
The NFO search candidate exists only for the title-only case and is defanged:
ID-less candidates lose provider-priority tie-breaks to ID-bearing group
members, `nfo` never counts as corroboration, and Phase-2 NFO results cannot
inject provider-id keys. A title-only NFO yields a `matched` item under a
path-deterministic `local:` content id; adding a `<uniqueid>` later promotes it
to that provider anchor on the next refresh (this promotion from an unanchored
id runs on scheduled refresh too — only an *already-anchored* identity is held
stable in the background). NFO field edits propagate on **manual refresh only**
(scheduled refresh is fill-empty by design).
## Sidecar artwork: copy into the S3 image cache
Sidecar images (`poster`/`folder`/`cover`, `fanart`/`backdrop`/`background`,
`logo`/`clearlogo`, and `<basename>-poster/-fanart/-logo` variants; extensions
`.jpg/.jpeg/.png/.webp`, 8 MiB cap, symlinked leaves rejected) are discovered
by the provider's `GetImages` (`internal/metadata/nfo/images.go`). Clients —
including jellycompat — always receive the normal presigned
`poster_url`/`backdrop_url`/`logo_url`; library files are never served
directly, and API nodes never need filesystem access to libraries.
- Sources are recorded as `file://<absolute-logical-path>` in `*_source_path`
columns and cached by the metadata image-cache processor under
`local/{contentType}/{contentID}/{hash8}/{imageType}/...` (`hash8` = content
hash; the discriminator sits before the imageType segment because variant
clamping reads the parent directory). Editing the file and refreshing rotates
the key; the stale hashed prefix is deleted after a successful re-cache, and
item deletion trims local paths to the content prefix.
- The processor confines each read to the owning library's `media_folders`
roots: a lexical check on the logical path, then a symlink-resolving re-check
(both path and roots are resolved, so a legitimately symlinked root stays
valid while an intermediate directory symlink escaping a root is rejected).
Missing/unreadable/out-of-root and structurally-unusable (non-regular,
over-cap) paths are stable failures with a long retry deferral; recovery is
refresh-driven — the provider-artwork backfill sweep deliberately skips
`file://` sources.
- Generic filenames attach only when the directory holds a single content
group; a shared `folder.jpg` in a flat multi-movie folder applies to none.
`<basename>-poster.jpg` style names always attach to their file.
- `applyBestImages`' final write has a local exemption so rating-0 local art
can fill already-matched items and cannot be stickily displaced by remote
art; the localization pass skips language-neutral local candidates.
**Deployment constraint:** the host running the metadata image-cache processor
must mount the media libraries at the same paths as the scanner/metadata
worker, otherwise local artwork jobs fail until the mount is present.
## Series depth and mixed libraries
`SeasonsRequest`/`EpisodesRequest` carry local path context; `season.nfo`
supplies season name/plot, `<basename>.nfo` + `<basename>-thumb.ext` supply
episode metadata/thumbs (`internal/metadata/nfo/series_depth.go`). **Naming
supplies structure; NFOs supply metadata**: NFO season/episode numbers are
advisory — directory/filename-derived numbers win with a warning. Episode NFOs
work without a `season.nfo`; `SynthesizeFallbackEpisodes` still covers
NFO-less episodes.
In `mixed` libraries, movie-vs-series is decided per file by naming before any
provider runs (`internal/naming/filename.go`); the NFO provider is seeded into
all levels and behaves as in dedicated libraries. The provider's root-element
type guard means a `tvshow.nfo` next to a movie-classified file is ignored,
never applied; the per-root Type override is the correction path. This is the
contract for sports/mixed libraries (events as movies, weekly shows as series).
## Known limitations
- The admin image picker does not surface local art (automatic chain path
only).
- Multi-part movies: a basename-mismatched NFO in a folder holding multiple
content groups is not found (directory candidates are suppressed there).
- No NFO writing, no music/audiobook/ebook NFO (those ecosystems use
OPF/metadata.json — future builtin providers via the same registry).
- Feature detection: the `nfo` capability's presence in
`GET /api/v1/libraries/provider-defaults` (see
`docs/architecture/v1-scope.md`).
docs/architecture/v1-scope.md
+5
View File
@@ -17,3 +17,8 @@ When the scope locks, this file becomes the source of truth and will contain:
Until lock: treat any capability not tracked as `Proposed`/`Locked` on the project as out of scope
for feature PRs (see the scope gate in `CLAUDE.md`).
Feature-detection precedent: clients discover which metadata providers (including the
built-in NFO provider, #216) apply to a library type via
`GET /api/v1/libraries/provider-defaults` rather than version sniffing. New capabilities
should follow the same capability-endpoint pattern.
docs/wiki/admin/media-folder-and-naming.md
+8
View File
@@ -199,6 +199,14 @@ Silo often sees sidecar files that mirror the media basename:
These are common and expected. Episode naming guidance in this page refers to the media files
themselves, but sidecars may legitimately reuse the same stem.
Some NFO sidecars are actively read: `movie.nfo`, `tvshow.nfo`, `<media basename>.nfo`,
`season.nfo`, `<episode basename>.nfo`, and sidecar artwork (season posters, episode
thumbnails) feed the built-in NFO metadata provider — including `<uniqueid>` provider IDs,
which act as trusted matching anchors just like `{tmdb-...}` folder tags. NFO files never
change how files are grouped; naming and layout still decide series/season/episode
structure. See [Local NFO Metadata](nfo-local-metadata.md) for the supported fields and
merge behavior.
Supplemental directories next to a movie (and directly under a series root)
are scanned as **extras** attached to that item, following the Jellyfin/Plex
folder convention:
docs/wiki/admin/nfo-local-metadata.md
+177
View File
@@ -0,0 +1,177 @@
---
title: Local NFO Metadata
description: How Silo reads Kodi/Jellyfin-style NFO sidecar files and merges them with online metadata providers.
summary: Supported NFO fields, merge and refresh semantics, and the naming-supplies-structure contract for curated libraries.
tags:
- silo
- docs
- wiki
- libraries
- metadata
- nfo
audience:
- operator
- end-user
last_reviewed: 2026-07-12
related:
- ../index.md
- media-folder-and-naming.md
---
# Local NFO Metadata
Silo ships a built-in **NFO Files** metadata provider that reads Kodi/Jellyfin-style `.nfo`
sidecar files. It runs inside the same provider chain as online providers (TMDB, TVDB, ...),
so one toggle and one priority per library control it. By default it sits at priority 1: the
NFO wins every field it declares, and online providers fill in whatever it leaves out.
Clients can detect whether a library type includes the NFO provider via
`GET /api/v1/libraries/provider-defaults`.
## Where Silo looks for NFO files
For each movie or series, Silo checks (in order):
1. `movie.nfo` / `tvshow.nfo` in the item's folder
2. `<media basename>.nfo` next to the media file (e.g. `My Movie (2021).nfo`)
For series, Silo additionally reads:
- `season.nfo` inside each season directory (`<season>` root: title, plot)
- `<episode basename>.nfo` next to each episode file (`<episodedetails>` root: title,
plot, air date, runtime, ratings)
- Season posters: `poster.jpg` (or `folder`/`cover`) inside the season directory, or
`seasonNN-poster.jpg` in the series root
- Episode thumbnails: `<episode basename>-thumb.jpg`
A sidecar only applies when its root element matches the item type: a stray `movie.nfo` in
a series folder is skipped in favor of `tvshow.nfo`, and a `tvshow.nfo` next to a movie
file is ignored. Files that fail to parse are skipped silently and the next candidate is
tried — a broken NFO never blocks matching.
Season and episode numbers declared inside an NFO (`<seasonnumber>`, `<season>`,
`<episode>`) are advisory: the directory name and the `SxxEyy` filename pattern decide
where metadata lands, and a mismatch is logged and resolved in the filename's favor.
Multi-episode NFO documents (several `<episodedetails>` blocks in one file) are not
supported; the first block is used and a warning is logged.
## Supported fields
| NFO element | Applies to | Silo field |
|---|---|---|
| `<title>` | movie, series | Title |
| `<originaltitle>` | movie, series | Original title |
| `<tagline>` | movie, series | Tagline |
| `<plot>` | movie, series | Overview |
| `<year>` | movie, series | Year (derived from the premiere date when omitted) |
| `<runtime>` | movie, series | Runtime in minutes (non-numeric values ignored) |
| `<premiered>` / `<releasedate>` | movie | Release date (`YYYY-MM-DD`) |
| `<premiered>` / `<aired>` | series | First air date (`YYYY-MM-DD`) |
| `<mpaa>` | movie, series | Content rating |
| `<genre>` (repeated) | movie, series | Genres |
| `<studio>` (repeated) | movie, series | Studios |
| `<country>` (repeated) | movie, series | Countries |
| `<tag>` (repeated) | movie, series | Keywords |
| `<ratings>` / `<rating>` | movie, series | Ratings (named sources: `imdb`, `tmdb`/`themoviedb`, `tomatometerallcritics`, `tomatometerallaudience`; a legacy bare `<rating>` fills the IMDB slot) |
| `<actor>` (`name`/`role`/`order`) | movie, series | Cast (actor `<thumb>` URLs are ignored) |
| `<director>`, `<credits>` | movie, series | Crew (directors and writers) |
| `<uniqueid type="tmdb|imdb|tvdb">` | movie, series | Trusted identity anchor for matching |
| `<title>`, `<plot>` (`season.nfo`) | season | Season name and overview |
| `<title>`, `<plot>`, `<aired>`, `<runtime>`, `<ratings>` (`<episodedetails>`) | episode | Episode title, overview, air date, runtime, ratings |
Not yet supported: `<set>` (collections) — a planned follow-up. `<userrating>` is read
but not stored (Silo has no per-user rating field). Unknown elements are ignored, so
exports from Kodi, Jellyfin, or tinyMediaManager work as-is.
## How NFO data merges with online providers
- **NFO wins what it declares.** With the NFO provider at priority 1, every field the NFO
populates beats the same field from online providers.
- **Online providers backfill the rest.** Anything the NFO leaves out (cast, artwork,
runtime, ratings, ...) is filled by the next providers in the chain, keyed by the
`<uniqueid>` values when present.
- **Genres are all-or-nothing.** The first provider that supplies any genres supplies the
whole list; later providers do not append to it. An NFO with no `<genre>` entries leaves
genres entirely to online providers.
- **Field locks are respected.** Fields locked in the Edit Metadata dialog (including
Year and release/air dates) are never overwritten by any provider, NFO included.
## Editing an NFO after the initial match
Scheduled background refreshes only fill empty fields — they intentionally do **not**
re-apply NFO changes over existing values. To pick up edits:
**Edit the NFO, then use Refresh metadata on the item.** A manual refresh re-reads the
sidecar and replaces all unlocked fields with the merged provider results, NFO first.
## Naming supplies structure, NFO supplies presentation
NFO files describe *what an item is called and about* — they do not change *how files are
grouped*. Folder layout and filenames still decide what is a series, a season, and an
episode (see [Supported Media Folder Structures and Naming](media-folder-and-naming.md)).
This makes fully local libraries with no online match work well, for example a fitness
library:
```
Fitness/
P90X/
tvshow.nfo # show title/plot (no <uniqueid> needed)
poster.jpg fanart.jpg
Season 01/
season.nfo # "Course A: Classic"
poster.jpg # season poster
P90X S01E01 - Chest and Back.mkv
P90X S01E01 - Chest and Back.nfo # episode title/plot
P90X S01E01 - Chest and Back-thumb.jpg # episode thumbnail
```
The folder and episode naming build the series/season/episode tree; the NFO sidecars
supply every name, description, and image — even when no online provider knows the
content. Episodes without an `.nfo` still appear with their generic "Episode N" title,
so partially curated libraries degrade gracefully.
## Mixed libraries (sports example)
A single library of type **Mixed** can hold movie-shaped and show-shaped content side by
side — the common case is sports, where pay-per-view events are movies and the weekly
show is a series, and most of it has no TMDB/TVDB presence:
```
WWE/
WrestleMania 41 (2025)/
WrestleMania 41 (2025).mkv
movie.nfo # event title/plot (a <uniqueid> is optional)
poster.jpg
WWE SmackDown/
tvshow.nfo
Season 27/
season.nfo
poster.jpg
WWE SmackDown S27E15.mkv
WWE SmackDown S27E15.nfo
WWE SmackDown S27E15-thumb.jpg
```
How classification works in a mixed library:
- **Naming decides the type, per file, before any metadata provider runs.** An `SxxEyy`
pattern or a `Season NN` folder routes a file to the series lane; everything else is a
movie. An NFO never flips the type: a `tvshow.nfo` sitting next to a movie-classified
file is ignored (the root element must match the item type), not applied.
- **NFOs supply metadata and identity only.** Once the type is fixed, the NFO provider
behaves exactly as in dedicated movie/series libraries — including season and episode
depth for the show-shaped content. An event whose `movie.nfo` carries a
`<uniqueid type="tmdb">` still gets full online enrichment, while ID-less events next
to it fall back to their local NFO and artwork.
- **Misclassified group?** Override its **Type** on the root in the library admin
(Auto → Movie/Series) and rescan — that is the correction path; fixing the NFO alone
will not re-route a file.
Tools that generate sports libraries (sportarr-style managers) should therefore write
standard `SxxEyy` naming for show content and standalone movie naming (`Title (Year)/`)
for events, and place NFOs/posters exactly as in the dedicated-library layouts above.
## Source References
- `internal/metadata/nfo/` — parser and provider
- `internal/metadata/merge.go` — merge semantics and field locks
docs/wiki/index.md
+2
View File
@@ -35,6 +35,8 @@ rewriting.
reference for the folder layouts and filenames Silo can scan and match today.
- [Collection Templates](admin/collection-templates.md) - Curated, one-click starting points for
synced library collections sourced from TMDB, Trakt, and MDBList.
- [Local NFO Metadata](admin/nfo-local-metadata.md) - Supported NFO sidecar fields, how they merge
with online providers, and the naming-supplies-structure contract.
## Deployment
internal/api/handlers/libraries.go
+1 -16
View File
@@ -2202,22 +2202,7 @@ func buildSeededChainEntries(level string, candidates []seedCandidate) []metadat
}
func metadataContentLevelsForLibraryType(libraryType string) []string {
switch libraryType {
case "series":
return []string{"series", "season", "episode"}
case "movies", "movie":
return []string{"movie"}
case "audiobooks", "audiobook":
return []string{"audiobook"}
case "ebooks", "ebook":
return []string{"ebook"}
case "manga":
return []string{"manga"}
case "mixed":
return []string{"movie", "series", "season", "episode", "audiobook", "ebook"}
default:
return nil
}
return metadata.ContentLevelsForLibraryType(libraryType)
}
// HandleListStaleIDs handles GET /libraries/stale-ids.
internal/api/handlers/plugins.go
+54 -1
View File
@@ -157,6 +157,7 @@ type pluginInstallationResponse struct {
Version string `json:"version"`
InstallPath string `json:"install_path"`
Enabled bool `json:"enabled"`
Kind string `json:"kind"`
UpdatePolicy string `json:"update_policy"`
AvailableVersion *string `json:"available_version,omitempty"`
SourceKind string `json:"source_kind"`
@@ -882,6 +883,10 @@ func (h *PluginHandler) HandleUpdateInstallation(w http.ResponseWriter, r *http.
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load plugin installation")
return
}
if currentInstallation.IsBuiltin() {
writeError(w, http.StatusConflict, "builtin_installation", "Built-in host providers cannot be modified")
return
}
if req.Enabled != nil && !*req.Enabled && currentInstallation.Enabled && h.service != nil {
if err := h.service.Stop(id); err != nil && !errors.Is(err, pluginhost.ErrClientNotFound) {
@@ -933,6 +938,9 @@ func (h *PluginHandler) HandleApplyUpdate(w http.ResponseWriter, r *http.Request
writeError(w, http.StatusBadRequest, "bad_request", "Invalid installation ID")
return
}
if h.rejectBuiltinInstallation(w, r, id) {
return
}
installation, err := h.service.UpdateToAvailableVersion(r.Context(), id)
if err != nil {
@@ -974,6 +982,9 @@ func (h *PluginHandler) HandlePutInstallationConfig(w http.ResponseWriter, r *ht
writeError(w, http.StatusInternalServerError, "internal_error", "Plugin service not configured")
return
}
if h.rejectBuiltinInstallation(w, r, id) {
return
}
if err := h.service.SetGlobalConfig(r.Context(), id, req.Key, req.Value); err != nil {
var validationErr *plugins.ConfigValidationError
@@ -1012,6 +1023,9 @@ func (h *PluginHandler) HandleTestInstallationConfig(w http.ResponseWriter, r *h
writeError(w, http.StatusBadRequest, "bad_request", "key is required")
return
}
if h.rejectBuiltinInstallation(w, r, id) {
return
}
if err := h.service.TestGlobalConfig(r.Context(), id, req.Key, req.Value); err != nil {
if errors.Is(err, plugins.ErrInstallationNotFound) {
@@ -1046,6 +1060,10 @@ func (h *PluginHandler) HandlePutAuthBinding(w http.ResponseWriter, r *http.Requ
return
}
if h.rejectBuiltinInstallation(w, r, id) {
return
}
var req pluginAuthBindingRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
@@ -1079,6 +1097,10 @@ func (h *PluginHandler) HandlePutTaskBinding(w http.ResponseWriter, r *http.Requ
return
}
if h.rejectBuiltinInstallation(w, r, id) {
return
}
capabilityID := chi.URLParam(r, "capability_id")
if strings.TrimSpace(capabilityID) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "capability_id is required")
@@ -1106,6 +1128,21 @@ func (h *PluginHandler) HandlePutTaskBinding(w http.ResponseWriter, r *http.Requ
writeJSON(w, http.StatusOK, pluginTaskBindingUpdateResponse{RestartRequired: true})
}
// rejectBuiltinInstallation writes a 409 and returns true when the target
// installation is the reserved builtin row, which no plugin-management
// endpoint may mutate. Lookup errors are left to the caller's own handling.
func (h *PluginHandler) rejectBuiltinInstallation(w http.ResponseWriter, r *http.Request, id int) bool {
installation, err := h.installations.GetByID(r.Context(), id)
if err != nil {
return false
}
if !installation.IsBuiltin() {
return false
}
writeError(w, http.StatusConflict, "builtin_installation", "Built-in host providers cannot be modified")
return true
}
func (h *PluginHandler) HandleDeleteInstallation(w http.ResponseWriter, r *http.Request) {
id, err := parseNamedIDParam(r, "id")
if err != nil {
@@ -1118,6 +1155,10 @@ func (h *PluginHandler) HandleDeleteInstallation(w http.ResponseWriter, r *http.
writeError(w, http.StatusNotFound, "not_found", "Plugin installation not found")
return
}
if errors.Is(err, plugins.ErrBuiltinInstallationImmutable) {
writeError(w, http.StatusConflict, "builtin_installation", "Built-in host providers cannot be uninstalled")
return
}
slog.ErrorContext(r.Context(), "deleting plugin installation", "component", "api", "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to delete plugin installation")
return
@@ -1156,6 +1197,11 @@ func (h *PluginHandler) HandleListUserPluginSettings(w http.ResponseWriter, r *h
Installations: make([]pluginUserSettingsSummary, 0, len(installations)),
}
for _, installation := range installations {
// The reserved builtin row has no manifest on disk; without this skip
// the whole user-scoped settings list would 500.
if installation.IsBuiltin() {
continue
}
manifest, err := plugins.LoadManifestFile(plugins.InstalledManifestPath(installation.InstallPath))
if err != nil {
slog.ErrorContext(r.Context(), "loading plugin manifest", "component", "api", "installation_id", installation.ID, "error", err)
@@ -1238,7 +1284,7 @@ func (h *PluginHandler) loadUserConfigInstallation(
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load plugin installation")
return nil, nil, err
}
if !installation.Enabled {
if !installation.Enabled || installation.IsBuiltin() {
writeError(w, http.StatusNotFound, "not_found", "Plugin installation not found")
return nil, nil, plugins.ErrInstallationNotFound
}
@@ -1282,6 +1328,12 @@ func (h *PluginHandler) buildInstallationResponses(
}
response := make([]pluginInstallationResponse, 0, len(installations))
for _, installation := range installations {
// The reserved builtin row is not a manageable plugin: old web builds
// would render a phantom entry with uninstall/upgrade buttons that
// error, and the chain editor does not need it in this list.
if installation.IsBuiltin() {
continue
}
item, err := h.buildInstallationResponseWithBindings(
ctx,
installation,
@@ -1395,6 +1447,7 @@ func (h *PluginHandler) buildInstallationResponseWithBindings(
Version: installation.Version,
InstallPath: installation.InstallPath,
Enabled: installation.Enabled,
Kind: installation.Kind,
UpdatePolicy: installation.UpdatePolicy,
AvailableVersion: installation.AvailableVersion,
SourceKind: sourceKind,
internal/api/handlers/plugins_builtin_test.go
+170
View File
@@ -0,0 +1,170 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"strconv"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/plugins"
)
func pluginBuiltinTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func builtinTestHandler(pool *pgxpool.Pool) *PluginHandler {
return &PluginHandler{
repositories: plugins.NewRepositoryStore(pool),
installations: plugins.NewInstallationStore(pool),
configs: plugins.NewRuntimeConfigStore(pool),
}
}
func seedHandlerBuiltinInstallation(t *testing.T, pool *pgxpool.Pool) int {
t.Helper()
var id int
pluginID := fmt.Sprintf("test.builtin.handler-%d", time.Now().UnixNano())
err := pool.QueryRow(context.Background(),
`INSERT INTO plugin_installations (plugin_id, version, install_path, enabled, update_policy, kind)
VALUES ($1, '0', '/nonexistent/silo-builtin-test', true, 'manual', 'builtin')
RETURNING id`, pluginID).Scan(&id)
if err != nil {
t.Fatalf("seed builtin installation: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM plugin_installations WHERE id = $1`, id)
})
return id
}
func requestWithIDParam(method, target, param string, id int) *http.Request {
return requestWithIDParamBody(method, target, param, id, nil)
}
func requestWithIDParamBody(method, target, param string, id int, body io.Reader) *http.Request {
req := httptest.NewRequest(method, target, body)
rctx := chi.NewRouteContext()
rctx.URLParams.Add(param, strconv.Itoa(id))
return req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, rctx))
}
// The manifest-less builtin row must not 500 the user-scoped plugin settings
// list the moment the migration lands (ship-blocker: the web sidebar fetches
// this for every logged-in user).
func TestHandleListUserPluginSettings_SkipsBuiltinRow(t *testing.T) {
pool := pluginBuiltinTestPool(t)
builtinID := seedHandlerBuiltinInstallation(t, pool)
h := builtinTestHandler(pool)
rec := httptest.NewRecorder()
h.HandleListUserPluginSettings(rec, httptest.NewRequest(http.MethodGet, "/api/v1/settings/plugins", nil))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp struct {
Installations []struct {
ID int `json:"id"`
} `json:"installations"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
for _, inst := range resp.Installations {
if inst.ID == builtinID {
t.Errorf("builtin installation %d leaked into user plugin settings", builtinID)
}
}
}
// loadUserConfigInstallation must 404 builtin ids.
func TestHandleGetUserPluginSettings_BuiltinIDIs404(t *testing.T) {
pool := pluginBuiltinTestPool(t)
builtinID := seedHandlerBuiltinInstallation(t, pool)
h := builtinTestHandler(pool)
rec := httptest.NewRecorder()
h.HandleGetUserPluginSettings(rec, requestWithIDParam(http.MethodGet, "/api/v1/settings/plugins/0", "installation_id", builtinID))
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404; body=%s", rec.Code, rec.Body.String())
}
}
// GET /plugins/installations must filter builtin rows server-side, and the
// remaining rows carry the additive kind field.
func TestHandleListInstallations_FiltersBuiltinAndExposesKind(t *testing.T) {
pool := pluginBuiltinTestPool(t)
builtinID := seedHandlerBuiltinInstallation(t, pool)
h := builtinTestHandler(pool)
rec := httptest.NewRecorder()
h.HandleListInstallations(rec, httptest.NewRequest(http.MethodGet, "/api/v1/plugins/installations", nil))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp []struct {
ID int `json:"id"`
Kind string `json:"kind"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
for _, inst := range resp {
if inst.ID == builtinID {
t.Errorf("builtin installation %d leaked into installations list", builtinID)
}
if inst.Kind == "" {
t.Errorf("installation %d missing additive kind field", inst.ID)
}
}
}
// Mutating endpoints must reject the builtin row with a clear 4xx.
func TestBuiltinInstallationMutationsRejected(t *testing.T) {
pool := pluginBuiltinTestPool(t)
builtinID := seedHandlerBuiltinInstallation(t, pool)
h := builtinTestHandler(pool)
cases := []struct {
name string
call func(rec *httptest.ResponseRecorder)
}{
{"delete", func(rec *httptest.ResponseRecorder) {
h.HandleDeleteInstallation(rec, requestWithIDParam(http.MethodDelete, "/api/v1/plugins/installations/0", "id", builtinID))
}},
{"update", func(rec *httptest.ResponseRecorder) {
req := requestWithIDParamBody(http.MethodPatch, "/api/v1/plugins/installations/0", "id", builtinID,
strings.NewReader(`{"enabled": false}`))
h.HandleUpdateInstallation(rec, req)
}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rec := httptest.NewRecorder()
tc.call(rec)
if rec.Code < 400 || rec.Code >= 500 {
t.Fatalf("status = %d, want 4xx; body=%s", rec.Code, rec.Body.String())
}
})
}
}
internal/catalog/folder_repo.go
+43 -1
View File
@@ -895,6 +895,22 @@ func dirSetToSlice(set map[string]struct{}) []string {
return dirs
}
// imageDeletePrefix returns the S3 prefix to sweep when deleting the content
// that owns the cached image at path. Local artwork keys carry a content-hash
// segment (local/{contentType}/{contentID}/{hash8}/{imageType}/...), so local
// paths trim to local/{contentType}/{contentID}/ — deleting every stale hash
// prefix, not just the live one. Other keys keep the per-image-type dir.
func imageDeletePrefix(path string) string {
if !strings.HasPrefix(path, "local/") {
return pathDir(path)
}
segments := strings.SplitN(path, "/", 4)
if len(segments) < 4 {
return pathDir(path)
}
return segments[0] + "/" + segments[1] + "/" + segments[2] + "/"
}
// pathDir extracts the directory portion of an S3 image path.
// e.g. "tmdb/movies/550/poster/original.webp" → "tmdb/movies/550/poster/"
func pathDir(path string) string {
@@ -964,7 +980,7 @@ func collectRawImageDirs(ctx context.Context, q rowQuerier, contentIDs []string)
}
for _, p := range []string{p1, p2, p3} {
if p != "" && !strings.Contains(p, "://") {
if dir := pathDir(p); dir != "" {
if dir := imageDeletePrefix(p); dir != "" {
dirSet[dir] = struct{}{}
}
}
@@ -1032,6 +1048,32 @@ func filterUnreferencedImageDirs(ctx context.Context, q rowQuerier, dirs, deleti
}
// UpdateLastScanned sets the last_scanned_at timestamp for the given folder.
// LibraryRootsForContent returns the media folder root paths the given
// content belongs to, resolved through media_item_libraries. The metadata
// image-cache processor uses this to confine local file:// artwork sources
// (metadata.LibraryRootResolver).
func (r *FolderRepository) LibraryRootsForContent(ctx context.Context, contentID string) ([]string, error) {
rows, err := r.pool.Query(ctx, `
SELECT DISTINCT mfp.path
FROM media_folder_paths mfp
JOIN media_item_libraries mil ON mil.media_folder_id = mfp.media_folder_id
WHERE mil.content_id = $1
`, contentID)
if err != nil {
return nil, fmt.Errorf("querying library roots for content %q: %w", contentID, err)
}
defer rows.Close()
var paths []string
for rows.Next() {
var path string
if err := rows.Scan(&path); err != nil {
return nil, fmt.Errorf("scanning library root path: %w", err)
}
paths = append(paths, path)
}
return paths, rows.Err()
}
func (r *FolderRepository) UpdateLastScanned(ctx context.Context, id int, scannedAt time.Time) error {
tag, err := r.pool.Exec(ctx,
"UPDATE media_folders SET last_scanned_at = $1 WHERE id = $2",
internal/catalog/folder_repo_local_roots_test.go
+77
View File
@@ -0,0 +1,77 @@
package catalog
import (
"context"
"fmt"
"os"
"sort"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestLibraryRootsForContent(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
suffix := time.Now().UnixNano()
contentID := fmt.Sprintf("roots-test-%d", suffix)
var folderID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_folders (type, name, enabled) VALUES ('movies', $1, true) RETURNING id`,
fmt.Sprintf("roots-test-%d", suffix)).Scan(&folderID); err != nil {
t.Fatalf("seed folder: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_item_libraries WHERE content_id = $1`, contentID)
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = $1`, contentID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folder_paths WHERE media_folder_id = $1`, folderID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
if _, err := pool.Exec(ctx,
`INSERT INTO media_items (content_id, type, title, status, genres) VALUES ($1, 'movie', 'Roots Test', 'matched', '{}'::text[])`,
contentID); err != nil {
t.Fatalf("seed item: %v", err)
}
for _, path := range []string{"/media/movies-a", "/media/movies-b"} {
if _, err := pool.Exec(ctx,
`INSERT INTO media_folder_paths (media_folder_id, path) VALUES ($1, $2)`,
folderID, fmt.Sprintf("%s-%d", path, suffix)); err != nil {
t.Fatalf("seed folder path: %v", err)
}
}
if _, err := pool.Exec(ctx,
`INSERT INTO media_item_libraries (content_id, media_folder_id) VALUES ($1, $2)`,
contentID, folderID); err != nil {
t.Fatalf("seed membership: %v", err)
}
repo := NewFolderRepository(pool)
roots, err := repo.LibraryRootsForContent(ctx, contentID)
if err != nil {
t.Fatalf("LibraryRootsForContent: %v", err)
}
sort.Strings(roots)
want := []string{
fmt.Sprintf("/media/movies-a-%d", suffix),
fmt.Sprintf("/media/movies-b-%d", suffix),
}
if len(roots) != 2 || roots[0] != want[0] || roots[1] != want[1] {
t.Fatalf("roots = %v, want %v", roots, want)
}
// Unknown content has no roots (and no error).
none, err := repo.LibraryRootsForContent(ctx, contentID+"-missing")
if err != nil || len(none) != 0 {
t.Fatalf("missing content: roots=%v err=%v", none, err)
}
}
internal/catalog/item_repo.go
+1 -1
View File
@@ -834,7 +834,7 @@ func (r *ItemRepository) Delete(ctx context.Context, contentID string) ([]string
}
for _, p := range []string{p1, p2, p3} {
if p != "" && !strings.Contains(p, "://") {
if dir := pathDir(p); dir != "" {
if dir := imageDeletePrefix(p); dir != "" {
imageDirs = append(imageDirs, dir)
}
}
internal/catalog/local_image_paths_test.go
+37
View File
@@ -0,0 +1,37 @@
package catalog
import "testing"
func TestImageTypeFromCachedPathLocalHashedLayout(t *testing.T) {
// Local keys interpose a content-hash segment BEFORE the image type
// (local/{contentType}/{contentID}/{hash8}/{imageType}/{variant}.{ext})
// so the variant's parent directory stays the image type.
cases := map[string]string{
"local/movies/movie-1/deadbeef/poster/original.webp": "poster",
"local/movies/movie-1/deadbeef/backdrop/original.webp": "backdrop",
"local/series/show-1/cafef00d/logo/w500.webp": "logo",
}
for path, want := range cases {
if got := imageTypeFromCachedPath(path); got != want {
t.Errorf("imageTypeFromCachedPath(%q) = %q, want %q", path, got, want)
}
}
}
func TestImageDeletePrefixTrimsLocalPathsToContentRoot(t *testing.T) {
// On item deletion, local paths trim to local/{contentType}/{contentID}
// so every stale hash prefix is swept, not just the live one.
cases := map[string]string{
"local/movies/movie-1/deadbeef/poster/original.webp": "local/movies/movie-1/",
"local/series/show-1/cafef00d/logo/w500.webp": "local/series/show-1/",
// Remote cached keys keep the existing per-image-type prefix.
"tmdb/movies/550/poster/original.webp": "tmdb/movies/550/poster/",
// Legacy scanner keys without a hash segment stay whole-path dirs.
"local/ebooks/book-1/poster/original.webp": "local/ebooks/book-1/",
}
for path, want := range cases {
if got := imageDeletePrefix(path); got != want {
t.Errorf("imageDeletePrefix(%q) = %q, want %q", path, got, want)
}
}
}
internal/imagecache/imagecache.go
+70 -29
View File
@@ -63,6 +63,10 @@ type CacheRequest struct {
EpisodeNumber *int
Language string
ImageResolver ImageURLResolver // optional; used when SourceURL is a plugin:// path
// KeyDiscriminator, when set, is inserted into the S3 key between the
// content ID and the image type (local sidecar art uses the file's 8-hex
// content hash) so re-cached art rotates to a fresh key.
KeyDiscriminator string
}
// CacheResult is returned by Cache on success.
@@ -120,6 +124,29 @@ func (c *Cacher) CacheImage(ctx context.Context, req metadata.CacheImageRequest)
if err != nil {
return nil, err
}
return cacheImageResultFromCacheResult(result), nil
}
// CacheImageBytes implements metadata.ImageByteCacher using CacheBytes. Used
// by the image cache processor for file:// sources that it reads itself.
func (c *Cacher) CacheImageBytes(ctx context.Context, data []byte, req metadata.CacheImageRequest) (*metadata.CacheImageResult, error) {
result, err := c.CacheBytes(ctx, data, CacheRequest{
ProviderID: req.ProviderID,
ContentType: req.ContentType,
ContentID: req.ContentID,
ImageType: req.ImageType,
SeasonNumber: req.SeasonNumber,
EpisodeNumber: req.EpisodeNumber,
Language: req.Language,
KeyDiscriminator: req.KeyDiscriminator,
})
if err != nil {
return nil, err
}
return cacheImageResultFromCacheResult(result), nil
}
func cacheImageResultFromCacheResult(result *CacheResult) *metadata.CacheImageResult {
return &metadata.CacheImageResult{
BasePath: result.BasePath,
OriginalPath: result.OriginalPath,
@@ -128,7 +155,7 @@ func (c *Cacher) CacheImage(ctx context.Context, req metadata.CacheImageRequest)
Ext: result.Ext,
UploadedVariants: result.UploadedVariants,
ExistingVariants: result.ExistingVariants,
}, nil
}
}
// CacheAudiobookCover is a thin convenience over CacheBytes specifically
@@ -165,19 +192,35 @@ func (c *Cacher) CacheEbookCover(ctx context.Context, data []byte, contentID str
return res.OriginalPath, res.Thumbhash, nil
}
// validateCacheRequest checks the required identity fields and the
// episode/season invariant shared by Cache and CacheBytes. Keeping it in one
// place prevents the season/episode guard from drifting between the two paths:
// buildBasePath only appends the episode segment inside the SeasonNumber branch,
// so an episode without a season would silently collide distinct episodes' art
// under the same S3 key.
func validateCacheRequest(req CacheRequest) error {
if strings.TrimSpace(req.ProviderID) == "" {
return fmt.Errorf("imagecache: provider ID is required")
}
if strings.TrimSpace(req.ContentType) == "" {
return fmt.Errorf("imagecache: content type is required")
}
if strings.TrimSpace(req.ContentID) == "" {
return fmt.Errorf("imagecache: content ID is required")
}
if req.EpisodeNumber != nil && req.SeasonNumber == nil {
return fmt.Errorf("imagecache: episode number requires a season number")
}
return nil
}
// CacheBytes performs the same variant generation, thumbhash, and S3 upload as
// Cache but starts from raw image bytes already in hand. Used by the
// audiobook scanner to push embedded M4B cover art into S3 without round-
// tripping through HTTP.
func (c *Cacher) CacheBytes(ctx context.Context, data []byte, req CacheRequest) (*CacheResult, error) {
if strings.TrimSpace(req.ProviderID) == "" {
return nil, fmt.Errorf("imagecache: provider ID is required")
}
if strings.TrimSpace(req.ContentType) == "" {
return nil, fmt.Errorf("imagecache: content type is required")
}
if strings.TrimSpace(req.ContentID) == "" {
return nil, fmt.Errorf("imagecache: content ID is required")
if err := validateCacheRequest(req); err != nil {
return nil, err
}
if len(data) == 0 {
return nil, fmt.Errorf("imagecache: image data is empty")
@@ -191,7 +234,7 @@ func (c *Cacher) CacheBytes(ctx context.Context, data []byte, req CacheRequest)
if err != nil {
return nil, fmt.Errorf("imagecache: generate variants: %w", err)
}
basePath := buildBasePath(req.ProviderID, req.ContentType, req.ContentID, req.ImageType, req.Language, req.SeasonNumber, req.EpisodeNumber)
basePath := buildBasePath(req)
bucket := c.s3.Bucket()
revision := variantRevision(result)
variantPaths := buildVariantPaths(basePath, revision, result)
@@ -218,17 +261,8 @@ func (c *Cacher) CacheBytes(ctx context.Context, data []byte, req CacheRequest)
// Cache downloads the image at req.SourceURL and stores it through the same
// variant, revision-tracking, and upload pipeline as CacheBytes.
func (c *Cacher) Cache(ctx context.Context, req CacheRequest) (*CacheResult, error) {
if strings.TrimSpace(req.ProviderID) == "" {
return nil, fmt.Errorf("imagecache: provider ID is required")
}
if strings.TrimSpace(req.ContentType) == "" {
return nil, fmt.Errorf("imagecache: content type is required")
}
if strings.TrimSpace(req.ContentID) == "" {
return nil, fmt.Errorf("imagecache: content ID is required")
}
if req.EpisodeNumber != nil && req.SeasonNumber == nil {
return nil, fmt.Errorf("imagecache: episode number requires a season number")
if err := validateCacheRequest(req); err != nil {
return nil, err
}
url := req.SourceURL
@@ -387,18 +421,25 @@ func putObjectWithRetry(ctx context.Context, putter ObjectPutter, bucket, key st
// season: {provider}/{type}/{id}/seasons/{n}/{imageType}
// localized season: {provider}/{type}/{id}/localizations/{lang}/seasons/{n}/{imageType}
// episode: {provider}/{type}/{id}/seasons/{n}/episodes/{m}/{imageType}
func buildBasePath(providerID, contentType, contentID string, t metadata.ImageType, language string, seasonNumber, episodeNumber *int) string {
imageTypeName := imageTypeName(t)
base := fmt.Sprintf("%s/%s/%s", providerID, contentType, contentID)
if lang := normalizeImageLanguage(language); lang != "" {
//
// A non-empty KeyDiscriminator (local sidecar content hash) is inserted
// immediately before the image type so the variant's parent directory stays
// the image type segment (the imageTypeFromCachedPath contract).
func buildBasePath(req CacheRequest) string {
imageTypeName := imageTypeName(req.ImageType)
base := fmt.Sprintf("%s/%s/%s", req.ProviderID, req.ContentType, req.ContentID)
if lang := normalizeImageLanguage(req.Language); lang != "" {
base = fmt.Sprintf("%s/localizations/%s", base, lang)
}
if seasonNumber != nil {
base = fmt.Sprintf("%s/seasons/%d", base, *seasonNumber)
if episodeNumber != nil {
base = fmt.Sprintf("%s/episodes/%d", base, *episodeNumber)
if req.SeasonNumber != nil {
base = fmt.Sprintf("%s/seasons/%d", base, *req.SeasonNumber)
if req.EpisodeNumber != nil {
base = fmt.Sprintf("%s/episodes/%d", base, *req.EpisodeNumber)
}
}
if discriminator := strings.TrimSpace(req.KeyDiscriminator); discriminator != "" {
base = base + "/" + discriminator
}
return base + "/" + imageTypeName
}
internal/imagecache/local_key_test.go
+76
View File
@@ -0,0 +1,76 @@
package imagecache
import (
"context"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
)
func TestCacheBytesKeyDiscriminatorPrecedesImageType(t *testing.T) {
s3 := &mockS3{bucket: "images"}
cacher := New(s3)
data := makeTestJPEG(t)
res, err := cacher.CacheBytes(context.Background(), data, CacheRequest{
ProviderID: "local",
ContentType: "movies",
ContentID: "movie-1",
ImageType: metadata.ImagePoster,
KeyDiscriminator: "deadbeef",
})
if err != nil {
t.Fatalf("CacheBytes: %v", err)
}
// Hash segment sits BEFORE the image type so the variant's parent
// directory stays the image type (imageTypeFromCachedPath contract).
if res.BasePath != "local/movies/movie-1/deadbeef/poster" {
t.Fatalf("BasePath = %q", res.BasePath)
}
for _, call := range s3.calls {
if !strings.HasPrefix(call.key, "local/movies/movie-1/deadbeef/poster/") {
t.Fatalf("uploaded key %q outside discriminated prefix", call.key)
}
}
}
func TestCacheBytesWithoutDiscriminatorKeepsLegacyLayout(t *testing.T) {
s3 := &mockS3{bucket: "images"}
cacher := New(s3)
res, err := cacher.CacheBytes(context.Background(), makeTestJPEG(t), CacheRequest{
ProviderID: "local",
ContentType: "ebooks",
ContentID: "book-1",
ImageType: metadata.ImagePoster,
})
if err != nil {
t.Fatalf("CacheBytes: %v", err)
}
if res.BasePath != "local/ebooks/book-1/poster" {
t.Fatalf("BasePath = %q", res.BasePath)
}
}
func TestCacheImageBytesAdapter(t *testing.T) {
s3 := &mockS3{bucket: "images"}
cacher := New(s3)
res, err := cacher.CacheImageBytes(context.Background(), makeTestJPEG(t), metadata.CacheImageRequest{
ProviderID: "local",
ContentType: "movies",
ContentID: "movie-1",
ImageType: metadata.ImageBackdrop,
KeyDiscriminator: "cafef00d",
})
if err != nil {
t.Fatalf("CacheImageBytes: %v", err)
}
if res.BasePath != "local/movies/movie-1/cafef00d/backdrop" {
t.Fatalf("BasePath = %q", res.BasePath)
}
if res.Thumbhash == "" {
t.Fatal("thumbhash missing")
}
}
internal/metadata/builtin.go
+73
View File
@@ -0,0 +1,73 @@
package metadata
import (
"context"
"sort"
"sync"
)
// Built-in host metadata providers are registered as data: one reserved
// plugin_installations row (kind='builtin', plugin_id='silo.builtin') carries
// ordinary plugin_capabilities rows, so the chain store, seeding, defaults
// endpoint, and admin UI all work unchanged. This file is the single
// in-process registration point mapping those capability ids to provider
// constructors: buildProviders returns the registered provider instead of a
// gRPC PluginProvider when the resolved entry's installation is a builtin.
//
// Providers self-register from their package's init (e.g. internal/metadata/nfo)
// to avoid an import cycle — this package defines the Provider interfaces the
// builtin packages implement. cmd/silo blank-imports each builtin package.
// Activating a future builtin (e.g. xattr) is a registration call plus a
// capability row in a migration.
// IdentityHintProvider is the narrow contract a built-in local provider
// implements to contribute trusted identity hints (tmdb/imdb/tvdb ids parsed
// from curated sidecar files) before Phase-1 search. Hints anchor candidate
// selection through the trusted-hint machinery, unlocking remote enrichment
// by ID even when remote search-by-title returns nothing.
type IdentityHintProvider interface {
IdentityHints(ctx context.Context, query SearchQuery) map[string]string
}
var (
builtinProvidersMu sync.RWMutex
builtinProviders = map[string]func() Provider{}
)
// RegisterBuiltinProvider registers the constructor for a built-in provider
// under its plugin_capabilities capability_id. Later registrations for the
// same capability id replace earlier ones.
func RegisterBuiltinProvider(capabilityID string, construct func() Provider) {
// Registration only happens from package init(); empty id or nil constructor
// is a programmer error that would silently disable a provider, so fail loud.
if capabilityID == "" || construct == nil {
panic("metadata: RegisterBuiltinProvider requires a non-empty capabilityID and non-nil constructor")
}
builtinProvidersMu.Lock()
builtinProviders[capabilityID] = construct
builtinProvidersMu.Unlock()
}
// BuiltinCapabilityIDs returns the registered builtin capability ids, sorted.
func BuiltinCapabilityIDs() []string {
builtinProvidersMu.RLock()
ids := make([]string, 0, len(builtinProviders))
for id := range builtinProviders {
ids = append(ids, id)
}
builtinProvidersMu.RUnlock()
sort.Strings(ids)
return ids
}
// builtinProvider constructs the registered in-process provider for a builtin
// capability id.
func builtinProvider(capabilityID string) (Provider, bool) {
builtinProvidersMu.RLock()
construct, ok := builtinProviders[capabilityID]
builtinProvidersMu.RUnlock()
if !ok {
return nil, false
}
return construct(), true
}
internal/metadata/builtin_sync.go
+150
View File
@@ -0,0 +1,150 @@
package metadata
import (
"context"
"fmt"
)
// ContentLevelsForLibraryType maps a media_folders.type to the metadata
// content levels it serves. It is the server-side source of truth shared by
// chain seeding, the provider-defaults endpoint, and the startup builtin
// chain sync.
func ContentLevelsForLibraryType(libraryType string) []string {
switch libraryType {
case "series":
return []string{"series", "season", "episode"}
case "movies", "movie":
return []string{"movie"}
case "audiobooks", "audiobook":
return []string{"audiobook"}
case "ebooks", "ebook":
return []string{"ebook"}
case "manga":
return []string{"manga"}
case "mixed":
return []string{"movie", "series", "season", "episode", "audiobook", "ebook"}
default:
return nil
}
}
// SyncBuiltinProviderChains makes every existing library chain aware of the
// registered builtin host providers. It is idempotent and runs at startup
// before serving (callers must invalidate the chain cache afterwards when the
// service is already constructed). It also doubles as the repair path: if a
// stale chain-editor save drops a builtin row (SetChain is
// delete-all-and-reinsert), the row is re-appended disabled on next startup.
//
// Two steps, in order:
//
// 1. Legacy-” materialization: a library whose chain rows are only
// content_level=” predates per-level chains. AppendProviderToAllChains
// cannot insert anything for it (no per-level groups exist), yet the chain
// editor overlays per-level defaults for levels with no saved rows and the
// first save would materialize those defaults over the admin's legacy
// ordering. The sync therefore copies the ” rows to every content level
// the library type serves (same order/enabled) so the editor shows real
// rows; the ” rows stay in place for old binaries. GetChain's legacy
// fallback only applied when a level had zero rows, so resolved chains are
// unchanged.
// 2. For every builtin metadata capability (read from the database, the
// source of truth the registry mirrors), append it to all existing
// per-level chains it supports — disabled at MAX(priority)+1,
// ON CONFLICT DO NOTHING — exactly like a plugin install does.
func SyncBuiltinProviderChains(ctx context.Context, chainRepo *ChainRepository) error {
if chainRepo == nil {
return nil
}
pool := chainRepo.Pool()
if err := materializeLegacyChains(ctx, chainRepo); err != nil {
return err
}
rows, err := pool.Query(ctx,
`SELECT pc.plugin_installation_id, pc.capability_id
FROM plugin_capabilities pc
JOIN plugin_installations pi ON pi.id = pc.plugin_installation_id
WHERE pi.kind = 'builtin' AND pc.capability_type = 'metadata_provider.v1'
ORDER BY pc.plugin_installation_id, pc.capability_id`)
if err != nil {
return fmt.Errorf("listing builtin metadata capabilities: %w", err)
}
type builtinCap struct {
installationID int
capabilityID string
}
var caps []builtinCap
for rows.Next() {
var c builtinCap
if err := rows.Scan(&c.installationID, &c.capabilityID); err != nil {
rows.Close()
return fmt.Errorf("scanning builtin capability: %w", err)
}
caps = append(caps, c)
}
rows.Close()
if err := rows.Err(); err != nil {
return fmt.Errorf("iterating builtin capabilities: %w", err)
}
for _, c := range caps {
if err := chainRepo.AppendProviderToAllChains(ctx, c.installationID, c.capabilityID, func(level string) SeedPlacement {
return LookupSeedPlacement(ctx, pool, c.installationID, c.capabilityID, level)
}); err != nil {
return fmt.Errorf("appending builtin capability %q to chains: %w", c.capabilityID, err)
}
}
return nil
}
// materializeLegacyChains copies content_level=” chain rows to every content
// level the library type serves, for libraries that have only legacy rows.
// The copy preserves order and enabled state; existing rows win via
// ON CONFLICT DO NOTHING, and the ” rows are left untouched.
func materializeLegacyChains(ctx context.Context, chainRepo *ChainRepository) error {
pool := chainRepo.Pool()
rows, err := pool.Query(ctx,
`SELECT c.media_folder_id, f.type
FROM library_provider_chains c
JOIN media_folders f ON f.id = c.media_folder_id
GROUP BY c.media_folder_id, f.type
HAVING COUNT(*) FILTER (WHERE c.content_level <> '') = 0`)
if err != nil {
return fmt.Errorf("listing legacy-chain libraries: %w", err)
}
type legacyFolder struct {
folderID int
libraryType string
}
var folders []legacyFolder
for rows.Next() {
var f legacyFolder
if err := rows.Scan(&f.folderID, &f.libraryType); err != nil {
rows.Close()
return fmt.Errorf("scanning legacy-chain library: %w", err)
}
folders = append(folders, f)
}
rows.Close()
if err := rows.Err(); err != nil {
return fmt.Errorf("iterating legacy-chain libraries: %w", err)
}
for _, f := range folders {
for _, level := range ContentLevelsForLibraryType(f.libraryType) {
if _, err := pool.Exec(ctx,
`INSERT INTO library_provider_chains
(media_folder_id, plugin_installation_id, capability_id, capability_type, content_level, priority, enabled)
SELECT media_folder_id, plugin_installation_id, capability_id, capability_type, $2, priority, enabled
FROM library_provider_chains
WHERE media_folder_id = $1 AND content_level = ''
ON CONFLICT DO NOTHING`,
f.folderID, level,
); err != nil {
return fmt.Errorf("materializing legacy chain for folder %d level %q: %w", f.folderID, level, err)
}
}
}
return nil
}
internal/metadata/builtin_test.go
+65
View File
@@ -0,0 +1,65 @@
package metadata
import (
"slices"
"testing"
)
type builtinStubProvider struct{ slug string }
func (p *builtinStubProvider) Slug() string { return p.slug }
func (p *builtinStubProvider) Name() string { return p.slug }
func (p *builtinStubProvider) ForTypes() []string { return []string{"movie", "series"} }
func TestBuiltinProviderRegistry(t *testing.T) {
// The test registers under a synthetic capability id so it cannot collide
// with real registrations (the nfo package self-registers on import).
RegisterBuiltinProvider("test-builtin-cap", func() Provider {
return &builtinStubProvider{slug: "test-builtin-cap"}
})
ids := BuiltinCapabilityIDs()
if !slices.Contains(ids, "test-builtin-cap") {
t.Fatalf("BuiltinCapabilityIDs() = %v, want to contain test-builtin-cap", ids)
}
p, ok := builtinProvider("test-builtin-cap")
if !ok || p == nil {
t.Fatalf("builtinProvider(test-builtin-cap) = %v, %v; want constructed provider", p, ok)
}
if p.Slug() != "test-builtin-cap" {
t.Errorf("provider slug = %q", p.Slug())
}
if _, ok := builtinProvider("unregistered"); ok {
t.Error("builtinProvider(unregistered) = ok, want miss")
}
}
// The chain-less fallback must skip capabilities that opt out via
// default_enabled=false (absent defaults to true) — otherwise a disabled
// seeded NFO row (or a default_enabled=false specialist plugin) would
// activate itself the moment a level's enabled-filter yields zero entries.
func TestProviderEligibleForFallback(t *testing.T) {
cases := []struct {
name string
metadataJSON string
contentLevel string
want bool
}{
{"default enabled provider eligible", tmdbCapMetadata, "movie", true},
{"no metadata eligible", ``, "movie", true},
{"default_enabled=false excluded", `{"metadata":{"default_priority":{"movie":1},"default_enabled":false}}`, "movie", false},
{"nfo capability metadata excluded", `{"display_name":"NFO Files","default_priority":{"movie":1,"series":1},"default_enabled":false}`, "movie", false},
{"default_enabled=false excluded even on declared level", `{"metadata":{"default_priority":{"series":50},"default_enabled":false}}`, "series", false},
{"unsupported level still excluded", tvdbCapMetadata, "movie", false},
{"explicit true eligible", `{"metadata":{"default_priority":{"movie":2},"default_enabled":true}}`, "movie", true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := providerEligibleForFallback([]byte(tc.metadataJSON), tc.contentLevel); got != tc.want {
t.Fatalf("providerEligibleForFallback(%q, %q) = %v, want %v", tc.metadataJSON, tc.contentLevel, got, tc.want)
}
})
}
}
internal/metadata/canonicalize.go
+20
View File
@@ -52,6 +52,26 @@ func (s *MetadataService) canonicalizeLocalContentID(
if s == nil || !contentid.IsLocal(from) {
return from, nil
}
return s.reanchorContentID(ctx, from, ids, itemType)
}
// reanchorContentID moves `from` to the provider-anchored content_id derived
// from `ids` when the two differ, merging onto an existing target or renaming a
// free one. It is the shared core behind two callers: the local-skeleton
// promotion (canonicalizeLocalContentID) and the manual-refresh corrected-
// identity re-anchor (a fixed <uniqueid> in an NFO whose item was already
// anchored to the wrong provider id). Both must hold the provider-dedup lock so
// the target id cannot be claimed underneath us. When ids do not derive a
// provider-anchored id, or it equals `from`, this is a no-op.
func (s *MetadataService) reanchorContentID(
ctx context.Context,
from string,
ids contentid.ProviderIDs,
itemType string,
) (string, error) {
if s == nil {
return from, nil
}
// Derive with no path fallback: we only want a provider-anchored id here, not
// another local value.
internal/metadata/chain.go
+81 -4
View File
@@ -345,7 +345,19 @@ func resolveEnabledProviders(
if err != nil {
return nil, err
}
return buildProviders(ctx, caps, resolver, pool, checker), nil
// Exclude capabilities that opt out of default-enable (e.g. the seeded-
// disabled built-in NFO provider). Like the chain-less fallback, an
// installed-but-off provider must not activate itself through this
// content-level-agnostic path (person refresh); only an admin enabling its
// chain row does. Absent/unparseable metadata stays eligible (legacy).
eligible := make([]CapabilityInfo, 0, len(caps))
for _, c := range caps {
if !extractDefaultEnabled(lookupCapabilityMetadata(ctx, pool, c.PluginInstallationID, c.CapabilityID)) {
continue
}
eligible = append(eligible, c)
}
return buildProviders(ctx, eligible, resolver, pool, checker), nil
}
// resolveEnabledProvidersByPriority returns all enabled providers sorted by
@@ -375,8 +387,8 @@ func resolveEnabledProvidersByPriority(
items := make([]ranked, 0, len(caps))
for _, c := range caps {
metadataJSON := lookupCapabilityMetadata(ctx, pool, c.PluginInstallationID, c.CapabilityID)
if !providerSupportsLevel(metadataJSON, contentLevel) {
slog.DebugContext(ctx, "skipping metadata provider: does not declare support for content level", "component", "metadata",
if !providerEligibleForFallback(metadataJSON, contentLevel) {
slog.DebugContext(ctx, "skipping metadata provider: not eligible for chain-less fallback", "component", "metadata",
"installation_id", c.PluginInstallationID,
"capability_id", c.CapabilityID,
"content_level", contentLevel)
@@ -421,6 +433,17 @@ func providerSupportsLevel(metadataJSON []byte, contentLevel string) bool {
return levels[contentLevel] > 0
}
// providerEligibleForFallback reports whether a capability may participate in
// the chain-less fallback for contentLevel. The fallback runs not only for
// chain-less libraries but whenever a level's enabled-filter yields zero
// entries, so a capability that opts out of being enabled by default
// (default_enabled=false — e.g. the seeded-disabled built-in NFO provider or a
// specialist plugin) must not activate itself through it. Absent or
// unparseable metadata defaults to eligible, preserving legacy behavior.
func providerEligibleForFallback(metadataJSON []byte, contentLevel string) bool {
return providerSupportsLevel(metadataJSON, contentLevel) && extractDefaultEnabled(metadataJSON)
}
// lookupCapabilityMetadata returns the raw plugin_capabilities.metadata JSON for
// a provider's exact metadata_provider.v1 capability, or nil if absent.
func lookupCapabilityMetadata(ctx context.Context, pool *pgxpool.Pool, pluginInstallationID int, capabilityID string) []byte {
@@ -599,12 +622,30 @@ func buildProviders(
providers := make([]Provider, 0, len(caps))
for _, c := range caps {
enabled, err := installationEnabled(ctx, checker, pool, c.PluginInstallationID)
if err != nil || !enabled {
if err != nil {
slog.WarnContext(ctx, "skipping metadata provider: installation enabled-check failed", "component", "metadata",
"installation_id", c.PluginInstallationID, "capability_id", c.CapabilityID, "error", err)
continue
}
if !enabled {
slog.DebugContext(ctx, "skipping metadata provider: plugin installation disabled", "component", "metadata",
"installation_id", c.PluginInstallationID, "capability_id", c.CapabilityID)
continue
}
// A builtin installation's capabilities resolve to in-process providers
// from the builtin registry; everything else is a gRPC plugin provider.
if installationIsBuiltin(ctx, checker, pool, c.PluginInstallationID) {
provider, ok := builtinProvider(c.CapabilityID)
if !ok {
slog.WarnContext(ctx, "skipping builtin metadata provider: no registered constructor", "component", "metadata",
"installation_id", c.PluginInstallationID, "capability_id", c.CapabilityID)
continue
}
providers = append(providers, provider)
continue
}
provider, err := NewPluginProviderFromCapability(c.PluginInstallationID, c.CapabilityID, c.DisplayName, resolver)
if err != nil {
slog.WarnContext(ctx, "skipping metadata provider during chain resolution", "component", "metadata",
@@ -619,6 +660,42 @@ func buildProviders(
return providers
}
// installationKindChecker is an optional extension of InstallationEnabledChecker
// that serves the installation kind from the plugins service's in-memory cache,
// letting chain resolution identify builtin rows without a per-capability DB
// query on the hot path. *plugins.Service implements it; when the checker does
// not (nil or a test fake), installationIsBuiltin falls back to a pool query.
type installationKindChecker interface {
InstallationKind(ctx context.Context, installationID int) (string, error)
}
// installationIsBuiltin reports whether a plugin installation row is the
// reserved builtin-host installation (kind='builtin'). It prefers the cached
// checker (no DB read) and falls back to a direct query. Errors (including a
// pre-migration schema without the kind column) fail closed to "plugin" so
// resolution behavior is unchanged.
func installationIsBuiltin(ctx context.Context, checker InstallationEnabledChecker, pool *pgxpool.Pool, installationID int) bool {
if kc, ok := checker.(installationKindChecker); ok {
kind, err := kc.InstallationKind(ctx, installationID)
if err != nil {
return false
}
return kind == "builtin"
}
if pool == nil {
return false
}
var kind string
err := pool.QueryRow(ctx,
"SELECT kind FROM plugin_installations WHERE id = $1",
installationID,
).Scan(&kind)
if err != nil {
return false
}
return kind == "builtin"
}
// lookupCapabilityDisplayName retrieves the display name from plugin capability metadata.
func lookupCapabilityDisplayName(ctx context.Context, pool *pgxpool.Pool, installationID int, capabilityID string) string {
var displayName string
internal/metadata/chain_builtin_db_test.go
+314
View File
@@ -0,0 +1,314 @@
package metadata
import (
"context"
"fmt"
"os"
"reflect"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func chainBuiltinTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func insertTestInstallation(t *testing.T, pool *pgxpool.Pool, kind string, enabled bool) int {
t.Helper()
var id int
pluginID := fmt.Sprintf("test.chain-%s-%d", kind, time.Now().UnixNano())
err := pool.QueryRow(context.Background(),
`INSERT INTO plugin_installations (plugin_id, version, install_path, enabled, update_policy, kind)
VALUES ($1, '0', '/nonexistent/test-chain-builtin', $2, 'manual', $3)
RETURNING id`, pluginID, enabled, kind).Scan(&id)
if err != nil {
t.Fatalf("seed %s installation: %v", kind, err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM plugin_installations WHERE id = $1`, id)
})
return id
}
func insertTestCapability(t *testing.T, pool *pgxpool.Pool, installationID int, capabilityID, metadataJSON string) {
t.Helper()
_, err := pool.Exec(context.Background(),
`INSERT INTO plugin_capabilities (plugin_installation_id, capability_type, capability_id, metadata)
VALUES ($1, 'metadata_provider.v1', $2, $3::jsonb)`,
installationID, capabilityID, metadataJSON)
if err != nil {
t.Fatalf("seed capability %s: %v", capabilityID, err)
}
}
func insertTestFolder(t *testing.T, pool *pgxpool.Pool, folderType string) int {
t.Helper()
var id int
err := pool.QueryRow(context.Background(),
`INSERT INTO media_folders (type, name, enabled) VALUES ($1, $2, true) RETURNING id`,
folderType, fmt.Sprintf("chain-builtin-test-%d", time.Now().UnixNano())).Scan(&id)
if err != nil {
t.Fatalf("seed folder: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM media_folders WHERE id = $1`, id)
})
return id
}
// The migration must seed the reserved builtin installation and the NFO
// capability exactly as specified.
func TestMigrationSeedsReservedBuiltinInstallation(t *testing.T) {
pool := chainBuiltinTestPool(t)
ctx := context.Background()
var (
id int
kind string
enabled bool
installPath string
updatePolicy string
)
err := pool.QueryRow(ctx,
`SELECT id, kind, enabled, install_path, update_policy
FROM plugin_installations WHERE plugin_id = 'silo.builtin'`).
Scan(&id, &kind, &enabled, &installPath, &updatePolicy)
if err != nil {
t.Fatalf("reserved silo.builtin installation missing: %v", err)
}
if kind != "builtin" || !enabled || installPath != "/nonexistent/silo-builtin" || updatePolicy != "manual" {
t.Errorf("reserved row = kind=%q enabled=%v install_path=%q update_policy=%q", kind, enabled, installPath, updatePolicy)
}
md := lookupCapabilityMetadata(ctx, pool, id, "nfo")
if md == nil {
t.Fatal("silo.builtin nfo capability missing")
}
if extractDefaultEnabled(md) {
t.Error("nfo capability default_enabled must be false")
}
if got := extractDefaultPriority(md, "movie"); got != 1 {
t.Errorf("nfo movie default_priority = %d, want 1", got)
}
if got := extractDefaultPriority(md, "series"); got != 1 {
t.Errorf("nfo series default_priority = %d, want 1", got)
}
// Phase D declares season/episode support so the builtin chain sync
// appends the NFO provider to those chains.
if got := extractDefaultPriority(md, "season"); got != 1 {
t.Errorf("nfo season default_priority = %d, want 1 (Phase D migration)", got)
}
if got := extractDefaultPriority(md, "episode"); got != 1 {
t.Errorf("nfo episode default_priority = %d, want 1 (Phase D migration)", got)
}
}
// An enabled chain entry pointing at a builtin capability must resolve to the
// in-process provider through the real enabled-check (generic installation
// reads must NOT filter builtins).
func TestResolveChainWithChecker_BuiltinEntryResolvesInProcessProvider(t *testing.T) {
pool := chainBuiltinTestPool(t)
ctx := context.Background()
capID := fmt.Sprintf("test-builtin-%d", time.Now().UnixNano())
RegisterBuiltinProvider(capID, func() Provider { return &builtinStubProvider{slug: capID} })
installationID := insertTestInstallation(t, pool, "builtin", true)
insertTestCapability(t, pool, installationID, capID,
`{"display_name":"Test Builtin","default_priority":{"movie":1},"default_enabled":false}`)
folderID := insertTestFolder(t, pool, "movies")
chainRepo := NewChainRepository(pool)
if err := chainRepo.SetChain(ctx, folderID, []ChainEntry{
{PluginInstallationID: installationID, CapabilityID: capID, ContentLevel: "movie", Priority: 0, Enabled: true},
}); err != nil {
t.Fatalf("set chain: %v", err)
}
providers, err := ResolveChainWithChecker(ctx, folderID, "movie", chainRepo, nil, nil)
if err != nil {
t.Fatalf("resolve chain: %v", err)
}
if len(providers) != 1 {
t.Fatalf("providers = %d, want 1", len(providers))
}
if _, ok := providers[0].(*builtinStubProvider); !ok {
t.Fatalf("provider type = %T, want in-process *builtinStubProvider", providers[0])
}
// A disabled builtin installation must be skipped like any plugin.
if _, err := pool.Exec(ctx, `UPDATE plugin_installations SET enabled = false WHERE id = $1`, installationID); err != nil {
t.Fatalf("disable installation: %v", err)
}
providers, err = ResolveChainWithChecker(ctx, folderID, "movie", chainRepo, nil, nil)
if err != nil {
t.Fatalf("resolve chain: %v", err)
}
// The chain entry is still enabled, so the fallback does not run; the
// disabled installation is filtered by buildProviders.
if len(providers) != 0 {
t.Fatalf("providers after disabling installation = %d, want 0", len(providers))
}
}
// The chain-less fallback must skip default_enabled=false capabilities: a
// chain-less library, and a library whose only chain row is the disabled
// seeded NFO entry, must not activate the provider.
func TestChainlessFallback_SkipsDefaultDisabledCapability(t *testing.T) {
pool := chainBuiltinTestPool(t)
ctx := context.Background()
capID := fmt.Sprintf("test-fallback-%d", time.Now().UnixNano())
RegisterBuiltinProvider(capID, func() Provider { return &builtinStubProvider{slug: capID} })
installationID := insertTestInstallation(t, pool, "builtin", true)
insertTestCapability(t, pool, installationID, capID,
`{"display_name":"Test Fallback","default_priority":{"movie":1},"default_enabled":false}`)
chainRepo := NewChainRepository(pool)
assertNotResolved := func(folderID int, scenario string) {
t.Helper()
providers, err := ResolveChainWithChecker(ctx, folderID, "movie", chainRepo, nil, nil)
if err != nil {
t.Fatalf("%s: resolve chain: %v", scenario, err)
}
for _, p := range providers {
if p.Slug() == capID {
t.Errorf("%s: default_enabled=false capability leaked into the fallback", scenario)
}
}
}
// Chain-less library.
assertNotResolved(insertTestFolder(t, pool, "movies"), "chain-less library")
// Library whose only row is the disabled seeded entry (all-disabled chain).
folderID := insertTestFolder(t, pool, "movies")
if err := chainRepo.SetChain(ctx, folderID, []ChainEntry{
{PluginInstallationID: installationID, CapabilityID: capID, ContentLevel: "movie", Priority: 0, Enabled: false},
}); err != nil {
t.Fatalf("set chain: %v", err)
}
assertNotResolved(folderID, "all-disabled chain")
}
// Startup sync: a library whose chain rows are only legacy content_level=”
// must be materialized per level (same order/enabled, ” rows left in place),
// then the builtin appended disabled at MAX(priority)+1. The resolved
// (enabled-filtered) chain must be identical before and after.
func TestSyncBuiltinProviderChains_LegacyMaterializationFixture(t *testing.T) {
pool := chainBuiltinTestPool(t)
ctx := context.Background()
pluginInstallID := insertTestInstallation(t, pool, "plugin", true)
capID := fmt.Sprintf("test-legacy-%d", time.Now().UnixNano())
insertTestCapability(t, pool, pluginInstallID, capID,
`{"display_name":"Legacy Plugin","metadata":{"default_priority":{"movie":2}}}`)
builtinInstallID := insertTestInstallation(t, pool, "builtin", true)
builtinCapID := fmt.Sprintf("test-legacy-builtin-%d", time.Now().UnixNano())
insertTestCapability(t, pool, builtinInstallID, builtinCapID,
`{"display_name":"Legacy Builtin","default_priority":{"movie":1,"series":1},"default_enabled":false}`)
folderID := insertTestFolder(t, pool, "movies")
chainRepo := NewChainRepository(pool)
if err := chainRepo.SetChain(ctx, folderID, []ChainEntry{
{PluginInstallationID: pluginInstallID, CapabilityID: capID, ContentLevel: "", Priority: 0, Enabled: true},
}); err != nil {
t.Fatalf("set legacy chain: %v", err)
}
resolvedBefore, err := chainRepo.GetChain(ctx, folderID, "movie")
if err != nil {
t.Fatalf("get chain before sync: %v", err)
}
if err := SyncBuiltinProviderChains(ctx, chainRepo); err != nil {
t.Fatalf("sync builtin provider chains: %v", err)
}
after, err := chainRepo.GetChain(ctx, folderID, "movie")
if err != nil {
t.Fatalf("get chain after sync: %v", err)
}
// Enabled-filtered resolution must be identical (modulo the now-real level).
var enabledAfter []ChainEntry
for _, e := range after {
if e.Enabled {
e.ContentLevel = ""
enabledAfter = append(enabledAfter, e)
}
}
var enabledBefore []ChainEntry
for _, e := range resolvedBefore {
if e.Enabled {
e.ContentLevel = ""
enabledBefore = append(enabledBefore, e)
}
}
if !reflect.DeepEqual(enabledBefore, enabledAfter) {
t.Errorf("resolved chain changed across sync:\nbefore=%+v\nafter=%+v", enabledBefore, enabledAfter)
}
// The editor now sees real per-level rows: the materialized plugin row plus
// the appended disabled builtin.
foundPlugin, foundBuiltin := false, false
maxPluginPriority := -1
for _, e := range after {
switch {
case e.PluginInstallationID == pluginInstallID && e.CapabilityID == capID:
foundPlugin = true
if !e.Enabled {
t.Error("materialized plugin row lost enabled state")
}
if e.Priority > maxPluginPriority {
maxPluginPriority = e.Priority
}
case e.PluginInstallationID == builtinInstallID && e.CapabilityID == builtinCapID:
foundBuiltin = true
if e.Enabled {
t.Error("builtin must be appended disabled")
}
}
}
if !foundPlugin {
t.Error("legacy '' plugin row was not materialized at the movie level")
}
if !foundBuiltin {
t.Error("builtin capability was not appended to the materialized movie level")
}
// The legacy '' rows stay in place for old binaries.
legacy, err := chainRepo.GetChain(ctx, folderID, "")
if err != nil {
t.Fatalf("get legacy chain: %v", err)
}
if len(legacy) != 1 {
t.Errorf("legacy '' rows = %d, want 1 (left in place)", len(legacy))
}
// Idempotency: running the sync again changes nothing.
if err := SyncBuiltinProviderChains(ctx, chainRepo); err != nil {
t.Fatalf("second sync: %v", err)
}
again, err := chainRepo.GetChain(ctx, folderID, "movie")
if err != nil {
t.Fatalf("get chain after second sync: %v", err)
}
if !reflect.DeepEqual(after, again) {
t.Errorf("sync is not idempotent:\nfirst=%+v\nsecond=%+v", after, again)
}
}
internal/metadata/export_nfo_test.go
+147
View File
@@ -0,0 +1,147 @@
package metadata
import (
"context"
"fmt"
"sort"
"sync"
"github.com/Silo-Server/silo-server/internal/models"
)
// NFOSeriesHarness bridges the in-package fakes to the external NFO
// integration tests (package metadata_test). Those tests must live outside
// the package so they can import internal/metadata/nfo (which itself imports
// this package) without an import cycle.
type NFOSeriesHarness struct {
h *testHarness
seasons *fakeSeasonRepo
episodes *fakeEpisodeRepo
jobs *capturingImageJobEnqueuer
}
type capturingImageJobEnqueuer struct {
mu sync.Mutex
inputs []EnqueueImageCacheJobInput
}
func (c *capturingImageJobEnqueuer) Enqueue(_ context.Context, in EnqueueImageCacheJobInput) error {
c.mu.Lock()
defer c.mu.Unlock()
c.inputs = append(c.inputs, in)
return nil
}
func (c *capturingImageJobEnqueuer) EnqueueBatch(_ context.Context, inputs []EnqueueImageCacheJobInput) (int, error) {
c.mu.Lock()
defer c.mu.Unlock()
c.inputs = append(c.inputs, inputs...)
return len(inputs), nil
}
func NewNFOSeriesHarness() *NFOSeriesHarness {
h := newTestHarness()
x := &NFOSeriesHarness{
h: h,
seasons: newFakeSeasonRepo(),
episodes: newFakeEpisodeRepo(),
jobs: &capturingImageJobEnqueuer{},
}
h.service.seasonRepo = x.seasons
h.service.episodeRepo = x.episodes
h.service.imageCacheJobs = x.jobs
h.service.autoCacheImages.Store(true)
return x
}
func (x *NFOSeriesHarness) Service() *MetadataService { return x.h.service }
// SeedSeriesSkeleton seeds a pending series item, mirroring the scanner's
// skeleton for an unmatched series root.
func (x *NFOSeriesHarness) SeedSeriesSkeleton(contentID, title string) error {
return x.seedSkeleton(contentID, title, "series")
}
// SeedMovieSkeleton seeds a pending movie item, mirroring the scanner's
// skeleton for an unmatched movie group (e.g. the movie lane of a mixed
// library).
func (x *NFOSeriesHarness) SeedMovieSkeleton(contentID, title string) error {
return x.seedSkeleton(contentID, title, "movie")
}
func (x *NFOSeriesHarness) seedSkeleton(contentID, title, contentType string) error {
return x.h.itemRepo.Upsert(context.Background(), &models.MediaItem{
ContentID: contentID,
Type: contentType,
Title: title,
Status: "pending_match",
Studios: []string{},
Networks: []string{},
Countries: []string{},
Genres: []string{},
})
}
func (x *NFOSeriesHarness) Item(contentID string) (*models.MediaItem, error) {
return x.h.itemRepo.GetByID(context.Background(), contentID)
}
// SeedSeriesFile registers a persisted media file linked to a content id, so
// refresh flows (which reconstruct local context from the file repo instead
// of scan hints) can find the series' episode files. observedRootPath is the
// series root eligible for directory-level sidecars.
func (x *NFOSeriesHarness) SeedSeriesFile(fileID, folderID int, filePath, observedRootPath, contentID string) {
x.h.fileRepo.setGroupFiles(folderID, 1, fmt.Sprintf("group-%s-%d", contentID, fileID), &models.MediaFile{
ID: fileID,
MediaFolderID: folderID,
FilePath: filePath,
ObservedRootPath: observedRootPath,
})
x.h.fileRepo.mu.Lock()
x.h.fileRepo.contentIDs[fileID] = contentID
x.h.fileRepo.mu.Unlock()
}
// Seasons returns the persisted seasons for a series sorted by season number.
func (x *NFOSeriesHarness) Seasons(seriesID string) []*models.Season {
x.seasons.mu.Lock()
defer x.seasons.mu.Unlock()
var out []*models.Season
for _, s := range x.seasons.seasons {
if s.SeriesID == seriesID {
cp := *s
out = append(out, &cp)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].SeasonNumber < out[j].SeasonNumber })
return out
}
// Episodes returns the persisted episodes for a series sorted by (season, episode).
func (x *NFOSeriesHarness) Episodes(seriesID string) []*models.Episode {
x.episodes.mu.Lock()
defer x.episodes.mu.Unlock()
var out []*models.Episode
for _, e := range x.episodes.episodes {
if e.SeriesID == seriesID {
cp := *e
out = append(out, &cp)
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].SeasonNumber != out[j].SeasonNumber {
return out[i].SeasonNumber < out[j].SeasonNumber
}
return out[i].EpisodeNumber < out[j].EpisodeNumber
})
return out
}
// EnqueuedImageJobs returns every image cache job the pipeline enqueued.
func (x *NFOSeriesHarness) EnqueuedImageJobs() []EnqueueImageCacheJobInput {
x.jobs.mu.Lock()
defer x.jobs.mu.Unlock()
out := make([]EnqueueImageCacheJobInput, len(x.jobs.inputs))
copy(out, x.jobs.inputs)
return out
}
internal/metadata/image_cache_job_repo.go
+51 -3
View File
@@ -81,7 +81,19 @@ func isStableProviderImageFailure(errText string) bool {
return strings.Contains(errText, "unexpected status 403") ||
strings.Contains(errText, "unexpected status 404") ||
strings.Contains(errText, "unexpected status 410") ||
strings.Contains(errText, "unexpected status 418")
strings.Contains(errText, "unexpected status 418") ||
// Local sidecar failures that will not heal on the normal backoff:
// deleted files (ENOENT), unreadable files (EPERM), paths outside the
// owning library's roots, and files that are structurally unusable
// (non-regular or over the size cap) — these do not self-heal without an
// on-disk change, so a hot retry loop is pointless; recovery is
// refresh-driven. Other local read errors keep the standard retry
// schedule. Texts match the processor's local branch.
strings.Contains(errText, "local image missing") ||
strings.Contains(errText, "local image forbidden") ||
strings.Contains(errText, "local image path outside library roots") ||
strings.Contains(errText, "local image is not a regular file") ||
strings.Contains(errText, "local image exceeds")
}
func (r *ImageCacheJobRepository) Enqueue(ctx context.Context, in EnqueueImageCacheJobInput) error {
@@ -127,7 +139,7 @@ func (r *ImageCacheJobRepository) enqueueBatch(ctx context.Context, inputs []Enq
func normalizeImageCacheJobInput(in EnqueueImageCacheJobInput) (EnqueueImageCacheJobInput, bool) {
in.SourcePath = strings.TrimSpace(in.SourcePath)
in.TargetLanguage = strings.TrimSpace(in.TargetLanguage)
if in.SourcePath == "" || !strings.Contains(in.SourcePath, "://") || isNonProviderImageScheme(strings.ToLower(in.SourcePath)) {
if !isCacheableImageSourcePath(in.SourcePath) {
return EnqueueImageCacheJobInput{}, false
}
if in.ContentType == "" {
@@ -468,6 +480,31 @@ func (r *ImageCacheJobRepository) CurrentTargetSourcePath(ctx context.Context, j
return current, nil
}
// CurrentTargetCachedPath reports the cached image path currently stored on
// the job's target row. The processor's local-artwork branch uses it to skip
// unchanged art and to delete the stale hashed local/ prefix after a
// successful re-cache. Returns ("", nil) when the row no longer exists or the
// target type is unknown.
func (r *ImageCacheJobRepository) CurrentTargetCachedPath(ctx context.Context, job *models.MetadataImageCacheJob) (string, error) {
if r == nil || r.pool == nil || job == nil {
return "", nil
}
query, args, ok := currentTargetSourceQuery(job)
if !ok {
return "", nil
}
query = strings.Replace(query, "_source_path", "_path", 1)
var current string
err := r.pool.QueryRow(ctx, query, args...).Scan(&current)
if errors.Is(err, pgx.ErrNoRows) {
return "", nil
}
if err != nil {
return "", fmt.Errorf("reading current target cached path: %w", err)
}
return current, nil
}
func currentTargetSourceQuery(job *models.MetadataImageCacheJob) (string, []any, bool) {
switch job.TargetType {
case ImageCacheTargetItem:
@@ -542,7 +579,11 @@ func (r *ImageCacheJobRepository) EnqueueExistingProviderArtwork(ctx context.Con
return 0, nil
}
// Each branch is restricted to provider-origin sources (LIKE '%://%' minus
// cached/system schemes) AND to targets whose stored *_path is not already a
// cached/system schemes). Local file:// sidecar sources are deliberately
// excluded from this sweep: their recovery is refresh-driven (a metadata
// refresh re-discovers the sidecar and enqueues a fresh job), so a stable
// local failure cannot be resurrected here every cycle.
// Each branch is also restricted to targets whose stored *_path is not already a
// cached relative path. The destination check makes the cached row itself the
// durable dedup marker, so pruning succeeded job rows does not cause the whole
// catalog to be re-downloaded once the rows age out.
@@ -798,7 +839,14 @@ func (r *ImageCacheJobRepository) EnqueueExistingProviderArtwork(ctx context.Con
return r.enqueueBatch(ctx, inputs, true)
}
// imageCacheLocalProviderID is the synthetic provider slug for local sidecar
// artwork; cached keys live under "local/..." like audiobook/ebook covers.
const imageCacheLocalProviderID = "local"
func imageCacheProviderIDFromSource(sourcePath, fallback string) string {
if isLocalImageSourcePath(sourcePath) {
return imageCacheLocalProviderID
}
if provider := providerIDFromPluginURL(sourcePath); provider != "" {
return provider
}
internal/metadata/image_cache_job_repo_test.go
-1
View File
@@ -57,7 +57,6 @@ func TestNormalizeImageCacheJobInputSkipsNonProviderArtwork(t *testing.T) {
"",
"tmdb/series/1396/poster/original.webp",
"s3://media/tmdb/series/1396/poster/original.webp",
"file:///media/poster.jpg",
"local://poster.jpg",
"generated://collections/1/poster.jpg",
} {
internal/metadata/image_cache_processor.go
+349 -59
View File
@@ -2,9 +2,17 @@ package metadata
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"io/fs"
"log/slog"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
@@ -12,6 +20,10 @@ import (
"github.com/Silo-Server/silo-server/internal/models"
)
// maxLocalImageSourceBytes caps local sidecar artwork reads at 8 MiB,
// matching the NFO provider's discovery guard.
const maxLocalImageSourceBytes = 8 << 20
// imageCacheDiscoveryInterval throttles the full-catalog backfill sweep so an
// idle installation does not re-scan every entity table on every task tick.
// Draining of already-queued jobs is unaffected and stays responsive.
@@ -27,6 +39,27 @@ type ImageCacheJobClaimer interface {
DeleteSucceededBefore(ctx context.Context, before time.Time, limit int) (int, error)
}
// imageCacheTargetCachedPathReader is optionally implemented by the job store
// (ImageCacheJobRepository does) to expose the target's currently stored
// cached path for the local-artwork unchanged-skip and stale-prefix cleanup.
type imageCacheTargetCachedPathReader interface {
CurrentTargetCachedPath(ctx context.Context, job *models.MetadataImageCacheJob) (string, error)
}
// LibraryRootResolver reports the media folder root paths a piece of content
// belongs to (media_folders.Paths via media_item_libraries). The processor
// confines local file:// sources to these roots before reading them.
type LibraryRootResolver interface {
LibraryRootsForContent(ctx context.Context, contentID string) ([]string, error)
}
// ImagePrefixDeleter deletes cached image objects under a key prefix; used to
// sweep the previous hashed local/ prefix after a successful re-cache.
type ImagePrefixDeleter interface {
DeletePrefix(ctx context.Context, bucket, prefix string) (int, error)
Bucket() string
}
type SeasonArtworkUpdater interface {
UpdateArtworkIfSourceMatches(ctx context.Context, contentID, sourcePath, cachedPath, thumbhash string) (bool, error)
}
@@ -69,6 +102,12 @@ type ImageCacheProcessor struct {
targets ImageCacheProcessorTargets
logger *slog.Logger
// Local file:// artwork support. libraryRoots confines reads to the
// owning library's roots; prefixDeleter sweeps stale hashed prefixes.
// The processor host must mount the libraries, like the metadata worker.
libraryRoots LibraryRootResolver
prefixDeleter ImagePrefixDeleter
enabled atomic.Bool
discoveryInterval time.Duration
@@ -76,6 +115,26 @@ type ImageCacheProcessor struct {
lastDiscovery time.Time
}
// SetLibraryRootResolver wires the folder repository used to confine local
// file:// sources to their owning library's roots. Without it, local jobs
// fail (and retry on the normal backoff) rather than reading arbitrary paths.
func (p *ImageCacheProcessor) SetLibraryRootResolver(resolver LibraryRootResolver) {
if p == nil {
return
}
p.libraryRoots = resolver
}
// SetImagePrefixDeleter wires the object store used to delete the previous
// hashed local/ prefix after a successful local re-cache. Optional: without
// it, stale prefixes are left behind (item deletion still sweeps them).
func (p *ImageCacheProcessor) SetImagePrefixDeleter(deleter ImagePrefixDeleter) {
if p == nil {
return
}
p.prefixDeleter = deleter
}
// SetEnabled toggles background caching. When disabled the processor performs
// no discovery, claiming, or uploading, honoring metadata.cache_images so that
// merely configuring object storage does not download the whole catalog.
@@ -370,6 +429,10 @@ func (p *ImageCacheProcessor) processOne(ctx context.Context, job *models.Metada
return imageCacheProcessResult{outcome: "skipped"}
}
if isLocalImageSourcePath(job.SourcePath) {
return p.processLocalOne(ctx, job, imageType)
}
downloadURL := job.SourcePath
if isProviderImagePath(downloadURL) {
if p.resolver == nil {
@@ -412,73 +475,300 @@ func (p *ImageCacheProcessor) processOne(ctx context.Context, job *models.Metada
processResult.outcome = "failed"
return processResult
}
var updated bool
switch job.TargetType {
case ImageCacheTargetItem:
if p.targets.Items == nil {
p.markFailed(ctx, job, "missing item updater")
processResult.outcome = "failed"
return processResult
}
updated, err = p.targets.Items.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.ImageType, job.SourcePath, cachedPath, result.Thumbhash)
case ImageCacheTargetItemLocalization:
if p.targets.ItemLocalizations == nil {
p.markFailed(ctx, job, "missing item localization updater")
processResult.outcome = "failed"
return processResult
}
updated, err = p.targets.ItemLocalizations.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.TargetLanguage, job.ImageType, job.SourcePath, cachedPath, result.Thumbhash)
case ImageCacheTargetSeason:
if p.targets.Seasons == nil {
p.markFailed(ctx, job, "missing season updater")
processResult.outcome = "failed"
return processResult
}
updated, err = p.targets.Seasons.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.SourcePath, cachedPath, result.Thumbhash)
case ImageCacheTargetSeasonLocalization:
if p.targets.SeasonLocalizations == nil {
p.markFailed(ctx, job, "missing season localization updater")
processResult.outcome = "failed"
return processResult
}
updated, err = p.targets.SeasonLocalizations.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.TargetLanguage, job.SourcePath, cachedPath, result.Thumbhash)
case ImageCacheTargetEpisode:
if p.targets.Episodes == nil {
p.markFailed(ctx, job, "missing episode updater")
processResult.outcome = "failed"
return processResult
}
updated, err = p.targets.Episodes.UpdateStillIfSourceMatches(ctx, job.TargetContentID, job.SourcePath, cachedPath, result.Thumbhash)
case ImageCacheTargetPerson:
if p.targets.People == nil {
p.markFailed(ctx, job, "missing person updater")
processResult.outcome = "failed"
return processResult
}
personID, parseErr := strconv.ParseInt(job.TargetContentID, 10, 64)
if parseErr != nil {
err = fmt.Errorf("invalid person image cache target %q: %w", job.TargetContentID, parseErr)
break
}
updated, err = p.targets.People.UpdatePhotoIfSourceMatches(ctx, personID, job.SourcePath, cachedPath, result.Thumbhash)
default:
err = fmt.Errorf("unknown image cache target type %q", job.TargetType)
}
p.finishJobWithTargetUpdate(ctx, job, cachedPath, result.Thumbhash, &processResult)
return processResult
}
// finishJobWithTargetUpdate persists the cached path/thumbhash to the job's
// target row and marks the job terminal, setting processResult.outcome.
func (p *ImageCacheProcessor) finishJobWithTargetUpdate(ctx context.Context, job *models.MetadataImageCacheJob, cachedPath, thumbhash string, processResult *imageCacheProcessResult) {
updated, err := p.updateTargetArtwork(ctx, job, cachedPath, thumbhash)
if err != nil {
p.markFailed(ctx, job, err.Error())
processResult.outcome = "failed"
return processResult
}
if !updated {
p.markSucceeded(ctx, job)
processResult.outcome = "skipped"
return processResult
return
}
p.markSucceeded(ctx, job)
processResult.outcome = "succeeded"
if updated {
processResult.outcome = "succeeded"
} else {
processResult.outcome = "skipped"
}
}
func (p *ImageCacheProcessor) updateTargetArtwork(ctx context.Context, job *models.MetadataImageCacheJob, cachedPath, thumbhash string) (bool, error) {
switch job.TargetType {
case ImageCacheTargetItem:
if p.targets.Items == nil {
return false, errors.New("missing item updater")
}
return p.targets.Items.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.ImageType, job.SourcePath, cachedPath, thumbhash)
case ImageCacheTargetItemLocalization:
if p.targets.ItemLocalizations == nil {
return false, errors.New("missing item localization updater")
}
return p.targets.ItemLocalizations.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.TargetLanguage, job.ImageType, job.SourcePath, cachedPath, thumbhash)
case ImageCacheTargetSeason:
if p.targets.Seasons == nil {
return false, errors.New("missing season updater")
}
return p.targets.Seasons.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.SourcePath, cachedPath, thumbhash)
case ImageCacheTargetSeasonLocalization:
if p.targets.SeasonLocalizations == nil {
return false, errors.New("missing season localization updater")
}
return p.targets.SeasonLocalizations.UpdateArtworkIfSourceMatches(ctx, job.TargetContentID, job.TargetLanguage, job.SourcePath, cachedPath, thumbhash)
case ImageCacheTargetEpisode:
if p.targets.Episodes == nil {
return false, errors.New("missing episode updater")
}
return p.targets.Episodes.UpdateStillIfSourceMatches(ctx, job.TargetContentID, job.SourcePath, cachedPath, thumbhash)
case ImageCacheTargetPerson:
if p.targets.People == nil {
return false, errors.New("missing person updater")
}
personID, parseErr := strconv.ParseInt(job.TargetContentID, 10, 64)
if parseErr != nil {
return false, fmt.Errorf("invalid person image cache target %q: %w", job.TargetContentID, parseErr)
}
return p.targets.People.UpdatePhotoIfSourceMatches(ctx, personID, job.SourcePath, cachedPath, thumbhash)
default:
return false, fmt.Errorf("unknown image cache target type %q", job.TargetType)
}
}
// processLocalOne caches a local sidecar image (file:// source). The file is
// confined to the owning library's roots by a lexical check on the logical path
// followed by a symlink-resolving re-check (both path and roots are resolved, so
// scanner-recorded logical paths under symlinked roots stay valid while an
// intermediate directory symlink escaping a root is rejected), then read through
// an opened handle with fstat re-checks and pushed to S3 under a content-hashed
// local/ key.
func (p *ImageCacheProcessor) processLocalOne(ctx context.Context, job *models.MetadataImageCacheJob, imageType ImageType) imageCacheProcessResult {
byteCacher, ok := p.cacher.(ImageByteCacher)
if !ok {
p.markFailed(ctx, job, "image cacher does not support local artwork")
return imageCacheProcessResult{outcome: "failed"}
}
if p.libraryRoots == nil {
p.markFailed(ctx, job, "missing library root resolver for local artwork")
return imageCacheProcessResult{outcome: "failed"}
}
localPath := filepath.Clean(strings.TrimSpace(job.SourcePath)[len("file://"):])
rootsContentID := firstNonEmpty(job.SeriesID, job.TargetContentID)
roots, err := p.libraryRoots.LibraryRootsForContent(ctx, rootsContentID)
if err != nil {
p.markFailed(ctx, job, fmt.Sprintf("resolving library roots: %v", err))
return imageCacheProcessResult{outcome: "failed"}
}
if !localImagePathWithinRoots(localPath, roots) {
p.markFailed(ctx, job, "local image path outside library roots: "+localPath)
return imageCacheProcessResult{outcome: "failed"}
}
// The lexical check above cannot see through symlinks. Resolve the path and
// roots and re-confine so an intermediate directory symlink planted inside a
// root (e.g. a link pointing out of the library) cannot pull an out-of-root
// file into the cache. EvalSymlinks resolves both sides, so a legitimately
// symlinked root still matches. This is a confinement GATE only: the read
// below still uses the logical path so readLocalImageFile's Lstat keeps
// rejecting a symlinked leaf. A not-yet-existent path (ErrNotExist) falls
// through to the reader, which classifies it as the stable "missing" failure.
if _, err := localImagePathResolvedWithinRoots(localPath, roots); err != nil && !errors.Is(err, fs.ErrNotExist) {
p.markFailed(ctx, job, "local image path outside library roots: "+localPath)
return imageCacheProcessResult{outcome: "failed"}
}
data, err := readLocalImageFile(localPath)
if err != nil {
p.markFailed(ctx, job, err.Error())
return imageCacheProcessResult{outcome: "failed"}
}
// The target's current cached path drives the unchanged-skip (same hash →
// same key → nothing to sweep) and the stale-prefix cleanup below.
previousCachedPath := ""
if reader, ok := p.jobs.(imageCacheTargetCachedPathReader); ok {
previousCachedPath, err = reader.CurrentTargetCachedPath(ctx, job)
if err != nil {
p.logger.WarnContext(ctx, "metadata image cache: failed to read current cached path", "job_id", job.ID, "error", err)
previousCachedPath = ""
}
}
digest := sha256.Sum256(data)
result, err := byteCacher.CacheImageBytes(ctx, data, CacheImageRequest{
SourceURL: job.SourcePath,
ProviderID: imageCacheLocalProviderID,
ContentType: job.ContentType,
ContentID: job.ProviderContentID,
ImageType: imageType,
SeasonNumber: job.SeasonNumber,
EpisodeNumber: job.EpisodeNumber,
Language: job.TargetLanguage,
KeyDiscriminator: hex.EncodeToString(digest[:4]),
})
if err != nil {
p.markFailed(ctx, job, err.Error())
return imageCacheProcessResult{outcome: "failed"}
}
if result == nil {
p.markFailed(ctx, job, "image cache returned no result")
return imageCacheProcessResult{outcome: "failed"}
}
processResult := imageCacheProcessResult{
uploadedVariants: result.UploadedVariants,
existingVariants: result.ExistingVariants,
}
cachedPath := CachedImageOriginalPath(result)
if cachedPath == "" {
p.markFailed(ctx, job, "image cache returned empty stored path")
processResult.outcome = "failed"
return processResult
}
p.finishJobWithTargetUpdate(ctx, job, cachedPath, result.Thumbhash, &processResult)
if processResult.outcome == "succeeded" {
p.deleteStaleLocalPrefix(ctx, previousCachedPath, cachedPath)
}
return processResult
}
// deleteStaleLocalPrefix removes the previous hashed local/ image prefix
// after a re-cache stored the artwork under a different key.
func (p *ImageCacheProcessor) deleteStaleLocalPrefix(ctx context.Context, previousCachedPath, cachedPath string) {
previousCachedPath = strings.TrimSpace(previousCachedPath)
if p.prefixDeleter == nil ||
previousCachedPath == "" ||
previousCachedPath == cachedPath ||
!strings.HasPrefix(previousCachedPath, "local/") {
return
}
lastSlash := strings.LastIndex(previousCachedPath, "/")
if lastSlash <= 0 {
return
}
prefix := previousCachedPath[:lastSlash+1]
if strings.HasPrefix(cachedPath, prefix) {
return
}
if _, err := p.prefixDeleter.DeletePrefix(ctx, p.prefixDeleter.Bucket(), prefix); err != nil {
p.logger.WarnContext(ctx, "metadata image cache: failed to delete stale local image prefix", "prefix", prefix, "error", err)
}
}
// errLocalImageOutsideRoots is returned by localImagePathResolvedWithinRoots
// when a fully symlink-resolved path escapes every resolved library root.
var errLocalImageOutsideRoots = errors.New("local image path outside library roots")
// localImagePathResolvedWithinRoots resolves symlinks on both the path and the
// library roots and returns the real path when it stays within a real root.
// This closes the gap the lexical check leaves open — an intermediate directory
// symlink planted inside a root that points outside it. Resolving both sides
// keeps a legitimately symlinked root valid. A path that does not exist yet
// surfaces as fs.ErrNotExist for the caller to treat as a stable "missing"
// failure via the reader; any other resolution failure fails closed.
func localImagePathResolvedWithinRoots(path string, roots []string) (string, error) {
resolvedPath, err := filepath.EvalSymlinks(path)
if err != nil {
return "", err
}
for _, root := range roots {
resolvedRoot, err := filepath.EvalSymlinks(strings.TrimSpace(root))
if err != nil {
continue
}
if localImagePathWithinRoots(resolvedPath, []string{resolvedRoot}) {
return resolvedPath, nil
}
}
return "", errLocalImageOutsideRoots
}
// localImagePathWithinRoots confines path (already cleaned, logical) to the
// given library roots with a separator-aware lexical prefix check.
func localImagePathWithinRoots(path string, roots []string) bool {
if !filepath.IsAbs(path) {
return false
}
for _, root := range roots {
root = filepath.Clean(strings.TrimSpace(root))
if root == "" || root == "." || !filepath.IsAbs(root) {
continue
}
if path == root {
return true
}
prefix := root
if prefix != string(filepath.Separator) {
prefix += string(filepath.Separator)
}
if strings.HasPrefix(path, prefix) {
return true
}
}
return false
}
// readLocalImageFile reads a sidecar image with the same guards as discovery:
// Lstat rejects symlinked leaves and non-regular files, the opened handle is
// fstat-re-checked, and reads cap at maxLocalImageSourceBytes. ENOENT/EPERM
// map to the stable-failure texts matched by isStableProviderImageFailure.
func readLocalImageFile(path string) ([]byte, error) {
classify := func(err error) error {
switch {
case errors.Is(err, fs.ErrNotExist):
return fmt.Errorf("local image missing: %s", path)
case errors.Is(err, fs.ErrPermission):
return fmt.Errorf("local image forbidden: %s", path)
default:
return fmt.Errorf("local image read failed: %w", err)
}
}
info, err := os.Lstat(path)
if err != nil {
return nil, classify(err)
}
if !info.Mode().IsRegular() {
return nil, fmt.Errorf("local image is not a regular file: %s", path)
}
if info.Size() > maxLocalImageSourceBytes {
return nil, fmt.Errorf("local image exceeds %d byte limit: %s", maxLocalImageSourceBytes, path)
}
file, err := os.Open(path)
if err != nil {
return nil, classify(err)
}
defer func() { _ = file.Close() }()
stat, err := file.Stat()
if err != nil {
return nil, classify(err)
}
// Close the symlink-swap window: os.Open follows symlinks, so a leaf swapped
// to a symlink between the Lstat above and this Open would be followed to its
// target. Reject unless the opened handle is the exact file Lstat inspected,
// so an out-of-root target can never be pulled into the public cache.
if !os.SameFile(info, stat) {
return nil, fmt.Errorf("local image is not a regular file: %s", path)
}
if !stat.Mode().IsRegular() {
return nil, fmt.Errorf("local image is not a regular file: %s", path)
}
if stat.Size() > maxLocalImageSourceBytes {
return nil, fmt.Errorf("local image exceeds %d byte limit: %s", maxLocalImageSourceBytes, path)
}
data, err := io.ReadAll(io.LimitReader(file, maxLocalImageSourceBytes+1))
if err != nil {
return nil, classify(err)
}
if len(data) > maxLocalImageSourceBytes {
return nil, fmt.Errorf("local image exceeds %d byte limit: %s", maxLocalImageSourceBytes, path)
}
if len(data) == 0 {
return nil, fmt.Errorf("local image read failed: empty file %s", path)
}
return data, nil
}
func imageCacheJobImageType(value string) (ImageType, error) {
switch value {
case ImageCacheImagePoster:
internal/metadata/image_cache_processor_local_test.go
+322
View File
@@ -0,0 +1,322 @@
package metadata
import (
"context"
"fmt"
"os"
"path/filepath"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// fakeLocalImageCacheJobs is a fakeImageCacheJobs that also reports the
// target's currently stored cached path, like the real repository.
type fakeLocalImageCacheJobs struct {
fakeImageCacheJobs
currentCached string
}
func (f *fakeLocalImageCacheJobs) CurrentTargetCachedPath(context.Context, *models.MetadataImageCacheJob) (string, error) {
return f.currentCached, nil
}
// fakeByteImageCacher supports both remote and byte caching. CacheImageBytes
// builds the base path the way the real cacher does so key assertions hold.
type fakeByteImageCacher struct {
fakeImageCacher
byteReqs [][]byte
bytesReq []CacheImageRequest
thumbhash string
}
func (f *fakeByteImageCacher) CacheImageBytes(_ context.Context, data []byte, req CacheImageRequest) (*CacheImageResult, error) {
f.byteReqs = append(f.byteReqs, data)
f.bytesReq = append(f.bytesReq, req)
if f.err != nil {
return nil, f.err
}
return &CacheImageResult{
BasePath: fmt.Sprintf("%s/%s/%s/%s/%s", req.ProviderID, req.ContentType, req.ContentID, req.KeyDiscriminator, ImageTypeToString(req.ImageType)),
Thumbhash: f.thumbhash,
Ext: ".webp",
}, nil
}
type fakeLibraryRootResolver struct {
roots []string
err error
}
func (f *fakeLibraryRootResolver) LibraryRootsForContent(context.Context, string) ([]string, error) {
return f.roots, f.err
}
type fakePrefixDeleter struct {
bucket string
prefixes []string
}
func (f *fakePrefixDeleter) DeletePrefix(_ context.Context, _ string, prefix string) (int, error) {
f.prefixes = append(f.prefixes, prefix)
return 1, nil
}
func (f *fakePrefixDeleter) Bucket() string { return f.bucket }
func localArtworkJob(sourcePath string) *models.MetadataImageCacheJob {
return &models.MetadataImageCacheJob{
ID: 7,
TargetType: ImageCacheTargetItem,
TargetContentID: "movie-1",
SeriesID: "movie-1",
SourcePath: sourcePath,
ProviderID: "local",
ProviderContentID: "movie-1",
ContentType: "movies",
ImageType: ImageCacheImagePoster,
}
}
func newLocalProcessorForTest(
job *models.MetadataImageCacheJob,
roots []string,
currentCached string,
) (*ImageCacheProcessor, *fakeLocalImageCacheJobs, *fakeByteImageCacher, *fakeItemArtworkUpdater, *fakePrefixDeleter) {
jobs := &fakeLocalImageCacheJobs{
fakeImageCacheJobs: fakeImageCacheJobs{claimed: []*models.MetadataImageCacheJob{job}},
currentCached: currentCached,
}
cacher := &fakeByteImageCacher{thumbhash: "th-new"}
items := &fakeItemArtworkUpdater{updated: true}
deleter := &fakePrefixDeleter{bucket: "images"}
p := NewImageCacheProcessorWithTargets(jobs, cacher, nil, ImageCacheProcessorTargets{Items: items})
p.SetLibraryRootResolver(&fakeLibraryRootResolver{roots: roots})
p.SetImagePrefixDeleter(deleter)
return p, jobs, cacher, items, deleter
}
func writeLocalPoster(t *testing.T, dir string) string {
t.Helper()
path := filepath.Join(dir, "poster.jpg")
if err := os.WriteFile(path, []byte("poster-bytes"), 0o644); err != nil {
t.Fatalf("writing poster: %v", err)
}
return path
}
func TestProcessLocalImageCachesWithinLibraryRoots(t *testing.T) {
root := t.TempDir()
posterPath := writeLocalPoster(t, root)
job := localArtworkJob("file://" + posterPath)
p, jobs, cacher, items, deleter := newLocalProcessorForTest(job, []string{root}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Succeeded != 1 {
t.Fatalf("stats = %+v, want 1 succeeded (failed text %q)", stats, jobs.failedText)
}
if len(cacher.bytesReq) != 1 {
t.Fatalf("CacheImageBytes calls = %d, want 1", len(cacher.bytesReq))
}
req := cacher.bytesReq[0]
if req.ProviderID != "local" || req.ContentType != "movies" || req.ContentID != "movie-1" {
t.Fatalf("cache request = %+v", req)
}
if len(req.KeyDiscriminator) != 8 {
t.Fatalf("KeyDiscriminator = %q, want 8-char content hash", req.KeyDiscriminator)
}
wantCached := "local/movies/movie-1/" + req.KeyDiscriminator + "/poster/original.webp"
if items.cachedPath != wantCached {
t.Fatalf("cachedPath = %q, want %q", items.cachedPath, wantCached)
}
if items.thumbhash != "th-new" {
t.Fatalf("thumbhash = %q", items.thumbhash)
}
if len(deleter.prefixes) != 0 {
t.Fatalf("no stale prefix to delete on first cache, got %v", deleter.prefixes)
}
}
func TestProcessLocalImageRejectsPathOutsideRoots(t *testing.T) {
root := t.TempDir()
other := t.TempDir()
posterPath := writeLocalPoster(t, other)
job := localArtworkJob("file://" + posterPath)
p, jobs, cacher, _, _ := newLocalProcessorForTest(job, []string{root}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Failed != 1 {
t.Fatalf("stats = %+v, want 1 failed", stats)
}
if !isStableProviderImageFailure(jobs.failedText) {
t.Fatalf("confinement failure %q must be a stable failure", jobs.failedText)
}
if len(cacher.bytesReq) != 0 {
t.Fatal("must not read or cache a file outside library roots")
}
}
func TestProcessLocalImageRejectsDotDotTraversal(t *testing.T) {
root := t.TempDir()
outside := t.TempDir()
posterPath := writeLocalPoster(t, outside)
traversal := root + "/../" + filepath.Base(outside) + "/" + filepath.Base(posterPath)
job := localArtworkJob("file://" + traversal)
p, jobs, cacher, _, _ := newLocalProcessorForTest(job, []string{root}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Failed != 1 {
t.Fatalf("stats = %+v, want 1 failed", stats)
}
if !isStableProviderImageFailure(jobs.failedText) {
t.Fatalf("traversal failure %q must be a stable failure", jobs.failedText)
}
if len(cacher.bytesReq) != 0 {
t.Fatal("must not cache a traversal path")
}
}
func TestProcessLocalImageAcceptsLogicalPathUnderSymlinkedRoot(t *testing.T) {
// The scanner records logical paths under symlinked roots; confinement is
// deliberately lexical-on-logical (no EvalSymlinks).
realRoot := t.TempDir()
writeLocalPoster(t, realRoot)
linkParent := t.TempDir()
linkedRoot := filepath.Join(linkParent, "library")
if err := os.Symlink(realRoot, linkedRoot); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
job := localArtworkJob("file://" + filepath.Join(linkedRoot, "poster.jpg"))
p, jobs, cacher, _, _ := newLocalProcessorForTest(job, []string{linkedRoot}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Succeeded != 1 {
t.Fatalf("stats = %+v (failed text %q), want success under symlinked root", stats, jobs.failedText)
}
if len(cacher.bytesReq) != 1 {
t.Fatal("expected the logical path to be read and cached")
}
}
func TestProcessLocalImageMissingFileIsStableFailure(t *testing.T) {
root := t.TempDir()
job := localArtworkJob("file://" + filepath.Join(root, "poster.jpg"))
p, jobs, _, _, _ := newLocalProcessorForTest(job, []string{root}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Failed != 1 {
t.Fatalf("stats = %+v, want 1 failed", stats)
}
if !isStableProviderImageFailure(jobs.failedText) {
t.Fatalf("ENOENT failure %q must be a stable failure", jobs.failedText)
}
}
func TestProcessLocalImageRejectsSymlinkedLeaf(t *testing.T) {
root := t.TempDir()
target := writeLocalPoster(t, root)
link := filepath.Join(root, "linked.jpg")
if err := os.Symlink(target, link); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
job := localArtworkJob("file://" + link)
p, _, cacher, _, _ := newLocalProcessorForTest(job, []string{root}, "")
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Failed != 1 {
t.Fatalf("stats = %+v, want 1 failed", stats)
}
if len(cacher.bytesReq) != 0 {
t.Fatal("must not cache a symlinked leaf")
}
}
func TestProcessLocalImageDeletesStalePrefixOnRecache(t *testing.T) {
root := t.TempDir()
posterPath := writeLocalPoster(t, root)
job := localArtworkJob("file://" + posterPath)
stale := "local/movies/movie-1/00000000/poster/original.webp"
p, _, cacher, _, deleter := newLocalProcessorForTest(job, []string{root}, stale)
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Succeeded != 1 {
t.Fatalf("stats = %+v, want success", stats)
}
if len(cacher.bytesReq) != 1 {
t.Fatal("expected one cache call")
}
if len(deleter.prefixes) != 1 || deleter.prefixes[0] != "local/movies/movie-1/00000000/poster/" {
t.Fatalf("stale prefixes deleted = %v", deleter.prefixes)
}
}
func TestProcessLocalImageSkipsPrefixDeleteWhenUnchanged(t *testing.T) {
root := t.TempDir()
posterPath := writeLocalPoster(t, root)
job := localArtworkJob("file://" + posterPath)
// First pass discovers the hash for these bytes.
probe, _, probeCacher, probeItems, _ := newLocalProcessorForTest(job, []string{root}, "")
if _, err := probe.RunOnce(context.Background(), "w1", 10, 1); err != nil {
t.Fatalf("probe RunOnce: %v", err)
}
if len(probeCacher.bytesReq) != 1 {
t.Fatal("probe cache call missing")
}
// Second pass with the same stored cached path must not delete anything.
p, _, _, _, deleter := newLocalProcessorForTest(job, []string{root}, probeItems.cachedPath)
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Succeeded != 1 {
t.Fatalf("stats = %+v, want success", stats)
}
if len(deleter.prefixes) != 0 {
t.Fatalf("unchanged art must not delete prefixes, got %v", deleter.prefixes)
}
}
func TestProcessLocalImageFailsWithoutRootResolver(t *testing.T) {
root := t.TempDir()
posterPath := writeLocalPoster(t, root)
job := localArtworkJob("file://" + posterPath)
jobs := &fakeLocalImageCacheJobs{
fakeImageCacheJobs: fakeImageCacheJobs{claimed: []*models.MetadataImageCacheJob{job}},
}
cacher := &fakeByteImageCacher{thumbhash: "th"}
p := NewImageCacheProcessorWithTargets(jobs, cacher, nil, ImageCacheProcessorTargets{Items: &fakeItemArtworkUpdater{updated: true}})
stats, err := p.RunOnce(context.Background(), "w1", 10, 1)
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if stats.Failed != 1 {
t.Fatalf("stats = %+v, want failure without a library root resolver", stats)
}
if len(cacher.bytesReq) != 0 {
t.Fatal("must not cache without confinement roots")
}
}
internal/metadata/local_artwork_db_test.go
+107
View File
@@ -0,0 +1,107 @@
package metadata
import (
"context"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
)
func localArtworkTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func TestCurrentTargetCachedPathItem(t *testing.T) {
pool := localArtworkTestPool(t)
ctx := context.Background()
contentID := fmt.Sprintf("local-art-%d", time.Now().UnixNano())
if _, err := pool.Exec(ctx, `
INSERT INTO media_items (content_id, type, title, status, genres, poster_path, poster_source_path)
VALUES ($1, 'movie', 'Local Art Test', 'matched', '{}'::text[], $2, $3)
`, contentID, "local/movies/"+contentID+"/deadbeef/poster/original.webp", "file:///media/movies/Film/poster.jpg"); err != nil {
t.Fatalf("seed item: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = $1`, contentID)
})
repo := NewImageCacheJobRepository(pool)
job := &models.MetadataImageCacheJob{
TargetType: ImageCacheTargetItem,
TargetContentID: contentID,
ImageType: ImageCacheImagePoster,
}
cached, err := repo.CurrentTargetCachedPath(ctx, job)
if err != nil {
t.Fatalf("CurrentTargetCachedPath: %v", err)
}
if want := "local/movies/" + contentID + "/deadbeef/poster/original.webp"; cached != want {
t.Fatalf("cached = %q, want %q", cached, want)
}
source, err := repo.CurrentTargetSourcePath(ctx, job)
if err != nil {
t.Fatalf("CurrentTargetSourcePath: %v", err)
}
if source != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("source = %q", source)
}
// Missing rows report empty, not an error.
missing, err := repo.CurrentTargetCachedPath(ctx, &models.MetadataImageCacheJob{
TargetType: ImageCacheTargetItem,
TargetContentID: contentID + "-missing",
ImageType: ImageCacheImagePoster,
})
if err != nil || missing != "" {
t.Fatalf("missing row: cached=%q err=%v", missing, err)
}
}
func TestEnqueueBatchAcceptsLocalSourceDB(t *testing.T) {
pool := localArtworkTestPool(t)
ctx := context.Background()
contentID := fmt.Sprintf("local-art-enq-%d", time.Now().UnixNano())
repo := NewImageCacheJobRepository(pool)
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM metadata_image_cache_jobs WHERE target_content_id = $1`, contentID)
})
n, err := repo.EnqueueBatch(ctx, []EnqueueImageCacheJobInput{{
TargetType: ImageCacheTargetItem,
TargetContentID: contentID,
SeriesID: contentID,
SourcePath: "file:///media/movies/Film/poster.jpg",
ContentType: "movies",
ImageType: ImageCacheImagePoster,
}})
if err != nil {
t.Fatalf("EnqueueBatch: %v", err)
}
if n != 1 {
t.Fatalf("enqueued %d, want 1", n)
}
var providerID string
if err := pool.QueryRow(ctx,
`SELECT provider_id FROM metadata_image_cache_jobs WHERE target_content_id = $1`, contentID,
).Scan(&providerID); err != nil {
t.Fatalf("read job: %v", err)
}
if providerID != "local" {
t.Fatalf("provider_id = %q, want local", providerID)
}
}
internal/metadata/local_image_source_test.go
+231
View File
@@ -0,0 +1,231 @@
package metadata
import (
"context"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestIsLocalImageSourcePath(t *testing.T) {
cases := []struct {
path string
want bool
}{
{"file:///media/movies/Film/poster.jpg", true},
{"FILE:///media/movies/Film/poster.jpg", true},
{" file:///media/poster.jpg ", true},
{"https://image.tmdb.org/t/p/original/a.jpg", false},
{"tvdb://banners/a.jpg", false},
{"local/movies/abc/poster/original.webp", false},
{"tmdb/movies/550/poster/original.webp", false},
{"", false},
}
for _, tc := range cases {
if got := isLocalImageSourcePath(tc.path); got != tc.want {
t.Errorf("isLocalImageSourcePath(%q) = %v, want %v", tc.path, got, tc.want)
}
}
}
func TestIsCachedImagePathExcludesLocalSources(t *testing.T) {
if isCachedImagePath("file:///media/movies/Film/poster.jpg") {
t.Fatal("file:// source must not be treated as a cached image path")
}
if !isCachedImagePath("local/movies/abc/deadbeef/poster/original.webp") {
t.Fatal("local/ cached key must remain a cached image path")
}
if !isCachedImagePath("tmdb/movies/550/poster/original.webp") {
t.Fatal("remote cached key must remain a cached image path")
}
}
func TestPreserveCachedArtworkRoutesLocalSource(t *testing.T) {
// No prior cached art: the file:// source must never land in *_path; the
// pre-cache window keeps the path empty until the cache job completes.
path, thumb, source := preserveCachedArtwork(
"file:///media/movies/Film/poster.jpg", "", "", "", "",
)
if path != "" {
t.Fatalf("path = %q, want empty during pre-cache window", path)
}
if thumb != "" {
t.Fatalf("thumb = %q, want empty", thumb)
}
if source != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("source = %q", source)
}
// Prior cached art keeps serving while the local source waits to cache.
path, thumb, source = preserveCachedArtwork(
"file:///media/movies/Film/poster.jpg", "",
"tmdb/movies/550/poster/original.webp", "https://image.tmdb.org/a.jpg", "th",
)
if path != "tmdb/movies/550/poster/original.webp" {
t.Fatalf("path = %q, want prior cached key", path)
}
if thumb != "th" {
t.Fatalf("thumb = %q", thumb)
}
if source != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("source = %q", source)
}
}
func TestPrepareItemImagesForQueueRoutesLocalSource(t *testing.T) {
item := &models.MediaItem{
ContentID: "movie-1",
Type: "movie",
PosterPath: "file:///media/movies/Film/poster.jpg",
}
prepareItemImagesForQueue(item, nil)
if item.PosterPath != "" {
t.Fatalf("PosterPath = %q, want empty during pre-cache window", item.PosterPath)
}
if item.PosterSourcePath != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("PosterSourcePath = %q", item.PosterSourcePath)
}
}
func TestApplyBestImagesLocalCandidateAlwaysApplies(t *testing.T) {
item := &models.MediaItem{PosterPath: "tmdb/movies/550/poster/original.webp"}
applyBestImages(item, []RemoteImage{
{ProviderID: "nfo", URL: "file:///media/movies/Film/poster.jpg", Type: ImagePoster, Rating: 0},
}, MergeFillEmpty, "en")
if item.PosterPath != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("PosterPath = %q, want local candidate to apply over existing art", item.PosterPath)
}
}
func TestApplyBestImagesRemoteSelectionUnchanged(t *testing.T) {
// Regression pins: a rating-0 remote candidate still cannot displace
// existing art in fill mode, while a rated one still can.
item := &models.MediaItem{PosterPath: "tmdb/movies/550/poster/original.webp"}
applyBestImages(item, []RemoteImage{
{ProviderID: "tmdb", URL: "https://image.tmdb.org/zero.jpg", Type: ImagePoster, Rating: 0},
}, MergeFillEmpty, "en")
if item.PosterPath != "tmdb/movies/550/poster/original.webp" {
t.Fatalf("PosterPath = %q, rating-0 remote must not displace existing art", item.PosterPath)
}
applyBestImages(item, []RemoteImage{
{ProviderID: "tmdb", URL: "https://image.tmdb.org/rated.jpg", Type: ImagePoster, Rating: 7.5},
}, MergeFillEmpty, "en")
if item.PosterPath != "https://image.tmdb.org/rated.jpg" {
t.Fatalf("PosterPath = %q, rated remote candidate must still apply", item.PosterPath)
}
empty := &models.MediaItem{}
applyBestImages(empty, []RemoteImage{
{ProviderID: "tmdb", URL: "https://image.tmdb.org/zero.jpg", Type: ImagePoster, Rating: 0},
}, MergeFillEmpty, "en")
if empty.PosterPath != "https://image.tmdb.org/zero.jpg" {
t.Fatalf("PosterPath = %q, rating-0 remote must still fill empty art", empty.PosterPath)
}
}
func TestBuildItemLocalizationRecordSkipsLocalCandidates(t *testing.T) {
// Language-neutral local files must not duplicate into per-language rows.
loc := buildItemLocalizationRecord(
nil,
"movie-1",
"de",
"movie",
&MetadataResult{},
[]RemoteImage{
{ProviderID: "nfo", URL: "file:///media/movies/Film/poster.jpg", Type: ImagePoster},
},
MergeFillEmpty,
"de",
false,
)
if loc.PosterPath != "" || loc.PosterSourcePath != "" {
t.Fatalf("localization picked up local candidate: path=%q source=%q", loc.PosterPath, loc.PosterSourcePath)
}
}
func TestItemImageCacheAttributionLocalSource(t *testing.T) {
item := &models.MediaItem{ContentID: "movie-1", TmdbID: "550"}
providerID, providerContentID := itemImageCacheAttribution(
item,
map[string]string{"tmdb": "550"},
nil,
"file:///media/movies/Film/poster.jpg",
)
if providerID != "local" {
t.Fatalf("providerID = %q, want local", providerID)
}
if providerContentID != "movie-1" {
t.Fatalf("providerContentID = %q, want the item content ID", providerContentID)
}
}
func TestEnqueueItemImagesAcceptsLocalSource(t *testing.T) {
service := &MetadataService{}
enqueuer := &recordingImageCacheJobEnqueuer{}
service.SetAutoCacheImages(true)
service.SetImageCacheJobEnqueuer(enqueuer)
item := &models.MediaItem{
ContentID: "movie-1",
Type: "movie",
PosterSourcePath: "file:///media/movies/Film/poster.jpg",
}
service.enqueueItemImages(context.Background(), item, nil, nil)
if len(enqueuer.inputs) != 1 {
t.Fatalf("enqueued %d jobs, want 1", len(enqueuer.inputs))
}
in := enqueuer.inputs[0]
if in.SourcePath != "file:///media/movies/Film/poster.jpg" {
t.Fatalf("SourcePath = %q", in.SourcePath)
}
if in.ProviderID != "local" {
t.Fatalf("ProviderID = %q, want local", in.ProviderID)
}
if in.ProviderContentID != "movie-1" {
t.Fatalf("ProviderContentID = %q", in.ProviderContentID)
}
}
func TestNormalizeImageCacheJobInputAcceptsLocalSource(t *testing.T) {
in, ok := normalizeImageCacheJobInput(EnqueueImageCacheJobInput{
TargetType: ImageCacheTargetItem,
TargetContentID: "movie-1",
SourcePath: "file:///media/movies/Film/poster.jpg",
ContentType: "movies",
})
if !ok {
t.Fatal("local file:// source must be accepted")
}
if in.ProviderID != "local" {
t.Fatalf("ProviderID = %q, want local", in.ProviderID)
}
// The other system schemes stay rejected.
for _, rejected := range []string{
"s3://bucket/key.jpg",
"local://x.jpg",
"upload://x.jpg",
"generated://x.jpg",
"tmdb/movies/550/poster/original.webp",
} {
if _, ok := normalizeImageCacheJobInput(EnqueueImageCacheJobInput{SourcePath: rejected}); ok {
t.Errorf("source %q must stay rejected", rejected)
}
}
}
func TestIsStableProviderImageFailureLocalClasses(t *testing.T) {
for _, text := range []string{
"local image missing: /media/movies/Film/poster.jpg",
"local image forbidden: /media/movies/Film/poster.jpg",
"local image path outside library roots: /etc/passwd",
"unexpected status 404",
} {
if !isStableProviderImageFailure(text) {
t.Errorf("expected stable failure classification for %q", text)
}
}
if isStableProviderImageFailure("local image read failed: io timeout") {
t.Error("transient local read errors must keep the normal backoff")
}
}
internal/metadata/match_candidates.go
+75 -10
View File
@@ -256,10 +256,17 @@ func NormalizeCandidates(results []SearchResult, contentType string) []MatchCand
sort.Strings(sources)
b.candidate.Sources = sources
// Compute agreement hints.
if len(sources) > 1 {
// Compute agreement hints from corroborating sources only: a local
// sidecar (nfo) echoing a remote result is not provider agreement.
corroborating := make([]string, 0, len(sources))
for _, s := range sources {
if !nonCorroboratingSources[strings.ToLower(s)] {
corroborating = append(corroborating, s)
}
}
if len(corroborating) > 1 {
b.candidate.AgreementHints = append(b.candidate.AgreementHints,
"agreed_by_"+strings.Join(sources, "_and_"))
"agreed_by_"+strings.Join(corroborating, "_and_"))
}
candidates = append(candidates, b.candidate)
@@ -413,32 +420,85 @@ func topTieGroup(scored []scoredMatchCandidate) []scoredMatchCandidate {
return group
}
// nonCorroboratingSources are provider slugs whose search results must never
// count as independent corroboration: a local sidecar (NFO) echoing a title
// is not a second provider agreeing on identity.
var nonCorroboratingSources = map[string]bool{"nfo": true}
// pickByProviderPriority returns the group candidate whose Sources include the
// highest-priority provider (providerPriority is ordered highest-first, e.g. the
// library's chain order). Falls back to the top-scored candidate when there is no
// priority info or no source matches.
//
// The group has already been proven to be one distinct show, so two defenses
// against ID-less local candidates apply: when any group member carries
// provider IDs, ID-less members are excluded from the priority pick (a
// title-only NFO first in the chain must not downgrade a remotely-found movie
// to an unenriched local item), and the winner adopts the union of the
// group's provider IDs (sound because candidatesAreSingleDistinctShow already
// verified they cannot conflict).
func pickByProviderPriority(group []scoredMatchCandidate, providerPriority []string) *MatchCandidate {
eligible := group
if anyCandidateHasProviderIDs(group) {
eligible = make([]scoredMatchCandidate, 0, len(group))
for _, c := range group {
if providerIDRichness(c.candidate.ProviderIDs) > 0 {
eligible = append(eligible, c)
}
}
}
winner := &eligible[0].candidate
for _, prov := range providerPriority {
for i := range group {
for _, s := range group[i].candidate.Sources {
for i := range eligible {
for _, s := range eligible[i].candidate.Sources {
if strings.EqualFold(s, prov) {
return &group[i].candidate
winner = &eligible[i].candidate
return adoptGroupProviderIDs(winner, group)
}
}
}
}
return &group[0].candidate
return adoptGroupProviderIDs(winner, group)
}
func anyCandidateHasProviderIDs(group []scoredMatchCandidate) bool {
for _, c := range group {
if providerIDRichness(c.candidate.ProviderIDs) > 0 {
return true
}
}
return false
}
// adoptGroupProviderIDs returns a copy of winner carrying the union of the
// group's canonical provider IDs.
func adoptGroupProviderIDs(winner *MatchCandidate, group []scoredMatchCandidate) *MatchCandidate {
adopted := *winner
adopted.ProviderIDs = make(map[string]string, len(winner.ProviderIDs))
for k, v := range winner.ProviderIDs {
adopted.ProviderIDs[k] = v
}
for _, c := range group {
for _, key := range canonicalCandidateIDKeys {
if v := strings.TrimSpace(c.candidate.ProviderIDs[key]); v != "" && adopted.ProviderIDs[key] == "" {
adopted.ProviderIDs[key] = v
}
}
}
return &adopted
}
// distinctSourceCount returns how many distinct providers (case-insensitive
// Sources values) appear across the candidate group — i.e. how many independent
// providers returned this show.
// providers returned this show. Non-corroborating local sources (nfo) never
// count.
func distinctSourceCount(group []scoredMatchCandidate) int {
seen := make(map[string]struct{})
for _, c := range group {
for _, s := range c.candidate.Sources {
s = strings.ToLower(strings.TrimSpace(s))
if s != "" {
if s != "" && !nonCorroboratingSources[s] {
seen[s] = struct{}{}
}
}
@@ -625,7 +685,11 @@ func candidatePrimaryProvider(candidate MatchCandidate) string {
return ""
}
func selectRefreshMatchCandidate(existing *models.MediaItem, candidates []MatchCandidate) (*MatchCandidate, bool) {
// selectRefreshMatchCandidate picks the refresh winner anchored on the
// existing item's identity. idOverrides carries identity-hint values that won
// the pre-search conflict policy (e.g. NFO <uniqueid> on a manual refresh)
// and takes precedence over the stored ids for anchoring.
func selectRefreshMatchCandidate(existing *models.MediaItem, idOverrides map[string]string, candidates []MatchCandidate) (*MatchCandidate, bool) {
if existing == nil || len(candidates) == 0 {
return nil, false
}
@@ -638,6 +702,7 @@ func selectRefreshMatchCandidate(existing *models.MediaItem, candidates []MatchC
TvdbID: existing.TvdbID,
ImdbID: existing.ImdbID,
}
overrideHintIDs(hints, idOverrides)
return selectInitialMatchCandidate(hints, candidates, nil)
}
internal/metadata/match_candidates_defang_test.go
+87
View File
@@ -0,0 +1,87 @@
package metadata
import (
"testing"
)
// An ID-less candidate (e.g. the NFO title-only search candidate) must never
// win the proven-single-show tie-break on chain priority when an ID-bearing
// member exists in the group — and the winner must adopt the union of the
// group's provider IDs.
func TestSelectInitialMatch_IDlessCandidateExcludedWhenIDBearingExists(t *testing.T) {
hints := &MatchHints{Title: "Inception", Year: 2010, Type: "movie"}
candidates := []MatchCandidate{
{Title: "Inception", Year: 2010, ContentType: "movie", ProviderIDs: map[string]string{}, Sources: []string{"nfo"}},
{Title: "Inception", Year: 2010, ContentType: "movie", ProviderIDs: map[string]string{"tmdb": "27205"}, Sources: []string{"tmdb"}},
{Title: "Inception", Year: 2010, ContentType: "movie", ProviderIDs: map[string]string{"tvdb": "83898"}, Sources: []string{"tvdb"}},
}
// nfo sits first in the library chain priority.
winner, ok := selectInitialMatchCandidate(hints, candidates, []string{"nfo", "tmdb", "tvdb"})
if !ok || winner == nil {
t.Fatalf("winner = %v ok = %v, want accepted candidate", winner, ok)
}
if winner.ProviderIDs["tmdb"] != "27205" {
t.Errorf("winner tmdb id = %q, want 27205 (ID-less candidate must not win; union adopted)", winner.ProviderIDs["tmdb"])
}
if winner.ProviderIDs["tvdb"] != "83898" {
t.Errorf("winner tvdb id = %q, want 83898 (union of the proven-single-show group's IDs)", winner.ProviderIDs["tvdb"])
}
}
// The nfo source must not count as independent corroboration: a local file
// echoing a title is not a second provider agreeing. With no hint year, a
// remote candidate corroborated only by the NFO must not be auto-accepted via
// the multi-source arm.
func TestSelectInitialMatch_NFOSourceDoesNotCorroborate(t *testing.T) {
hints := &MatchHints{Title: "Some Show", Type: "movie"} // no year hint
candidates := []MatchCandidate{
{Title: "Some Show", Year: 2015, ContentType: "movie", ProviderIDs: map[string]string{"tmdb": "1"}, Sources: []string{"tmdb"}},
{Title: "Some Show", Year: 2015, ContentType: "movie", ProviderIDs: map[string]string{}, Sources: []string{"nfo"}},
}
winner, ok := selectInitialMatchCandidate(hints, candidates, nil)
if ok {
t.Fatalf("winner = %+v, want rejection (nfo echo must not stand in for a missing year)", winner)
}
}
// A genuinely multi-source group (two remote providers) still corroborates.
func TestSelectInitialMatch_RemoteMultiSourceStillCorroborates(t *testing.T) {
hints := &MatchHints{Title: "Some Show", Type: "movie"} // no year hint
candidates := []MatchCandidate{
{Title: "Some Show", Year: 2015, ContentType: "movie", ProviderIDs: map[string]string{"tmdb": "1"}, Sources: []string{"tmdb"}},
{Title: "Some Show", Year: 2015, ContentType: "movie", ProviderIDs: map[string]string{"tvdb": "2"}, Sources: []string{"tvdb"}},
}
winner, ok := selectInitialMatchCandidate(hints, candidates, nil)
if !ok || winner == nil {
t.Fatalf("winner = %v ok = %v, want accepted candidate", winner, ok)
}
}
// The nfo source must not manufacture an "agreed_by" hint with a remote
// provider when their results merge into one candidate.
func TestNormalizeCandidates_NFOExcludedFromAgreementHints(t *testing.T) {
results := []SearchResult{
{Name: "Movie", Year: 2020, ProviderIDs: map[string]string{"tmdb": "7"}, Provider: "tmdb"},
{Name: "Movie", Year: 2020, ProviderIDs: map[string]string{"tmdb": "7"}, Provider: "nfo"},
}
candidates := NormalizeCandidates(results, "movie")
if len(candidates) != 1 {
t.Fatalf("candidates = %d, want 1 merged candidate", len(candidates))
}
if len(candidates[0].AgreementHints) != 0 {
t.Errorf("agreement hints = %v, want none (nfo does not corroborate)", candidates[0].AgreementHints)
}
// Two genuine remote sources still agree.
remote := []SearchResult{
{Name: "Movie", Year: 2020, ProviderIDs: map[string]string{"tmdb": "7"}, Provider: "tmdb"},
{Name: "Movie", Year: 2020, ProviderIDs: map[string]string{"tmdb": "7"}, Provider: "trakt"},
}
candidates = NormalizeCandidates(remote, "movie")
if len(candidates) != 1 || len(candidates[0].AgreementHints) != 1 {
t.Fatalf("remote agreement hints = %+v, want one", candidates)
}
}
internal/metadata/match_candidates_test.go
+3
View File
@@ -751,6 +751,7 @@ func TestSelectRefreshMatchCandidate_AcceptsCandidateWithPartialTrustedIDCoverag
TmdbID: "603",
ImdbID: "tt0133093",
},
nil,
[]MatchCandidate{
{
Title: "The Matrix",
@@ -775,6 +776,7 @@ func TestSelectRefreshMatchCandidate_RejectsCandidateWithoutTrustedIDMatches(t *
TmdbID: "603",
ImdbID: "tt0133093",
},
nil,
[]MatchCandidate{
{
Title: "The Matrix",
@@ -799,6 +801,7 @@ func TestSelectRefreshMatchCandidate_RejectsConflictingTrustedIDCandidate(t *tes
TmdbID: "603",
ImdbID: "tt0133093",
},
nil,
[]MatchCandidate{
{
Title: "The Matrix",
internal/metadata/merge.go
+13 -9
View File
@@ -40,15 +40,17 @@ func MergeMetadata(source, target *MetadataResult, locked []MetadataField, mode
mergeFloat(&target.Ratings.RTAudience, source.Ratings.RTAudience, mode)
}
// Always fill year (not lockable — identity field)
mergeInt(&target.Year, source.Year, mode)
// Year and release/air dates lock together under FieldReleaseDates.
if !isLocked(FieldReleaseDates) {
mergeInt(&target.Year, source.Year, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
}
mergeScalar(&target.OriginalLanguage, source.OriginalLanguage, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
// Series fields (not lockable — structural)
mergeInt(&target.SeasonCount, source.SeasonCount, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
mergeScalar(&target.ShowStatus, source.ShowStatus, mode)
if !isLocked(FieldAirSchedule) {
mergeScalar(&target.AirTime, source.AirTime, mode)
@@ -117,13 +119,15 @@ func MergeGlobalMetadata(source, target *MetadataResult, locked []MetadataField,
mergeFloat(&target.Ratings.RTAudience, source.Ratings.RTAudience, mode)
}
mergeInt(&target.Year, source.Year, mode)
if !isLocked(FieldReleaseDates) {
mergeInt(&target.Year, source.Year, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
}
mergeScalar(&target.OriginalTitle, source.OriginalTitle, mode)
mergeScalar(&target.OriginalLanguage, source.OriginalLanguage, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
mergeInt(&target.SeasonCount, source.SeasonCount, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
mergeScalar(&target.ShowStatus, source.ShowStatus, mode)
if !isLocked(FieldAirSchedule) {
mergeScalar(&target.AirTime, source.AirTime, mode)
internal/metadata/nfo/builtin_registration_test.go
+64
View File
@@ -0,0 +1,64 @@
package nfo
import (
"context"
"os"
"path/filepath"
"slices"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
)
// Importing the nfo package must register the provider in the builtin
// registry under its capability id — buildProviders resolves the seeded
// 'nfo' chain entry through this single registration point.
func TestPackageImportRegistersBuiltinProvider(t *testing.T) {
if !slices.Contains(metadata.BuiltinCapabilityIDs(), "nfo") {
t.Fatalf("BuiltinCapabilityIDs() = %v, want to contain nfo", metadata.BuiltinCapabilityIDs())
}
}
// The provider implements the narrow IdentityHintProvider contract: a curated
// <uniqueid> becomes a trusted identity hint before Phase-1 search.
func TestProviderIdentityHintsFromNFOFile(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Obscure Film (2019).mkv")
nfoBody := `<?xml version="1.0" encoding="UTF-8"?>
<movie>
<title>Obscure Film</title>
<year>2019</year>
<uniqueid type="tmdb">424242</uniqueid>
<uniqueid type="imdb">tt0424242</uniqueid>
</movie>`
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte(nfoBody), 0o644); err != nil {
t.Fatalf("write nfo: %v", err)
}
var hinter metadata.IdentityHintProvider = NewProvider()
hints := hinter.IdentityHints(context.Background(), metadata.SearchQuery{
Title: "Obscure Film",
ContentType: "movie",
FilePath: moviePath,
})
if hints["tmdb"] != "424242" {
t.Errorf("tmdb hint = %q, want 424242", hints["tmdb"])
}
if hints["imdb"] != "tt0424242" {
t.Errorf("imdb hint = %q, want tt0424242", hints["imdb"])
}
// A title-only NFO yields no identity hints (the title flows through the
// defanged search candidate instead).
titleOnly := t.TempDir()
if err := os.WriteFile(filepath.Join(titleOnly, "movie.nfo"), []byte(`<movie><title>Home Video</title></movie>`), 0o644); err != nil {
t.Fatalf("write nfo: %v", err)
}
hints = hinter.IdentityHints(context.Background(), metadata.SearchQuery{
ContentType: "movie",
FilePath: filepath.Join(titleOnly, "Home Video.mkv"),
})
if len(hints) != 0 {
t.Errorf("title-only NFO hints = %v, want none", hints)
}
}
internal/metadata/nfo/images.go
+96
View File
@@ -0,0 +1,96 @@
package nfo
import (
"context"
"os"
"path/filepath"
"strings"
"github.com/Silo-Server/silo-server/internal/metadata"
)
// maxLocalArtworkBytes caps sidecar artwork reads at 8 MiB (mirrors the
// audiobook sidecar cover guard). Discovery rejects larger files up front so
// the cache processor never queues an oversized read.
const maxLocalArtworkBytes = 8 << 20
var localArtworkExtensions = []string{".jpg", ".jpeg", ".png", ".webp"}
// localArtworkNames maps each image type to its sidecar base names in
// precedence order. "{basename}" expands to the media file's basename and is
// searched next to the media files; plain names are searched only in the
// directory-level sidecar search paths, which upstream code restricts to
// single-content observed roots (canUseObservedRootForDirectorySidecars), so
// one folder.jpg in a flat multi-movie folder attaches to none of them.
var localArtworkNames = map[metadata.ImageType][]string{
metadata.ImagePoster: {"poster", "folder", "cover", "{basename}-poster"},
metadata.ImageBackdrop: {"fanart", "backdrop", "background", "{basename}-fanart"},
metadata.ImageLogo: {"logo", "clearlogo", "{basename}-logo"},
}
// GetImages implements metadata.ImageProvider by discovering sidecar artwork
// (poster.jpg, fanart.jpg, clearlogo.png, <basename>-poster.jpg, ...) next to
// the item's media files. Returned URLs use the file:// scheme with the
// absolute logical path; the image cache processor copies them into S3 so
// library files are never served directly. Local art applies even when no
// .nfo file is present.
func (p *Provider) GetImages(_ context.Context, req metadata.ImageRequest) ([]metadata.RemoteImage, error) {
searchDirs := compactNFOPaths(req.PrimarySidecarSearchPaths)
mediaFiles := compactNFOPaths(append([]string{req.RepresentativeFilePath}, req.AllGroupFilePaths...))
if len(searchDirs) == 0 && len(mediaFiles) == 0 {
return nil, nil
}
var images []metadata.RemoteImage
for _, imageType := range []metadata.ImageType{metadata.ImagePoster, metadata.ImageBackdrop, metadata.ImageLogo} {
if path := findLocalArtwork(localArtworkNames[imageType], searchDirs, mediaFiles); path != "" {
images = append(images, metadata.RemoteImage{
ProviderID: p.Slug(),
URL: "file://" + path,
Type: imageType,
Rating: 0,
})
}
}
return images, nil
}
// findLocalArtwork returns the absolute path of the first matching sidecar
// image, trying names in precedence order across every candidate location.
func findLocalArtwork(names []string, searchDirs []string, mediaFiles []string) string {
for _, name := range names {
if suffix, ok := strings.CutPrefix(name, "{basename}"); ok {
for _, mediaFile := range mediaFiles {
dir := filepath.Dir(mediaFile)
base := strings.TrimSuffix(filepath.Base(mediaFile), filepath.Ext(mediaFile))
if path := findLocalArtworkFile(dir, base+suffix); path != "" {
return path
}
}
continue
}
for _, dir := range searchDirs {
if path := findLocalArtworkFile(dir, name); path != "" {
return path
}
}
}
return ""
}
func findLocalArtworkFile(dir, name string) string {
for _, ext := range localArtworkExtensions {
path := filepath.Join(dir, name+ext)
info, err := os.Lstat(path)
if err != nil {
continue
}
// Reject symlinked leaves and non-regular files, and cap the size so
// the cache processor's re-checks agree with discovery.
if !info.Mode().IsRegular() || info.Size() == 0 || info.Size() > maxLocalArtworkBytes {
continue
}
return path
}
return ""
}
internal/metadata/nfo/images_test.go
+258
View File
@@ -0,0 +1,258 @@
package nfo
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
)
func writeImageFile(t *testing.T, path string) {
t.Helper()
if err := os.WriteFile(path, []byte("fake-image-bytes"), 0o644); err != nil {
t.Fatalf("writing %s: %v", path, err)
}
}
func imagesByType(images []metadata.RemoteImage) map[metadata.ImageType]metadata.RemoteImage {
byType := make(map[metadata.ImageType]metadata.RemoteImage, len(images))
for _, img := range images {
if _, ok := byType[img.Type]; !ok {
byType[img.Type] = img
}
}
return byType
}
func TestGetImagesFindsSidecarArtwork(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film (2020).mkv")
writeImageFile(t, moviePath)
writeImageFile(t, filepath.Join(dir, "poster.jpg"))
writeImageFile(t, filepath.Join(dir, "fanart.png"))
writeImageFile(t, filepath.Join(dir, "clearlogo.webp"))
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
AllGroupFilePaths: []string{moviePath},
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
byType := imagesByType(images)
if len(byType) != 3 {
t.Fatalf("expected poster/backdrop/logo, got %d images: %+v", len(images), images)
}
poster := byType[metadata.ImagePoster]
if poster.URL != "file://"+filepath.Join(dir, "poster.jpg") {
t.Fatalf("poster URL = %q", poster.URL)
}
if poster.Rating != 0 {
t.Fatalf("poster rating = %v, want 0", poster.Rating)
}
if poster.ProviderID != "nfo" {
t.Fatalf("poster provider = %q", poster.ProviderID)
}
if got := byType[metadata.ImageBackdrop].URL; got != "file://"+filepath.Join(dir, "fanart.png") {
t.Fatalf("backdrop URL = %q", got)
}
if got := byType[metadata.ImageLogo].URL; got != "file://"+filepath.Join(dir, "clearlogo.webp") {
t.Fatalf("logo URL = %q", got)
}
}
func TestGetImagesFilenamePrecedence(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film.mkv")
writeImageFile(t, moviePath)
// poster beats folder and cover; folder beats cover.
writeImageFile(t, filepath.Join(dir, "poster.png"))
writeImageFile(t, filepath.Join(dir, "folder.jpg"))
writeImageFile(t, filepath.Join(dir, "cover.jpg"))
// backdrop names: fanart beats backdrop/background.
writeImageFile(t, filepath.Join(dir, "background.jpg"))
writeImageFile(t, filepath.Join(dir, "fanart.jpg"))
// logo: logo beats clearlogo.
writeImageFile(t, filepath.Join(dir, "clearlogo.png"))
writeImageFile(t, filepath.Join(dir, "logo.png"))
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
byType := imagesByType(images)
if got := byType[metadata.ImagePoster].URL; got != "file://"+filepath.Join(dir, "poster.png") {
t.Fatalf("poster URL = %q, want poster.png", got)
}
if got := byType[metadata.ImageBackdrop].URL; got != "file://"+filepath.Join(dir, "fanart.jpg") {
t.Fatalf("backdrop URL = %q, want fanart.jpg", got)
}
if got := byType[metadata.ImageLogo].URL; got != "file://"+filepath.Join(dir, "logo.png") {
t.Fatalf("logo URL = %q, want logo.png", got)
}
}
func TestGetImagesBasenameScopedArtwork(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film (2020).mkv")
writeImageFile(t, moviePath)
writeImageFile(t, filepath.Join(dir, "Film (2020)-poster.jpg"))
writeImageFile(t, filepath.Join(dir, "Film (2020)-fanart.jpg"))
writeImageFile(t, filepath.Join(dir, "Film (2020)-logo.png"))
p := NewProvider()
// No directory search paths: basename-scoped art still applies.
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
byType := imagesByType(images)
if got := byType[metadata.ImagePoster].URL; got != "file://"+filepath.Join(dir, "Film (2020)-poster.jpg") {
t.Fatalf("poster URL = %q", got)
}
if got := byType[metadata.ImageBackdrop].URL; got != "file://"+filepath.Join(dir, "Film (2020)-fanart.jpg") {
t.Fatalf("backdrop URL = %q", got)
}
if got := byType[metadata.ImageLogo].URL; got != "file://"+filepath.Join(dir, "Film (2020)-logo.png") {
t.Fatalf("logo URL = %q", got)
}
}
func TestGetImagesFlatFolderGenericArtSuppressed(t *testing.T) {
// Multi-movie flat folder: the directory is not a safe sidecar search
// path (canUseObservedRootForDirectorySidecars suppression upstream), so
// generic names like folder.jpg must not attach to any movie in it.
dir := t.TempDir()
movieA := filepath.Join(dir, "Movie A.mkv")
writeImageFile(t, movieA)
writeImageFile(t, filepath.Join(dir, "Movie B.mkv"))
writeImageFile(t, filepath.Join(dir, "folder.jpg"))
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: movieA,
AllGroupFilePaths: []string{movieA},
// PrimarySidecarSearchPaths deliberately empty.
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
if len(images) != 0 {
t.Fatalf("expected no images for flat multi-movie folder, got %+v", images)
}
}
func TestGetImagesWorksWithoutNFO(t *testing.T) {
// Documented bonus: local art applies even with no .nfo present.
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film.mkv")
writeImageFile(t, moviePath)
writeImageFile(t, filepath.Join(dir, "poster.jpg"))
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
if len(images) != 1 || images[0].Type != metadata.ImagePoster {
t.Fatalf("expected exactly the poster, got %+v", images)
}
}
func TestGetImagesExtensionSet(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film.mkv")
writeImageFile(t, moviePath)
// .gif and .bmp are not in the accepted extension set.
writeImageFile(t, filepath.Join(dir, "poster.gif"))
writeImageFile(t, filepath.Join(dir, "fanart.bmp"))
writeImageFile(t, filepath.Join(dir, "logo.jpeg"))
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
if len(images) != 1 || images[0].Type != metadata.ImageLogo {
t.Fatalf("expected only logo.jpeg, got %+v", images)
}
}
func TestGetImagesRejectsSymlinksAndNonRegularFiles(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film.mkv")
writeImageFile(t, moviePath)
target := filepath.Join(dir, "elsewhere.jpg")
writeImageFile(t, target)
if err := os.Symlink(target, filepath.Join(dir, "poster.jpg")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
if err := os.Mkdir(filepath.Join(dir, "fanart.jpg"), 0o755); err != nil {
t.Fatalf("mkdir: %v", err)
}
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
if len(images) != 0 {
t.Fatalf("expected symlinked/non-regular sidecars rejected, got %+v", images)
}
}
func TestGetImagesRejectsOversizedFiles(t *testing.T) {
dir := t.TempDir()
moviePath := filepath.Join(dir, "Film.mkv")
writeImageFile(t, moviePath)
oversized := filepath.Join(dir, "poster.jpg")
f, err := os.Create(oversized)
if err != nil {
t.Fatalf("create: %v", err)
}
if err := f.Truncate(maxLocalArtworkBytes + 1); err != nil {
t.Fatalf("truncate: %v", err)
}
if err := f.Close(); err != nil {
t.Fatalf("close: %v", err)
}
p := NewProvider()
images, err := p.GetImages(context.Background(), metadata.ImageRequest{
ContentType: "movie",
RepresentativeFilePath: moviePath,
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetImages: %v", err)
}
if len(images) != 0 {
t.Fatalf("expected oversized sidecar rejected, got %+v", images)
}
}
internal/metadata/nfo/nfo.go
+369 -46
View File
@@ -4,7 +4,20 @@ import (
"bytes"
"encoding/xml"
"fmt"
"regexp"
"strconv"
"strings"
"github.com/Silo-Server/silo-server/internal/models"
)
// Content types emitted in parsedNFO.Type, matching the metadata package's
// content-type/level vocabulary.
const (
typeMovie = "movie"
typeSeries = "series"
typeSeason = "season"
typeEpisode = "episode"
)
// nfoUniqueID represents a <uniqueid> element inside an NFO XML file.
@@ -14,45 +27,135 @@ type nfoUniqueID struct {
Value string `xml:",chardata"`
}
// nfoActor represents an <actor> element (Kodi/Jellyfin cast entry).
// <thumb> URLs are intentionally ignored in v1.
type nfoActor struct {
Name string `xml:"name"`
Role string `xml:"role"`
Order *int `xml:"order"`
}
// nfoNamedRating represents a <rating> entry inside a <ratings> block.
type nfoNamedRating struct {
Name string `xml:"name,attr"`
Max string `xml:"max,attr"`
Value string `xml:"value"`
}
// nfoRatingsBlock represents the modern multi-source <ratings> container.
type nfoRatingsBlock struct {
Ratings []nfoNamedRating `xml:"rating"`
}
// nfoCommon holds the fields shared by <movie> and <tvshow> roots.
// encoding/xml flattens the embedded struct so both roots decode it in place.
//
// <set><name> (movie collections) is deliberately not mapped: MetadataResult
// has no collection field yet — see
// docs/superpowers/plans/2026-07-09-local-nfo-metadata-and-artwork.md §9.
type nfoCommon struct {
Title string `xml:"title"`
OriginalTitle string `xml:"originaltitle"`
Tagline string `xml:"tagline"`
Year int `xml:"year"`
Plot string `xml:"plot"`
Runtime string `xml:"runtime"`
Premiered string `xml:"premiered"`
MPAA string `xml:"mpaa"`
Genres []string `xml:"genre"`
Studios []string `xml:"studio"`
Countries []string `xml:"country"`
Tags []string `xml:"tag"`
LegacyRating string `xml:"rating"`
Ratings nfoRatingsBlock `xml:"ratings"`
UserRating string `xml:"userrating"`
Actors []nfoActor `xml:"actor"`
Directors []string `xml:"director"`
Credits []string `xml:"credits"`
UniqueIDs []nfoUniqueID `xml:"uniqueid"`
}
// nfoMovie represents the <movie> root element.
type nfoMovie struct {
XMLName xml.Name `xml:"movie"`
Title string `xml:"title"`
Year int `xml:"year"`
Plot string `xml:"plot"`
UniqueIDs []nfoUniqueID `xml:"uniqueid"`
XMLName xml.Name `xml:"movie"`
nfoCommon
ReleaseDate string `xml:"releasedate"`
}
// nfoTVShow represents the <tvshow> root element.
type nfoTVShow struct {
XMLName xml.Name `xml:"tvshow"`
Title string `xml:"title"`
Year int `xml:"year"`
Plot string `xml:"plot"`
UniqueIDs []nfoUniqueID `xml:"uniqueid"`
XMLName xml.Name `xml:"tvshow"`
nfoCommon
Aired string `xml:"aired"`
}
// nfoEpisode represents the <episodedetails> root element.
// nfoEpisode represents the <episodedetails> root element. Season/Episode are
// pointers so "declared as 0" (specials) is distinguishable from "absent" —
// declared numbers are advisory and checked against the filename-derived
// structure by the provider.
type nfoEpisode struct {
XMLName xml.Name `xml:"episodedetails"`
Title string `xml:"title"`
Season int `xml:"season"`
Episode int `xml:"episode"`
Plot string `xml:"plot"`
UniqueIDs []nfoUniqueID `xml:"uniqueid"`
XMLName xml.Name `xml:"episodedetails"`
Title string `xml:"title"`
Season *int `xml:"season"`
Episode *int `xml:"episode"`
Plot string `xml:"plot"`
Aired string `xml:"aired"`
Runtime string `xml:"runtime"`
LegacyRating string `xml:"rating"`
Ratings nfoRatingsBlock `xml:"ratings"`
UniqueIDs []nfoUniqueID `xml:"uniqueid"`
}
// parsedNFO holds extracted data from an NFO file.
// nfoSeason represents the <season> root element (Kodi/Jellyfin season
// sidecar). SeasonNumber is a pointer for the same absent-vs-zero reason as
// nfoEpisode.
type nfoSeason struct {
XMLName xml.Name `xml:"season"`
Title string `xml:"title"`
Plot string `xml:"plot"`
SeasonNumber *int `xml:"seasonnumber"`
}
// parsedNFO holds extracted data from an NFO file. Collections stay nil when
// the NFO declares no entries so downstream MergeFillEmpty early-returns and
// remote providers can fill them (no placeholder emissions).
type parsedNFO struct {
Title string
Year int
Overview string
TmdbID string
ImdbID string
TvdbID string
Type string // movie, series, episode
Season int
Episode int
Title string
OriginalTitle string
Tagline string
Year int
Overview string
Runtime int // minutes
ReleaseDate string // movies: <premiered> then <releasedate>
FirstAirDate string // series: <premiered> then <aired>
ContentRating string
Genres []string
Studios []string
Countries []string
Keywords []string
// Ratings normalized to their MetadataResult scales: IMDB/TMDB out of
// 10, Rotten Tomatoes out of 100. <userrating> is parsed but never
// emitted — MetadataResult has no per-user rating slot.
RatingIMDB float64
RatingTMDB float64
RatingRTCritic float64
RatingRTAudience float64
People []models.ItemPerson
TmdbID string
ImdbID string
TvdbID string
Type string // movie, series, season, episode
// Season/Episode carry the NFO's declared numbers when the matching Set
// flag is true. They are advisory only: naming owns structure, so the
// directory/filename-derived numbers win on conflict.
Season int
SeasonSet bool
Episode int
EpisodeSet bool
// MultiEpisode marks a document with more than one <episodedetails>
// root (multi-episode file). Out of scope in v1: the first block is
// parsed and the caller warns.
MultiEpisode bool
}
// parseNFOData parses NFO XML data and returns structured results.
@@ -66,10 +169,12 @@ func parseNFOData(data []byte) (*parsedNFO, error) {
}
switch rootTag {
case "movie":
case typeMovie:
return parseMovieNFO(data)
case "tvshow":
return parseTVShowNFO(data)
case typeSeason:
return parseSeasonNFO(data)
case "episodedetails":
return parseEpisodeNFO(data)
default:
@@ -95,13 +200,9 @@ func parseMovieNFO(data []byte) (*parsedNFO, error) {
if err := xmlUnmarshal(data, &m); err != nil {
return nil, fmt.Errorf("nfo: failed to parse movie: %w", err)
}
p := &parsedNFO{
Title: strings.TrimSpace(m.Title),
Year: m.Year,
Overview: strings.TrimSpace(m.Plot),
Type: "movie",
}
applyUniqueIDs(p, m.UniqueIDs)
p := parsedFromCommon(&m.nfoCommon, typeMovie)
p.ReleaseDate = firstDate(m.Premiered, m.ReleaseDate)
deriveYear(p, m.Premiered, m.ReleaseDate)
return p, nil
}
@@ -110,32 +211,254 @@ func parseTVShowNFO(data []byte) (*parsedNFO, error) {
if err := xmlUnmarshal(data, &s); err != nil {
return nil, fmt.Errorf("nfo: failed to parse tvshow: %w", err)
}
p := parsedFromCommon(&s.nfoCommon, typeSeries)
p.FirstAirDate = firstDate(s.Premiered, s.Aired)
deriveYear(p, s.Premiered, s.Aired)
return p, nil
}
func parseSeasonNFO(data []byte) (*parsedNFO, error) {
var s nfoSeason
if err := xmlUnmarshal(data, &s); err != nil {
return nil, fmt.Errorf("nfo: failed to parse season: %w", err)
}
p := &parsedNFO{
Title: strings.TrimSpace(s.Title),
Year: s.Year,
Overview: strings.TrimSpace(s.Plot),
Type: "series",
Type: typeSeason,
}
if s.SeasonNumber != nil {
p.Season = *s.SeasonNumber
p.SeasonSet = true
}
applyUniqueIDs(p, s.UniqueIDs)
return p, nil
}
func parseEpisodeNFO(data []byte) (*parsedNFO, error) {
decoder := xml.NewDecoder(bytes.NewReader(data))
var e nfoEpisode
if err := xmlUnmarshal(data, &e); err != nil {
if err := decoder.Decode(&e); err != nil {
return nil, fmt.Errorf("nfo: failed to parse episodedetails: %w", err)
}
p := &parsedNFO{
Title: strings.TrimSpace(e.Title),
Overview: strings.TrimSpace(e.Plot),
Type: "episode",
Season: e.Season,
Episode: e.Episode,
Title: strings.TrimSpace(e.Title),
Overview: strings.TrimSpace(e.Plot),
FirstAirDate: firstDate(e.Aired),
Type: typeEpisode,
MultiEpisode: hasMoreEpisodeDetails(decoder),
}
if e.Season != nil {
p.Season = *e.Season
p.SeasonSet = true
}
if e.Episode != nil {
p.Episode = *e.Episode
p.EpisodeSet = true
}
if minutes, err := strconv.Atoi(strings.TrimSpace(e.Runtime)); err == nil && minutes > 0 {
p.Runtime = minutes
}
applyRatingValues(p, e.Ratings, e.LegacyRating)
applyUniqueIDs(p, e.UniqueIDs)
return p, nil
}
// hasMoreEpisodeDetails reports whether the decoder (positioned after the
// first decoded root) still holds another <episodedetails> element. Kodi
// multi-episode files concatenate several roots into one .nfo.
func hasMoreEpisodeDetails(decoder *xml.Decoder) bool {
for {
tok, err := decoder.Token()
if err != nil {
return false
}
if se, ok := tok.(xml.StartElement); ok {
return se.Name.Local == "episodedetails"
}
}
}
// parsedFromCommon maps the shared <movie>/<tvshow> field set.
func parsedFromCommon(c *nfoCommon, contentType string) *parsedNFO {
p := &parsedNFO{
Title: strings.TrimSpace(c.Title),
OriginalTitle: strings.TrimSpace(c.OriginalTitle),
Tagline: strings.TrimSpace(c.Tagline),
Year: c.Year,
Overview: strings.TrimSpace(c.Plot),
ContentRating: strings.TrimSpace(c.MPAA),
Genres: cleanStrings(c.Genres),
Studios: cleanStrings(c.Studios),
Countries: cleanStrings(c.Countries),
Keywords: cleanStrings(c.Tags),
Type: contentType,
}
if minutes, err := strconv.Atoi(strings.TrimSpace(c.Runtime)); err == nil && minutes > 0 {
p.Runtime = minutes
}
applyRatings(p, c)
applyPeople(p, c)
applyUniqueIDs(p, c.UniqueIDs)
return p
}
var nfoDatePattern = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`)
// firstDate returns the first candidate that looks like an ISO date
// (YYYY-MM-DD); anything else is ignored rather than persisted.
func firstDate(candidates ...string) string {
for _, c := range candidates {
c = strings.TrimSpace(c)
if nfoDatePattern.MatchString(c) {
return c
}
}
return ""
}
// deriveYear fills Year when the NFO omits <year>, from the first candidate
// carrying a leading 4-digit year. Kodi exports commonly ship only <premiered>,
// sometimes as a bare year (2021) or a datetime (2021-05-01T20:00:00) that
// firstDate's strict ISO match rejects for storage; the year is still
// recoverable here for display.
func deriveYear(p *parsedNFO, candidates ...string) {
if p.Year != 0 {
return
}
for _, c := range candidates {
c = strings.TrimSpace(c)
if len(c) < 4 {
continue
}
if year, err := strconv.Atoi(c[:4]); err == nil && year > 0 {
p.Year = year
return
}
}
}
// applyRatings maps the modern multi-source <ratings> block with the legacy
// bare <rating> as an IMDB-slot fallback (Kodi's legacy field historically
// held the scraper's IMDB rating).
func applyRatings(p *parsedNFO, c *nfoCommon) {
applyRatingValues(p, c.Ratings, c.LegacyRating)
}
// applyRatingValues is the shared ratings mapper for <movie>/<tvshow>
// (via nfoCommon) and <episodedetails> roots.
func applyRatingValues(p *parsedNFO, block nfoRatingsBlock, legacyRating string) {
for _, r := range block.Ratings {
value, ok := parseRatingValue(r.Value)
if !ok {
continue
}
name := strings.ToLower(strings.TrimSpace(r.Name))
// Default the source scale when <rating> omits max: Rotten Tomatoes
// sources are 0-100, everything else 0-10. A blanket default of 10 for a
// bare RT score (e.g. tomatometer 60) would scale ×10 and clamp to 100.
// An explicit max attribute always wins.
max := 10.0
if name == "tomatometerallcritics" || name == "tomatometerallaudience" || name == "rottentomatoes" {
max = 100.0
}
if parsed, parsedOK := parseRatingValue(r.Max); parsedOK {
max = parsed
}
switch name {
case "imdb":
p.RatingIMDB = scaleRating(value, max, 10)
case "tmdb", "themoviedb":
p.RatingTMDB = scaleRating(value, max, 10)
case "tomatometerallcritics", "rottentomatoes":
p.RatingRTCritic = scaleRating(value, max, 100)
case "tomatometerallaudience":
p.RatingRTAudience = scaleRating(value, max, 100)
}
}
if p.RatingIMDB == 0 {
if value, ok := parseRatingValue(legacyRating); ok {
p.RatingIMDB = scaleRating(value, 10, 10)
}
}
// <userrating> is intentionally not emitted; see parsedNFO.
}
func parseRatingValue(raw string) (float64, bool) {
value, err := strconv.ParseFloat(strings.TrimSpace(raw), 64)
if err != nil || value <= 0 {
return 0, false
}
return value, true
}
func scaleRating(value, max, targetMax float64) float64 {
if max <= 0 {
max = targetMax
}
scaled := value * targetMax / max
if scaled > targetMax {
return targetMax
}
return scaled
}
// applyPeople maps <actor> entries to cast and <director>/<credits> to crew.
func applyPeople(p *parsedNFO, c *nfoCommon) {
people := make([]models.ItemPerson, 0, len(c.Actors)+len(c.Directors)+len(c.Credits))
for i, actor := range c.Actors {
name := strings.TrimSpace(actor.Name)
if name == "" {
continue
}
order := i
if actor.Order != nil {
order = *actor.Order
}
people = append(people, models.ItemPerson{
Person: models.Person{Name: name},
Kind: models.PersonKindActor,
Character: strings.TrimSpace(actor.Role),
SortOrder: order,
})
}
people = append(people, crewPeople(c.Directors, models.PersonKindDirector)...)
people = append(people, crewPeople(c.Credits, models.PersonKindWriter)...)
if len(people) > 0 {
p.People = people
}
}
func crewPeople(names []string, kind models.PersonKind) []models.ItemPerson {
people := make([]models.ItemPerson, 0, len(names))
for _, raw := range names {
name := strings.TrimSpace(raw)
if name == "" {
continue
}
people = append(people, models.ItemPerson{
Person: models.Person{Name: name},
Kind: kind,
SortOrder: len(people),
})
}
return people
}
// cleanStrings trims entries and drops empties; returns nil (not an empty
// slice) when nothing remains so merge early-returns apply.
func cleanStrings(values []string) []string {
out := make([]string, 0, len(values))
for _, v := range values {
if trimmed := strings.TrimSpace(v); trimmed != "" {
out = append(out, trimmed)
}
}
if len(out) == 0 {
return nil
}
return out
}
func xmlUnmarshal(data []byte, v any) error {
decoder := xml.NewDecoder(bytes.NewReader(data))
return decoder.Decode(v)
@@ -144,10 +467,10 @@ func xmlUnmarshal(data []byte, v any) error {
func applyUniqueIDs(p *parsedNFO, ids []nfoUniqueID) {
for _, uid := range ids {
val := strings.TrimSpace(uid.Value)
switch strings.ToLower(uid.Type) {
switch strings.ToLower(strings.TrimSpace(uid.Type)) {
case "imdb":
p.ImdbID = val
case "tmdb":
case "tmdb", "themoviedb":
p.TmdbID = val
case "tvdb":
p.TvdbID = val
internal/metadata/nfo/nfo_test.go
+413
View File
@@ -0,0 +1,413 @@
package nfo
import (
"reflect"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// fullMovieNFO is a realistic Kodi/Jellyfin-style export exercising the whole
// Phase-B field set: repeated collections, multi-source ratings, cast with
// role/order, directors and writers, and legacy + modern rating forms.
const fullMovieNFO = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<movie>
<title>Blade Runner</title>
<originaltitle>Blade Runner: The Original</originaltitle>
<sorttitle>Blade Runner 1</sorttitle>
<tagline>Man has made his match</tagline>
<plot>A blade runner must pursue replicants.</plot>
<runtime>117</runtime>
<premiered>1982-06-25</premiered>
<mpaa>R</mpaa>
<genre>Science Fiction</genre>
<genre>Thriller</genre>
<studio>Warner Bros.</studio>
<studio>The Ladd Company</studio>
<country>United States</country>
<country>Hong Kong</country>
<tag>dystopia</tag>
<tag>neo-noir</tag>
<userrating>9</userrating>
<ratings>
<rating name="imdb" max="10" default="true">
<value>8.1</value>
<votes>700000</votes>
</rating>
<rating name="themoviedb" max="10">
<value>7.9</value>
</rating>
<rating name="tomatometerallcritics" max="100">
<value>89</value>
</rating>
<rating name="tomatometerallaudience" max="100">
<value>91</value>
</rating>
</ratings>
<actor>
<name>Harrison Ford</name>
<role>Rick Deckard</role>
<order>0</order>
<thumb>https://example.com/ford.jpg</thumb>
</actor>
<actor>
<name>Rutger Hauer</name>
<role>Roy Batty</role>
<order>1</order>
</actor>
<director>Ridley Scott</director>
<credits>Hampton Fancher</credits>
<credits>David Peoples</credits>
<uniqueid type="imdb">tt0083658</uniqueid>
<uniqueid type="tmdb" default="true">78</uniqueid>
<set>
<name>Blade Runner Collection</name>
</set>
<fileinfo>
<streamdetails>
<video><codec>h264</codec></video>
</streamdetails>
</fileinfo>
</movie>`
// fullTVShowNFO exercises the series root, including <aired> as the
// FirstAirDate fallback and series-shaped collections.
const fullTVShowNFO = `<?xml version="1.0" encoding="utf-8"?>
<tvshow>
<title>Mr. Robot</title>
<originaltitle>Mr. Robot (Original)</originaltitle>
<plot>A cybersecurity engineer by day, hacker by night.</plot>
<runtime>49</runtime>
<premiered>2015-06-24</premiered>
<mpaa>TV-MA</mpaa>
<genre>Drama</genre>
<genre>Crime</genre>
<studio>USA Network</studio>
<tag>hacking</tag>
<ratings>
<rating name="imdb" max="10">
<value>8.5</value>
</rating>
</ratings>
<actor>
<name>Rami Malek</name>
<role>Elliot Alderson</role>
<order>0</order>
</actor>
<uniqueid type="tvdb" default="true">289590</uniqueid>
</tvshow>`
func TestParseNFOData_FullMovie(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(fullMovieNFO))
if err != nil {
t.Fatalf("parseNFOData() error = %v", err)
}
if p.Type != "movie" {
t.Errorf("Type = %q, want movie", p.Type)
}
if p.Title != "Blade Runner" {
t.Errorf("Title = %q", p.Title)
}
if p.OriginalTitle != "Blade Runner: The Original" {
t.Errorf("OriginalTitle = %q", p.OriginalTitle)
}
if p.Tagline != "Man has made his match" {
t.Errorf("Tagline = %q", p.Tagline)
}
if p.Runtime != 117 {
t.Errorf("Runtime = %d, want 117", p.Runtime)
}
if p.ReleaseDate != "1982-06-25" {
t.Errorf("ReleaseDate = %q, want 1982-06-25", p.ReleaseDate)
}
if p.FirstAirDate != "" {
t.Errorf("FirstAirDate = %q, want empty on a movie", p.FirstAirDate)
}
if p.Year != 1982 {
t.Errorf("Year = %d, want 1982 (derived from premiered)", p.Year)
}
if p.ContentRating != "R" {
t.Errorf("ContentRating = %q, want R", p.ContentRating)
}
if want := []string{"Science Fiction", "Thriller"}; !reflect.DeepEqual(p.Genres, want) {
t.Errorf("Genres = %#v, want %#v", p.Genres, want)
}
if want := []string{"Warner Bros.", "The Ladd Company"}; !reflect.DeepEqual(p.Studios, want) {
t.Errorf("Studios = %#v, want %#v", p.Studios, want)
}
if want := []string{"United States", "Hong Kong"}; !reflect.DeepEqual(p.Countries, want) {
t.Errorf("Countries = %#v, want %#v", p.Countries, want)
}
if want := []string{"dystopia", "neo-noir"}; !reflect.DeepEqual(p.Keywords, want) {
t.Errorf("Keywords = %#v, want %#v", p.Keywords, want)
}
if p.RatingIMDB != 8.1 {
t.Errorf("RatingIMDB = %v, want 8.1", p.RatingIMDB)
}
if p.RatingTMDB != 7.9 {
t.Errorf("RatingTMDB = %v, want 7.9", p.RatingTMDB)
}
if p.RatingRTCritic != 89 {
t.Errorf("RatingRTCritic = %v, want 89", p.RatingRTCritic)
}
if p.RatingRTAudience != 91 {
t.Errorf("RatingRTAudience = %v, want 91", p.RatingRTAudience)
}
wantPeople := []models.ItemPerson{
{Person: models.Person{Name: "Harrison Ford"}, Kind: models.PersonKindActor, Character: "Rick Deckard", SortOrder: 0},
{Person: models.Person{Name: "Rutger Hauer"}, Kind: models.PersonKindActor, Character: "Roy Batty", SortOrder: 1},
{Person: models.Person{Name: "Ridley Scott"}, Kind: models.PersonKindDirector, SortOrder: 0},
{Person: models.Person{Name: "Hampton Fancher"}, Kind: models.PersonKindWriter, SortOrder: 0},
{Person: models.Person{Name: "David Peoples"}, Kind: models.PersonKindWriter, SortOrder: 1},
}
if !reflect.DeepEqual(p.People, wantPeople) {
t.Errorf("People = %#v, want %#v", p.People, wantPeople)
}
if p.ImdbID != "tt0083658" || p.TmdbID != "78" {
t.Errorf("ids = imdb %q tmdb %q", p.ImdbID, p.TmdbID)
}
}
func TestParseNFOData_FullTVShow(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(fullTVShowNFO))
if err != nil {
t.Fatalf("parseNFOData() error = %v", err)
}
if p.Type != "series" {
t.Errorf("Type = %q, want series", p.Type)
}
if p.FirstAirDate != "2015-06-24" {
t.Errorf("FirstAirDate = %q, want 2015-06-24", p.FirstAirDate)
}
if p.ReleaseDate != "" {
t.Errorf("ReleaseDate = %q, want empty on a series", p.ReleaseDate)
}
if p.Year != 2015 {
t.Errorf("Year = %d, want 2015 (derived from premiered)", p.Year)
}
if p.ContentRating != "TV-MA" {
t.Errorf("ContentRating = %q, want TV-MA", p.ContentRating)
}
if want := []string{"Drama", "Crime"}; !reflect.DeepEqual(p.Genres, want) {
t.Errorf("Genres = %#v, want %#v", p.Genres, want)
}
if p.RatingIMDB != 8.5 {
t.Errorf("RatingIMDB = %v, want 8.5", p.RatingIMDB)
}
if len(p.People) != 1 || p.People[0].Name != "Rami Malek" || p.People[0].Character != "Elliot Alderson" {
t.Errorf("People = %#v", p.People)
}
if p.TvdbID != "289590" {
t.Errorf("TvdbID = %q", p.TvdbID)
}
}
func TestParseNFOData_TVShowAiredFallsBackToFirstAirDate(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(`<tvshow><title>Show</title><aired>2010-09-01</aired></tvshow>`))
if err != nil {
t.Fatalf("parseNFOData() error = %v", err)
}
if p.FirstAirDate != "2010-09-01" {
t.Errorf("FirstAirDate = %q, want 2010-09-01 (from <aired>)", p.FirstAirDate)
}
}
func TestParseNFOData_MovieReleaseDateFallback(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(`<movie><title>M</title><releasedate>1999-03-31</releasedate></movie>`))
if err != nil {
t.Fatalf("parseNFOData() error = %v", err)
}
if p.ReleaseDate != "1999-03-31" {
t.Errorf("ReleaseDate = %q, want 1999-03-31 (from <releasedate>)", p.ReleaseDate)
}
}
func TestParseNFOData_TableCases(t *testing.T) {
t.Parallel()
tests := []struct {
name string
data string
wantErr bool
check func(t *testing.T, p *parsedNFO)
}{
{
name: "BOM prefix retained",
data: "\xEF\xBB\xBF<movie><title>BOM Movie</title></movie>",
check: func(t *testing.T, p *parsedNFO) {
if p.Title != "BOM Movie" {
t.Errorf("Title = %q", p.Title)
}
},
},
{
name: "malformed XML is an error (no NFO)",
data: `<movie><title>Broken`,
wantErr: true,
},
{
name: "unsupported root element",
data: `<musicvideo><title>x</title></musicvideo>`,
wantErr: true,
},
{
name: "unknown elements ignored",
data: `<movie><title>X</title><bogusfield>zzz</bogusfield><another attr="1"/></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.Title != "X" {
t.Errorf("Title = %q", p.Title)
}
},
},
{
name: "empty collections stay nil (no placeholder emissions)",
data: `<movie><title>Bare</title><genre></genre><studio> </studio><tag/></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.Genres != nil {
t.Errorf("Genres = %#v, want nil", p.Genres)
}
if p.Studios != nil {
t.Errorf("Studios = %#v, want nil", p.Studios)
}
if p.Countries != nil {
t.Errorf("Countries = %#v, want nil", p.Countries)
}
if p.Keywords != nil {
t.Errorf("Keywords = %#v, want nil", p.Keywords)
}
if p.People != nil {
t.Errorf("People = %#v, want nil", p.People)
}
},
},
{
name: "non-numeric runtime ignored",
data: `<movie><title>X</title><runtime>two hours</runtime></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.Runtime != 0 {
t.Errorf("Runtime = %d, want 0", p.Runtime)
}
},
},
{
name: "runtime with surrounding whitespace",
data: `<movie><title>X</title><runtime> 95 </runtime></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.Runtime != 95 {
t.Errorf("Runtime = %d, want 95", p.Runtime)
}
},
},
{
name: "non-date premiered ignored",
data: `<movie><title>X</title><premiered>next summer</premiered></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.ReleaseDate != "" {
t.Errorf("ReleaseDate = %q, want empty", p.ReleaseDate)
}
},
},
{
name: "explicit year wins over premiered-derived year",
data: `<movie><title>X</title><year>1981</year><premiered>1982-06-25</premiered></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.Year != 1981 {
t.Errorf("Year = %d, want 1981", p.Year)
}
},
},
{
name: "legacy bare rating fills the imdb slot",
data: `<movie><title>X</title><rating>7.4</rating></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.RatingIMDB != 7.4 {
t.Errorf("RatingIMDB = %v, want 7.4", p.RatingIMDB)
}
},
},
{
name: "named imdb rating wins over legacy bare rating",
data: `<movie><title>X</title><rating>6.0</rating><ratings><rating name="imdb" max="10"><value>8.2</value></rating></ratings></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.RatingIMDB != 8.2 {
t.Errorf("RatingIMDB = %v, want 8.2", p.RatingIMDB)
}
},
},
{
name: "ratings normalized to their slot scale",
data: `<movie><title>X</title><ratings>` +
`<rating name="imdb" max="5"><value>4.0</value></rating>` +
`<rating name="tomatometerallcritics" max="10"><value>9</value></rating>` +
`</ratings></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.RatingIMDB != 8.0 {
t.Errorf("RatingIMDB = %v, want 8.0 (scaled from max=5)", p.RatingIMDB)
}
if p.RatingRTCritic != 90 {
t.Errorf("RatingRTCritic = %v, want 90 (scaled from max=10)", p.RatingRTCritic)
}
},
},
{
name: "userrating alone emits no ratings",
data: `<movie><title>X</title><userrating>9</userrating></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.RatingIMDB != 0 || p.RatingTMDB != 0 || p.RatingRTCritic != 0 || p.RatingRTAudience != 0 {
t.Errorf("ratings = %v/%v/%v/%v, want all zero", p.RatingIMDB, p.RatingTMDB, p.RatingRTCritic, p.RatingRTAudience)
}
},
},
{
name: "actor without order gets file order",
data: `<movie><title>X</title>` +
`<actor><name>A One</name><role>R1</role></actor>` +
`<actor><name>B Two</name></actor></movie>`,
check: func(t *testing.T, p *parsedNFO) {
want := []models.ItemPerson{
{Person: models.Person{Name: "A One"}, Kind: models.PersonKindActor, Character: "R1", SortOrder: 0},
{Person: models.Person{Name: "B Two"}, Kind: models.PersonKindActor, SortOrder: 1},
}
if !reflect.DeepEqual(p.People, want) {
t.Errorf("People = %#v, want %#v", p.People, want)
}
},
},
{
name: "actor without a name is dropped",
data: `<movie><title>X</title><actor><role>Ghost</role></actor></movie>`,
check: func(t *testing.T, p *parsedNFO) {
if p.People != nil {
t.Errorf("People = %#v, want nil", p.People)
}
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(tc.data))
if tc.wantErr {
if err == nil {
t.Fatalf("parseNFOData() error = nil, want error")
}
return
}
if err != nil {
t.Fatalf("parseNFOData() error = %v", err)
}
if tc.check != nil {
tc.check(t, p)
}
})
}
}
internal/metadata/nfo/provider.go
+79 -39
View File
@@ -16,24 +16,38 @@ func NewProvider() *Provider { return &Provider{} }
func (p *Provider) Slug() string { return "nfo" }
func (p *Provider) Name() string { return "NFO Files" }
func (p *Provider) ForTypes() []string { return []string{"movie", "series"} }
func (p *Provider) ForTypes() []string { return []string{typeMovie, typeSeries} }
// IdentityHints implements metadata.IdentityHintProvider: the external IDs a
// curated NFO declares (<uniqueid> tmdb/imdb/tvdb) are trusted identity hints
// that anchor Phase-1 candidate selection. A title-only NFO contributes no
// hints; its title participates only as a defanged search candidate.
func (p *Provider) IdentityHints(_ context.Context, query metadata.SearchQuery) map[string]string {
parsed := findNFOForQuery(query)
if parsed == nil {
return nil
}
hints := make(map[string]string)
if parsed.TmdbID != "" {
hints["tmdb"] = parsed.TmdbID
}
if parsed.ImdbID != "" {
hints["imdb"] = parsed.ImdbID
}
if parsed.TvdbID != "" {
hints["tvdb"] = parsed.TvdbID
}
if len(hints) == 0 {
return nil
}
return hints
}
// Search extracts external IDs from NFO for matching.
func (p *Provider) Search(ctx context.Context, query metadata.SearchQuery) ([]metadata.SearchResult, error) {
nfoPath := findNFOForQuery(query)
if nfoPath == "" {
return nil, nil
}
data, err := os.ReadFile(nfoPath)
if err != nil {
return nil, nil // NFO read errors are not fatal
}
parsed, err := parseNFOData(data)
if err != nil {
return nil, nil // NFO parse errors are not fatal
}
if query.ContentType != "" && parsed.Type != "" && parsed.Type != query.ContentType {
return nil, nil
parsed := findNFOForQuery(query)
if parsed == nil {
return nil, nil // missing/unreadable/mismatched NFOs are not fatal
}
ids := make(map[string]string)
if parsed.TmdbID != "" {
@@ -56,25 +70,36 @@ func (p *Provider) Search(ctx context.Context, query metadata.SearchQuery) ([]me
}}, nil
}
// GetMetadata parses full metadata from NFO file.
// GetMetadata parses full metadata from an NFO file. Like Search, it carries
// the ContentType guard (enforced inside findNFO): a tvshow.nfo next to a
// movie file must not inject series data into a movie item at top priority.
func (p *Provider) GetMetadata(ctx context.Context, req metadata.MetadataRequest) (*metadata.MetadataResult, error) {
nfoPath := findNFOForRequest(req)
if nfoPath == "" {
return &metadata.MetadataResult{}, nil
}
data, err := os.ReadFile(nfoPath)
if err != nil {
return &metadata.MetadataResult{}, nil
}
parsed, err := parseNFOData(data)
if err != nil {
parsed := findNFOForRequest(req)
if parsed == nil {
return &metadata.MetadataResult{}, nil
}
result := &metadata.MetadataResult{
HasMetadata: true,
Title: parsed.Title,
Year: parsed.Year,
Overview: parsed.Overview,
HasMetadata: true,
Title: parsed.Title,
OriginalTitle: parsed.OriginalTitle,
Tagline: parsed.Tagline,
Year: parsed.Year,
Overview: parsed.Overview,
Runtime: parsed.Runtime,
ReleaseDate: parsed.ReleaseDate,
FirstAirDate: parsed.FirstAirDate,
ContentRating: parsed.ContentRating,
Genres: parsed.Genres,
Studios: parsed.Studios,
Countries: parsed.Countries,
Keywords: parsed.Keywords,
Ratings: metadata.Ratings{
IMDB: parsed.RatingIMDB,
TMDB: parsed.RatingTMDB,
RTCritic: parsed.RatingRTCritic,
RTAudience: parsed.RatingRTAudience,
},
People: parsed.People,
ProviderIDs: make(map[string]string),
}
if parsed.TmdbID != "" {
@@ -89,7 +114,7 @@ func (p *Provider) GetMetadata(ctx context.Context, req metadata.MetadataRequest
return result, nil
}
func findNFOForQuery(query metadata.SearchQuery) string {
func findNFOForQuery(query metadata.SearchQuery) *parsedNFO {
candidatePaths := candidateSidecarPaths(
query.FilePath,
query.RepresentativeFilePath,
@@ -97,17 +122,19 @@ func findNFOForQuery(query metadata.SearchQuery) string {
query.PrimarySidecarSearchPaths,
query.ProviderIDs["_filepath"],
)
return findNFO(candidatePaths)
_, parsed := findNFO(candidatePaths, query.ContentType)
return parsed
}
func findNFOForRequest(req metadata.MetadataRequest) string {
func findNFOForRequest(req metadata.MetadataRequest) *parsedNFO {
candidatePaths := candidateSidecarPaths(
req.FilePath,
req.RepresentativeFilePath,
req.AllGroupFilePaths,
req.PrimarySidecarSearchPaths,
)
return findNFO(candidatePaths)
_, parsed := findNFO(candidatePaths, req.ContentType)
return parsed
}
func candidateSidecarPaths(primary string, representative string, groupFiles []string, searchPaths []string, extras ...string) []string {
@@ -142,19 +169,32 @@ func compactNFOPaths(paths []string) []string {
// findNFO locates the best candidate NFO file across media-file and directory
// search paths. File paths check legacy directory-level sidecars first, then a
// basename-matched sidecar in the same directory.
func findNFO(paths []string) string {
// basename-matched sidecar in the same directory. A candidate that exists but
// fails to parse, or whose root type mismatches contentType, falls through to
// the next candidate — a stray movie.nfo in a series root must not shadow
// tvshow.nfo, and a tvshow.nfo beside a movie file must not inject series
// data. Returns the winning path and its parsed contents, or ("", nil).
func findNFO(paths []string, contentType string) (string, *parsedNFO) {
candidates := make([]string, 0, len(paths)*3)
for _, path := range paths {
candidates = append(candidates, nfoCandidatesForPath(path)...)
}
candidates = compactNFOPaths(candidates)
for _, c := range candidates {
if _, err := os.Stat(c); err == nil {
return c
data, err := os.ReadFile(c)
if err != nil {
continue
}
parsed, err := parseNFOData(data)
if err != nil {
continue
}
if contentType != "" && parsed.Type != "" && parsed.Type != contentType {
continue
}
return c, parsed
}
return ""
return "", nil
}
func nfoCandidatesForPath(path string) []string {
internal/metadata/nfo/provider_test.go
+204 -10
View File
@@ -1,10 +1,13 @@
package nfo
import (
"context"
"os"
"path/filepath"
"slices"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
)
func TestFindNFO_FilePathFallsBackToDirectoryLevelSidecar(t *testing.T) {
@@ -15,12 +18,12 @@ func TestFindNFO_FilePathFallsBackToDirectoryLevelSidecar(t *testing.T) {
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte("dir"), 0o644); err != nil {
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte("<movie><title>Dir</title></movie>"), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
want := filepath.Join(dir, "movie.nfo")
if got := findNFO([]string{filePath}); got != want {
if got, _ := findNFO([]string{filePath}, ""); got != want {
t.Fatalf("findNFO(file path) = %q, want %q", got, want)
}
}
@@ -30,11 +33,11 @@ func TestFindNFO_DirectoryPathUsesDirectoryLevelSidecar(t *testing.T) {
dir := t.TempDir()
want := filepath.Join(dir, "movie.nfo")
if err := os.WriteFile(want, []byte("dir"), 0o644); err != nil {
if err := os.WriteFile(want, []byte("<movie><title>Dir</title></movie>"), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
if got := findNFO([]string{dir}); got != want {
if got, _ := findNFO([]string{dir}, ""); got != want {
t.Fatalf("findNFO(directory path) = %q, want %q", got, want)
}
}
@@ -48,11 +51,11 @@ func TestFindNFO_FilePathUsesBasenameMatchedNFO(t *testing.T) {
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(want, []byte("file"), 0o644); err != nil {
if err := os.WriteFile(want, []byte("<movie><title>File</title></movie>"), 0o644); err != nil {
t.Fatalf("WriteFile(file.nfo) error = %v", err)
}
if got := findNFO([]string{filePath}); got != want {
if got, _ := findNFO([]string{filePath}, ""); got != want {
t.Fatalf("findNFO(file path) = %q, want %q", got, want)
}
}
@@ -66,11 +69,11 @@ func TestFindNFO_ExtensionlessFilePathUsesDirectoryLevelSidecar(t *testing.T) {
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(want, []byte("dir"), 0o644); err != nil {
if err := os.WriteFile(want, []byte("<movie><title>Dir</title></movie>"), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
if got := findNFO([]string{filePath}); got != want {
if got, _ := findNFO([]string{filePath}, ""); got != want {
t.Fatalf("findNFO(extensionless file path) = %q, want %q", got, want)
}
}
@@ -84,11 +87,11 @@ func TestFindNFO_ExtensionlessFilePathUsesBasenameMatchedNFO(t *testing.T) {
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(want, []byte("file"), 0o644); err != nil {
if err := os.WriteFile(want, []byte("<movie><title>File</title></movie>"), 0o644); err != nil {
t.Fatalf("WriteFile(file.nfo) error = %v", err)
}
if got := findNFO([]string{filePath}); got != want {
if got, _ := findNFO([]string{filePath}, ""); got != want {
t.Fatalf("findNFO(extensionless file path) = %q, want %q", got, want)
}
}
@@ -108,3 +111,194 @@ func TestNFOCandidatesForMissingDottedDirectoryPathIncludeDirectoryLevelSidecars
t.Fatalf("nfoCandidatesForPath(%q) missing %q in %#v", path, wantTV, got)
}
}
// A tvshow.nfo sitting next to a movie file must not inject series data into
// a movie item at top priority: GetMetadata carries the same ContentType
// guard Search has.
func TestGetMetadata_TVShowNFOBesideMovieFileDoesNotInject(t *testing.T) {
t.Parallel()
dir := t.TempDir()
filePath := filepath.Join(dir, "Home Movie (2021).mkv")
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
nfoXML := `<tvshow><title>Some Series</title><uniqueid type="tvdb">123</uniqueid></tvshow>`
if err := os.WriteFile(filepath.Join(dir, "tvshow.nfo"), []byte(nfoXML), 0o644); err != nil {
t.Fatalf("WriteFile(tvshow.nfo) error = %v", err)
}
p := NewProvider()
result, err := p.GetMetadata(context.Background(), metadata.MetadataRequest{
ContentType: "movie",
FilePath: filePath,
})
if err != nil {
t.Fatalf("GetMetadata() error = %v", err)
}
if result.HasMetadata {
t.Fatalf("GetMetadata() = %#v, want no metadata (series NFO must not inject into a movie)", result)
}
}
// A stray movie.nfo in a series root must not shadow tvshow.nfo: findNFO
// falls through to the next candidate on type mismatch.
func TestFindNFO_StrayMovieNFOInSeriesRootFallsThroughToTVShow(t *testing.T) {
t.Parallel()
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte(`<movie><title>Stray</title></movie>`), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
want := filepath.Join(dir, "tvshow.nfo")
if err := os.WriteFile(want, []byte(`<tvshow><title>The Show</title></tvshow>`), 0o644); err != nil {
t.Fatalf("WriteFile(tvshow.nfo) error = %v", err)
}
got, parsed := findNFO([]string{dir}, "series")
if got != want {
t.Fatalf("findNFO(series root) = %q, want %q", got, want)
}
if parsed == nil || parsed.Title != "The Show" {
t.Fatalf("parsed = %#v, want tvshow content", parsed)
}
}
// A candidate that exists but fails to parse falls through to the next
// candidate instead of masking a valid sidecar.
func TestFindNFO_ParseFailureFallsThroughToNextCandidate(t *testing.T) {
t.Parallel()
dir := t.TempDir()
filePath := filepath.Join(dir, "Blade Runner (1982).mkv")
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte("not xml at all"), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
want := filepath.Join(dir, "Blade Runner (1982).nfo")
if err := os.WriteFile(want, []byte(`<movie><title>Blade Runner</title></movie>`), 0o644); err != nil {
t.Fatalf("WriteFile(basename nfo) error = %v", err)
}
got, parsed := findNFO([]string{filePath}, "movie")
if got != want {
t.Fatalf("findNFO(parse-failure fallthrough) = %q, want %q", got, want)
}
if parsed == nil || parsed.Title != "Blade Runner" {
t.Fatalf("parsed = %#v, want valid movie content", parsed)
}
}
// GetMetadata surfaces the full Phase-B field set from a curated movie NFO.
func TestGetMetadata_FullMovieFieldSet(t *testing.T) {
t.Parallel()
dir := t.TempDir()
filePath := filepath.Join(dir, "Blade Runner (1982).mkv")
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte(fullMovieNFO), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
p := NewProvider()
result, err := p.GetMetadata(context.Background(), metadata.MetadataRequest{
ContentType: "movie",
FilePath: filePath,
})
if err != nil {
t.Fatalf("GetMetadata() error = %v", err)
}
if !result.HasMetadata {
t.Fatal("HasMetadata = false, want true")
}
if result.OriginalTitle != "Blade Runner: The Original" {
t.Errorf("OriginalTitle = %q", result.OriginalTitle)
}
if result.Tagline != "Man has made his match" {
t.Errorf("Tagline = %q", result.Tagline)
}
if result.Runtime != 117 {
t.Errorf("Runtime = %d", result.Runtime)
}
if result.ReleaseDate != "1982-06-25" {
t.Errorf("ReleaseDate = %q", result.ReleaseDate)
}
if result.ContentRating != "R" {
t.Errorf("ContentRating = %q", result.ContentRating)
}
if len(result.Genres) != 2 || len(result.Studios) != 2 || len(result.Countries) != 2 || len(result.Keywords) != 2 {
t.Errorf("collections = genres %d studios %d countries %d keywords %d, want 2 each",
len(result.Genres), len(result.Studios), len(result.Countries), len(result.Keywords))
}
if result.Ratings.IMDB != 8.1 || result.Ratings.TMDB != 7.9 || result.Ratings.RTCritic != 89 || result.Ratings.RTAudience != 91 {
t.Errorf("Ratings = %#v", result.Ratings)
}
if len(result.People) != 5 {
t.Errorf("People len = %d, want 5", len(result.People))
}
if result.FirstAirDate != "" {
t.Errorf("FirstAirDate = %q, want empty for a movie", result.FirstAirDate)
}
}
// A minimal NFO must not emit placeholder collections: empty slices stay nil
// so MergeFillEmpty's early-return lets remote providers fill them.
func TestGetMetadata_MinimalNFOEmitsNoPlaceholders(t *testing.T) {
t.Parallel()
dir := t.TempDir()
filePath := filepath.Join(dir, "Home Movie (2021).mkv")
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
t.Fatalf("WriteFile(file) error = %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "movie.nfo"), []byte(`<movie><title>Home Movie</title></movie>`), 0o644); err != nil {
t.Fatalf("WriteFile(movie.nfo) error = %v", err)
}
p := NewProvider()
result, err := p.GetMetadata(context.Background(), metadata.MetadataRequest{
ContentType: "movie",
FilePath: filePath,
})
if err != nil {
t.Fatalf("GetMetadata() error = %v", err)
}
if result.Genres != nil || result.Studios != nil || result.Countries != nil || result.Keywords != nil || result.People != nil {
t.Errorf("placeholder collections emitted: %#v", result)
}
if result.Runtime != 0 || result.ReleaseDate != "" || result.ContentRating != "" {
t.Errorf("placeholder scalars emitted: %#v", result)
}
}
// The series GetMetadata path maps aired/premiered onto FirstAirDate.
func TestGetMetadata_SeriesFirstAirDate(t *testing.T) {
t.Parallel()
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "tvshow.nfo"), []byte(fullTVShowNFO), 0o644); err != nil {
t.Fatalf("WriteFile(tvshow.nfo) error = %v", err)
}
p := NewProvider()
result, err := p.GetMetadata(context.Background(), metadata.MetadataRequest{
ContentType: "series",
PrimarySidecarSearchPaths: []string{dir},
})
if err != nil {
t.Fatalf("GetMetadata() error = %v", err)
}
if !result.HasMetadata {
t.Fatal("HasMetadata = false, want true")
}
if result.FirstAirDate != "2015-06-24" {
t.Errorf("FirstAirDate = %q, want 2015-06-24", result.FirstAirDate)
}
if result.ReleaseDate != "" {
t.Errorf("ReleaseDate = %q, want empty for a series", result.ReleaseDate)
}
}
internal/metadata/nfo/register.go
+14
View File
@@ -0,0 +1,14 @@
package nfo
import (
"github.com/Silo-Server/silo-server/internal/metadata"
)
// The NFO provider is a built-in host provider: it is represented in the
// database by the reserved 'silo.builtin' installation's 'nfo' capability and
// resolved in-process through the builtin registry. Registration lives here
// (not in the metadata package) because this package implements the metadata
// package's interfaces; cmd/silo blank-imports this package to activate it.
func init() {
metadata.RegisterBuiltinProvider("nfo", func() metadata.Provider { return NewProvider() })
}
internal/metadata/nfo/series_depth.go
+198
View File
@@ -0,0 +1,198 @@
package nfo
import (
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"sort"
"strings"
"github.com/Silo-Server/silo-server/internal/metadata"
)
// Provider implements metadata.EpisodeProvider for its content types using
// the request plumbing added in Phase D: per-season directory paths and
// per-episode media file paths. NFO files never create structure — season
// and episode numbers come from directory/filename parsing upstream and the
// numbers declared inside an NFO are advisory only.
// GetSeasons reads season.nfo sidecars and season posters for each
// directory-derived season. A season with neither a season.nfo nor a poster
// contributes nothing, leaving remote providers (or fallback synthesis) in
// charge.
func (p *Provider) GetSeasons(ctx context.Context, req metadata.SeasonsRequest) ([]metadata.SeasonResult, error) {
if req.ContentType != "" && req.ContentType != typeSeries {
return nil, nil
}
if len(req.SeasonDirectoryPaths) == 0 {
return nil, nil
}
seriesRoots := compactNFOPaths(req.SeriesRootPaths)
seasonNumbers := make([]int, 0, len(req.SeasonDirectoryPaths))
for number := range req.SeasonDirectoryPaths {
seasonNumbers = append(seasonNumbers, number)
}
sort.Ints(seasonNumbers)
var seasons []metadata.SeasonResult
for _, number := range seasonNumbers {
dirs := compactNFOPaths(req.SeasonDirectoryPaths[number])
parsed := parseSeasonSidecar(ctx, dirs, number)
poster := findSeasonPoster(dirs, seriesRoots, number)
if parsed == nil && poster == "" {
continue
}
season := metadata.SeasonResult{SeasonNumber: number}
if parsed != nil {
season.Title = parsed.Title
season.Overview = parsed.Overview
}
if poster != "" {
season.PosterPath = "file://" + poster
}
seasons = append(seasons, season)
}
return seasons, nil
}
// parseSeasonSidecar loads the first parseable season.nfo across the season's
// candidate directories. A declared <seasonnumber> that disagrees with the
// directory-derived number is advisory: naming owns structure, so the
// directory number wins with a warning.
func parseSeasonSidecar(ctx context.Context, seasonDirs []string, seasonNumber int) *parsedNFO {
for _, dir := range seasonDirs {
path := filepath.Join(dir, "season.nfo")
data, err := os.ReadFile(path)
if err != nil {
continue
}
parsed, err := parseNFOData(data)
if err != nil || parsed.Type != typeSeason {
continue
}
if parsed.SeasonSet && parsed.Season != seasonNumber {
slog.WarnContext(ctx, "nfo: season.nfo seasonnumber disagrees with directory; preferring directory number",
"component", "metadata", "path", path,
"nfo_season", parsed.Season, "directory_season", seasonNumber)
}
return parsed
}
return nil
}
// findSeasonPoster locates season artwork: poster/folder/cover inside the
// season directory first, then the Kodi-style seasonNN-poster form in the
// series root.
func findSeasonPoster(seasonDirs []string, seriesRoots []string, seasonNumber int) string {
for _, name := range []string{"poster", "folder", "cover"} {
for _, dir := range seasonDirs {
if path := findLocalArtworkFile(dir, name); path != "" {
return path
}
}
}
rootNames := []string{fmt.Sprintf("season%02d-poster", seasonNumber)}
if seasonNumber == 0 {
rootNames = append(rootNames, "season-specials-poster")
}
for _, name := range rootNames {
for _, dir := range seriesRoots {
if path := findLocalArtworkFile(dir, name); path != "" {
return path
}
}
}
return ""
}
// GetEpisodes reads <basename>.nfo and <basename>-thumb sidecars for each
// episode media file of the requested season. Episodes with neither
// contribute nothing (synthesized fallback stays in charge of them).
func (p *Provider) GetEpisodes(ctx context.Context, req metadata.EpisodesRequest) ([]metadata.EpisodeResult, error) {
if len(req.EpisodeFilePaths) == 0 {
return nil, nil
}
episodeNumbers := make([]int, 0, len(req.EpisodeFilePaths))
for number := range req.EpisodeFilePaths {
episodeNumbers = append(episodeNumbers, number)
}
sort.Ints(episodeNumbers)
var episodes []metadata.EpisodeResult
for _, number := range episodeNumbers {
paths := compactNFOPaths(req.EpisodeFilePaths[number])
parsed, thumb := findEpisodeSidecars(ctx, paths, req.SeasonNumber, number)
if parsed == nil && thumb == "" {
continue
}
episode := metadata.EpisodeResult{
SeasonNumber: req.SeasonNumber,
EpisodeNumber: number,
}
if parsed != nil {
episode.Title = parsed.Title
episode.Overview = parsed.Overview
episode.AirDate = parsed.FirstAirDate
episode.Runtime = parsed.Runtime
episode.Ratings = metadata.Ratings{
IMDB: parsed.RatingIMDB,
TMDB: parsed.RatingTMDB,
RTCritic: parsed.RatingRTCritic,
RTAudience: parsed.RatingRTAudience,
}
}
if thumb != "" {
episode.StillPath = "file://" + thumb
}
episodes = append(episodes, episode)
}
return episodes, nil
}
// findEpisodeSidecars locates the episode's <basename>.nfo and
// <basename>-thumb image across its media file paths. Numbers declared in
// the NFO are checked against the filename-derived SxxEyy, which wins on
// conflict (naming owns structure).
func findEpisodeSidecars(ctx context.Context, mediaPaths []string, seasonNumber, episodeNumber int) (*parsedNFO, string) {
var parsed *parsedNFO
thumb := ""
for _, mediaPath := range mediaPaths {
dir := filepath.Dir(mediaPath)
base := strings.TrimSuffix(filepath.Base(mediaPath), filepath.Ext(mediaPath))
if parsed == nil {
parsed = parseEpisodeSidecar(ctx, filepath.Join(dir, base+".nfo"), seasonNumber, episodeNumber)
}
if thumb == "" {
thumb = findLocalArtworkFile(dir, base+"-thumb")
}
if parsed != nil && thumb != "" {
break
}
}
return parsed, thumb
}
func parseEpisodeSidecar(ctx context.Context, path string, seasonNumber, episodeNumber int) *parsedNFO {
data, err := os.ReadFile(path)
if err != nil {
return nil
}
parsed, err := parseNFOData(data)
if err != nil || parsed.Type != typeEpisode {
return nil
}
if parsed.MultiEpisode {
slog.WarnContext(ctx, "nfo: multi-episode NFO documents are not supported; using the first <episodedetails> block",
"component", "metadata", "path", path)
}
if (parsed.SeasonSet && parsed.Season != seasonNumber) ||
(parsed.EpisodeSet && parsed.Episode != episodeNumber) {
slog.WarnContext(ctx, "nfo: episode NFO numbers disagree with filename; preferring filename numbers",
"component", "metadata", "path", path,
"nfo_season", parsed.Season, "nfo_episode", parsed.Episode,
"file_season", seasonNumber, "file_episode", episodeNumber)
}
return parsed
}
internal/metadata/nfo/series_depth_test.go
+376
View File
@@ -0,0 +1,376 @@
package nfo
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
)
// ---------------------------------------------------------------------------
// Parser tables: <season>
// ---------------------------------------------------------------------------
func TestParseSeasonNFO(t *testing.T) {
t.Parallel()
cases := []struct {
name string
xml string
wantTitle string
wantOverview string
wantSeason int
wantSeasonSet bool
}{
{
name: "full season",
xml: `<season><title>Course A: Classic</title><plot>The classic 90-day schedule.</plot><seasonnumber>1</seasonnumber></season>`,
wantTitle: "Course A: Classic",
wantOverview: "The classic 90-day schedule.",
wantSeason: 1,
wantSeasonSet: true,
},
{
name: "no seasonnumber",
xml: `<season><title>Course B</title></season>`,
wantTitle: "Course B",
wantSeasonSet: false,
},
{
name: "specials season zero",
xml: `<season><title>Bonus Workouts</title><seasonnumber>0</seasonnumber></season>`,
wantTitle: "Bonus Workouts",
wantSeason: 0,
wantSeasonSet: true,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(tc.xml))
if err != nil {
t.Fatalf("parseNFOData: %v", err)
}
if p.Type != "season" {
t.Errorf("Type = %q, want season", p.Type)
}
if p.Title != tc.wantTitle {
t.Errorf("Title = %q, want %q", p.Title, tc.wantTitle)
}
if p.Overview != tc.wantOverview {
t.Errorf("Overview = %q, want %q", p.Overview, tc.wantOverview)
}
if p.SeasonSet != tc.wantSeasonSet {
t.Errorf("SeasonSet = %v, want %v", p.SeasonSet, tc.wantSeasonSet)
}
if tc.wantSeasonSet && p.Season != tc.wantSeason {
t.Errorf("Season = %d, want %d", p.Season, tc.wantSeason)
}
})
}
}
// ---------------------------------------------------------------------------
// Parser tables: extended <episodedetails>
// ---------------------------------------------------------------------------
func TestParseEpisodeNFO_ExtendedFields(t *testing.T) {
t.Parallel()
xml := `<episodedetails>
<title>Chest and Back</title>
<plot>Push-ups and pull-ups.</plot>
<aired>2004-01-01</aired>
<season>1</season>
<episode>1</episode>
<runtime>53</runtime>
<ratings><rating name="imdb" max="10"><value>8.4</value></rating></ratings>
</episodedetails>`
p, err := parseNFOData([]byte(xml))
if err != nil {
t.Fatalf("parseNFOData: %v", err)
}
if p.Type != "episode" {
t.Errorf("Type = %q, want episode", p.Type)
}
if p.Title != "Chest and Back" || p.Overview != "Push-ups and pull-ups." {
t.Errorf("title/plot = %q/%q", p.Title, p.Overview)
}
if p.FirstAirDate != "2004-01-01" {
t.Errorf("FirstAirDate = %q, want 2004-01-01", p.FirstAirDate)
}
if !p.SeasonSet || p.Season != 1 || !p.EpisodeSet || p.Episode != 1 {
t.Errorf("numbers = season(%v,%d) episode(%v,%d), want set 1/1", p.SeasonSet, p.Season, p.EpisodeSet, p.Episode)
}
if p.Runtime != 53 {
t.Errorf("Runtime = %d, want 53", p.Runtime)
}
if p.RatingIMDB != 8.4 {
t.Errorf("RatingIMDB = %v, want 8.4", p.RatingIMDB)
}
if p.MultiEpisode {
t.Error("MultiEpisode = true for single-episode document")
}
}
func TestParseEpisodeNFO_LegacyRatingAndNoNumbers(t *testing.T) {
t.Parallel()
p, err := parseNFOData([]byte(`<episodedetails><title>Core</title><rating>7.5</rating></episodedetails>`))
if err != nil {
t.Fatalf("parseNFOData: %v", err)
}
if p.SeasonSet || p.EpisodeSet {
t.Errorf("SeasonSet/EpisodeSet = %v/%v, want false (numbers absent)", p.SeasonSet, p.EpisodeSet)
}
if p.RatingIMDB != 7.5 {
t.Errorf("legacy RatingIMDB = %v, want 7.5", p.RatingIMDB)
}
}
// Multi-<episodedetails> documents (multi-episode files) are out of scope in
// v1: the first block wins and the parser flags the document so the provider
// can warn.
func TestParseEpisodeNFO_MultiEpisodeDocumentTakesFirst(t *testing.T) {
t.Parallel()
xml := `<episodedetails><title>First Half</title><episode>1</episode></episodedetails>
<episodedetails><title>Second Half</title><episode>2</episode></episodedetails>`
p, err := parseNFOData([]byte(xml))
if err != nil {
t.Fatalf("parseNFOData: %v", err)
}
if p.Title != "First Half" {
t.Errorf("Title = %q, want First Half (take first)", p.Title)
}
if !p.MultiEpisode {
t.Error("MultiEpisode = false, want true")
}
}
// ---------------------------------------------------------------------------
// Provider: GetSeasons
// ---------------------------------------------------------------------------
func writeTestFile(t *testing.T, path, contents string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatalf("MkdirAll(%s): %v", path, err)
}
if err := os.WriteFile(path, []byte(contents), 0o644); err != nil {
t.Fatalf("WriteFile(%s): %v", path, err)
}
}
func TestGetSeasons_SeasonNFOAndPoster(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
s2 := filepath.Join(root, "Season 02")
writeTestFile(t, filepath.Join(s1, "season.nfo"),
`<season><title>Course A: Classic</title><plot>Classic plan.</plot><seasonnumber>1</seasonnumber></season>`)
writeTestFile(t, filepath.Join(s1, "poster.jpg"), "img")
writeTestFile(t, filepath.Join(s2, "season.nfo"), `<season><title>Course B</title></season>`)
// Season 2 poster lives at the series root using the seasonNN-poster form.
writeTestFile(t, filepath.Join(root, "season02-poster.jpg"), "img")
p := NewProvider()
seasons, err := p.GetSeasons(context.Background(), metadata.SeasonsRequest{
ContentType: "series",
SeriesRootPaths: []string{root},
SeasonDirectoryPaths: map[int][]string{
1: {s1},
2: {s2},
},
})
if err != nil {
t.Fatalf("GetSeasons: %v", err)
}
if len(seasons) != 2 {
t.Fatalf("len(seasons) = %d, want 2 (%#v)", len(seasons), seasons)
}
if seasons[0].SeasonNumber != 1 || seasons[0].Title != "Course A: Classic" || seasons[0].Overview != "Classic plan." {
t.Errorf("season 1 = %#v", seasons[0])
}
if want := "file://" + filepath.Join(s1, "poster.jpg"); seasons[0].PosterPath != want {
t.Errorf("season 1 poster = %q, want %q", seasons[0].PosterPath, want)
}
if seasons[1].SeasonNumber != 2 || seasons[1].Title != "Course B" {
t.Errorf("season 2 = %#v", seasons[1])
}
if want := "file://" + filepath.Join(root, "season02-poster.jpg"); seasons[1].PosterPath != want {
t.Errorf("season 2 poster = %q, want %q", seasons[1].PosterPath, want)
}
}
// A mismatched <seasonnumber> is advisory: naming owns structure, so the
// directory-derived season number wins.
func TestGetSeasons_NumberConflictPrefersDirectoryNumber(t *testing.T) {
t.Parallel()
root := t.TempDir()
s2 := filepath.Join(root, "Season 02")
writeTestFile(t, filepath.Join(s2, "season.nfo"),
`<season><title>Course B</title><seasonnumber>7</seasonnumber></season>`)
p := NewProvider()
seasons, err := p.GetSeasons(context.Background(), metadata.SeasonsRequest{
ContentType: "series",
SeriesRootPaths: []string{root},
SeasonDirectoryPaths: map[int][]string{2: {s2}},
})
if err != nil {
t.Fatalf("GetSeasons: %v", err)
}
if len(seasons) != 1 || seasons[0].SeasonNumber != 2 || seasons[0].Title != "Course B" {
t.Fatalf("seasons = %#v, want season 2 titled Course B", seasons)
}
}
func TestGetSeasons_NoSidecarsReturnsNothing(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
if err := os.MkdirAll(s1, 0o755); err != nil {
t.Fatal(err)
}
p := NewProvider()
seasons, err := p.GetSeasons(context.Background(), metadata.SeasonsRequest{
ContentType: "series",
SeriesRootPaths: []string{root},
SeasonDirectoryPaths: map[int][]string{1: {s1}},
})
if err != nil {
t.Fatalf("GetSeasons: %v", err)
}
if len(seasons) != 0 {
t.Fatalf("seasons = %#v, want none", seasons)
}
}
func TestGetSeasons_NonSeriesContentTypeReturnsNothing(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
writeTestFile(t, filepath.Join(s1, "season.nfo"), `<season><title>Nope</title></season>`)
p := NewProvider()
seasons, err := p.GetSeasons(context.Background(), metadata.SeasonsRequest{
ContentType: "movie",
SeasonDirectoryPaths: map[int][]string{1: {s1}},
})
if err != nil {
t.Fatalf("GetSeasons: %v", err)
}
if len(seasons) != 0 {
t.Fatalf("seasons = %#v, want none for non-series content", seasons)
}
}
// ---------------------------------------------------------------------------
// Provider: GetEpisodes
// ---------------------------------------------------------------------------
func TestGetEpisodes_EpisodeNFOAndThumb(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
e1 := filepath.Join(s1, "P90X S01E01 - Chest and Back.mkv")
writeTestFile(t, e1, "video")
writeTestFile(t, filepath.Join(s1, "P90X S01E01 - Chest and Back.nfo"),
`<episodedetails><title>Chest and Back</title><plot>Push and pull.</plot><aired>2004-01-01</aired><season>1</season><episode>1</episode><runtime>53</runtime></episodedetails>`)
writeTestFile(t, filepath.Join(s1, "P90X S01E01 - Chest and Back-thumb.jpg"), "img")
// Episode 2 has a media file but no NFO and no thumb: the provider must
// return nothing for it (synthesized fallback stays in charge).
e2 := filepath.Join(s1, "P90X S01E02 - Plyometrics.mkv")
writeTestFile(t, e2, "video")
p := NewProvider()
episodes, err := p.GetEpisodes(context.Background(), metadata.EpisodesRequest{
SeasonNumber: 1,
SeriesRootPaths: []string{root},
EpisodeFilePaths: map[int][]string{
1: {e1},
2: {e2},
},
})
if err != nil {
t.Fatalf("GetEpisodes: %v", err)
}
if len(episodes) != 1 {
t.Fatalf("len(episodes) = %d, want 1 (%#v)", len(episodes), episodes)
}
ep := episodes[0]
if ep.SeasonNumber != 1 || ep.EpisodeNumber != 1 {
t.Errorf("numbers = S%dE%d, want S1E1", ep.SeasonNumber, ep.EpisodeNumber)
}
if ep.Title != "Chest and Back" || ep.Overview != "Push and pull." {
t.Errorf("title/plot = %q/%q", ep.Title, ep.Overview)
}
if ep.AirDate != "2004-01-01" {
t.Errorf("AirDate = %q, want 2004-01-01", ep.AirDate)
}
if ep.Runtime != 53 {
t.Errorf("Runtime = %d, want 53", ep.Runtime)
}
if want := "file://" + filepath.Join(s1, "P90X S01E01 - Chest and Back-thumb.jpg"); ep.StillPath != want {
t.Errorf("StillPath = %q, want %q", ep.StillPath, want)
}
}
// NFO numbers are advisory: the filename-parsed SxxEyy (the request key) wins
// on conflict.
func TestGetEpisodes_NumberConflictFilenameWins(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
e3 := filepath.Join(s1, "Show S01E03.mkv")
writeTestFile(t, e3, "video")
writeTestFile(t, filepath.Join(s1, "Show S01E03.nfo"),
`<episodedetails><title>Third</title><season>4</season><episode>9</episode></episodedetails>`)
p := NewProvider()
episodes, err := p.GetEpisodes(context.Background(), metadata.EpisodesRequest{
SeasonNumber: 1,
EpisodeFilePaths: map[int][]string{3: {e3}},
})
if err != nil {
t.Fatalf("GetEpisodes: %v", err)
}
if len(episodes) != 1 || episodes[0].SeasonNumber != 1 || episodes[0].EpisodeNumber != 3 {
t.Fatalf("episodes = %#v, want S1E3 from filename", episodes)
}
if episodes[0].Title != "Third" {
t.Errorf("Title = %q, want Third", episodes[0].Title)
}
}
// A non-episode NFO next to the media file (e.g. a stray movie.nfo-shaped
// basename sidecar) must not inject data into an episode.
func TestGetEpisodes_RootTypeMismatchIgnored(t *testing.T) {
t.Parallel()
root := t.TempDir()
s1 := filepath.Join(root, "Season 01")
e1 := filepath.Join(s1, "Show S01E01.mkv")
writeTestFile(t, e1, "video")
writeTestFile(t, filepath.Join(s1, "Show S01E01.nfo"), `<movie><title>Not an episode</title></movie>`)
p := NewProvider()
episodes, err := p.GetEpisodes(context.Background(), metadata.EpisodesRequest{
SeasonNumber: 1,
EpisodeFilePaths: map[int][]string{1: {e1}},
})
if err != nil {
t.Fatalf("GetEpisodes: %v", err)
}
if len(episodes) != 0 {
t.Fatalf("episodes = %#v, want none for mismatched root type", episodes)
}
}
internal/metadata/nfo_identity_test.go
+494
View File
@@ -0,0 +1,494 @@
package metadata
import (
"context"
"strings"
"sync"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// localHintStubProvider mimics the built-in NFO provider's shape in the chain:
// it implements SearchProvider + MetadataProvider + IdentityHintProvider with
// canned data, exactly like a parsed sidecar NFO would supply.
type localHintStubProvider struct {
mu sync.Mutex
hints map[string]string
searchResults []SearchResult
metadata *MetadataResult
hintCalls int
searchCalls int
metadataCalls int
}
func (p *localHintStubProvider) Slug() string { return "nfo" }
func (p *localHintStubProvider) Name() string { return "NFO Files" }
func (p *localHintStubProvider) ForTypes() []string { return []string{"movie", "series"} }
func (p *localHintStubProvider) IdentityHints(_ context.Context, _ SearchQuery) map[string]string {
p.mu.Lock()
defer p.mu.Unlock()
p.hintCalls++
return copyMap(p.hints)
}
func (p *localHintStubProvider) Search(_ context.Context, _ SearchQuery) ([]SearchResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.searchCalls++
out := make([]SearchResult, len(p.searchResults))
copy(out, p.searchResults)
return out, nil
}
func (p *localHintStubProvider) GetMetadata(_ context.Context, _ MetadataRequest) (*MetadataResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.metadataCalls++
if p.metadata != nil {
cp := *p.metadata
cp.ProviderIDs = copyMap(p.metadata.ProviderIDs)
return &cp, nil
}
return &MetadataResult{}, nil
}
// remoteStubProvider is a canned remote search+metadata provider that records
// the provider IDs it was asked to fetch metadata for.
type remoteStubProvider struct {
mu sync.Mutex
slug string
searchResults []SearchResult
metadata *MetadataResult
lastSearchIDs map[string]string
lastMetaIDs map[string]string
metadataCalls int
}
func (p *remoteStubProvider) Slug() string { return p.slug }
func (p *remoteStubProvider) Name() string { return p.slug }
func (p *remoteStubProvider) ForTypes() []string { return []string{"movie", "series"} }
func (p *remoteStubProvider) Search(_ context.Context, query SearchQuery) ([]SearchResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.lastSearchIDs = copyMap(query.ProviderIDs)
out := make([]SearchResult, len(p.searchResults))
copy(out, p.searchResults)
return out, nil
}
func (p *remoteStubProvider) GetMetadata(_ context.Context, req MetadataRequest) (*MetadataResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.metadataCalls++
p.lastMetaIDs = copyMap(req.ProviderIDs)
if p.metadata != nil {
cp := *p.metadata
cp.ProviderIDs = copyMap(p.metadata.ProviderIDs)
return &cp, nil
}
return &MetadataResult{}, nil
}
func (p *remoteStubProvider) lastMetadataIDs() map[string]string {
p.mu.Lock()
defer p.mu.Unlock()
return copyMap(p.lastMetaIDs)
}
func seedMovieItem(t *testing.T, h *testHarness, contentID, title string, year int) {
t.Helper()
if err := h.itemRepo.Upsert(context.Background(), &models.MediaItem{
ContentID: contentID,
Type: "movie",
Title: title,
Year: year,
Status: "matched",
Studios: []string{},
Networks: []string{},
Countries: []string{},
Genres: []string{},
}); err != nil {
t.Fatalf("seed item: %v", err)
}
}
// A title-only NFO with no remote search results must still produce an
// accepted match: the item stays on its path-deterministic local: content id,
// is persisted as matched, and gains no durable provider IDs (#216).
func TestInitialMatch_TitleOnlyNFO_NoRemoteResults_MatchesUnderLocalID(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
if err := h.itemRepo.Upsert(ctx, &models.MediaItem{
ContentID: "local-abc123",
Type: "movie",
Title: "My Home Movie",
Status: "pending_match",
Studios: []string{},
Networks: []string{},
Countries: []string{},
Genres: []string{},
}); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
nfo := &localHintStubProvider{
searchResults: []SearchResult{{Name: "My Home Movie", Year: 2021, Provider: "nfo"}},
metadata: &MetadataResult{HasMetadata: true, Title: "My Home Movie", Year: 2021, Overview: "Curated."},
}
remote := &remoteStubProvider{slug: "tmdb"}
result, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "local-abc123",
Hints: &MatchHints{
Title: "My Home Movie",
Year: 2021,
Type: "movie",
},
Language: "en",
Mode: ModeInitialMatch,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
item, err := h.itemRepo.GetByID(ctx, "local-abc123")
if err != nil {
t.Fatalf("item not found under local id after title-only NFO match: %v", err)
}
if item.Status != "matched" {
t.Errorf("item status = %q, want matched", item.Status)
}
if item.Title != "My Home Movie" {
t.Errorf("item title = %q, want My Home Movie", item.Title)
}
if item.TmdbID != "" || item.TvdbID != "" || item.ImdbID != "" {
t.Errorf("title-only NFO match must carry no provider ids, got tmdb=%q tvdb=%q imdb=%q",
item.TmdbID, item.TvdbID, item.ImdbID)
}
}
// The same title-only flow at the worker level: the queued movie file must
// leave the match queue (row deleted) with the item matched under local:.
func TestWorker_TitleOnlyNFO_MovieLeavesMatchQueueMatched(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
h.service.folderRepo = &fakeWorkerFolderRepo{
folders: map[int]*models.MediaFolder{
10: {ID: 10, Type: "movies", Enabled: true},
},
}
nfo := &localHintStubProvider{
searchResults: []SearchResult{{Name: "My Home Movie", Year: 2021, Provider: "nfo"}},
metadata: &MetadataResult{HasMetadata: true, Title: "My Home Movie", Year: 2021},
}
remote := &remoteStubProvider{slug: "tmdb"}
h.service.hooks.process = func(ctx context.Context, req ProcessRequest) (*ProcessResult, error) {
return h.service.ProcessWithProviders(ctx, req, []Provider{nfo, remote})
}
file := &models.MediaFile{
ID: 1,
MediaFolderID: 10,
FilePath: "/media/movies/My Home Movie (2021)/My Home Movie (2021).mkv",
}
movieRepo := newFakeMovieQueueRepo(file)
worker := NewMatchWorker(h.service, h.fileRepo, 1, 10, 0)
worker.movieClaimer = movieRepo
worker.processUnmatched(ctx)
if _, deleted := movieRepo.deleted[file.ID]; !deleted {
t.Fatalf("movie queue row not deleted; errors=%v", movieRepo.errors)
}
contentID := h.fileRepo.contentIDs[file.ID]
if contentID == "" {
t.Fatal("file was not linked to a content id")
}
if !strings.HasPrefix(contentID, "local-") {
t.Errorf("content id = %q, want local- prefix", contentID)
}
item, err := h.itemRepo.GetByID(ctx, contentID)
if err != nil {
t.Fatalf("load matched item: %v", err)
}
if item.Status != "matched" {
t.Errorf("item status = %q, want matched", item.Status)
}
}
// A curated <uniqueid> must anchor identity even when remote search-by-title
// returns nothing: the NFO's tmdb id is seeded as a trusted hint and the
// remote provider's GetMetadata runs with it (full enrichment by ID).
func TestInitialMatch_NFOUniqueIDAnchorsWhenRemoteSearchEmpty(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
nfo := &localHintStubProvider{
hints: map[string]string{"tmdb": "424242"},
searchResults: []SearchResult{{Name: "Obscure Film", Year: 2019, ProviderIDs: map[string]string{"tmdb": "424242"}, Provider: "nfo"}},
metadata: &MetadataResult{HasMetadata: true, Title: "Obscure Film", Year: 2019},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Obscure Film: Remote Cut", Year: 2019, Overview: "From TMDB.", ProviderIDs: map[string]string{"tmdb": "424242", "imdb": "tt0424242"}},
}
result, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
Hints: &MatchHints{
Title: "Obscure Film",
Year: 2019,
Type: "movie",
},
Language: "en",
Mode: ModeInitialMatch,
}, []Provider{remote, nfo})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "424242" {
t.Errorf("remote GetMetadata tmdb id = %q, want 424242 (NFO hint must anchor identity)", got)
}
item, err := h.itemRepo.GetByID(ctx, result.ContentID)
if err != nil {
t.Fatalf("load matched item: %v", err)
}
if item.TmdbID != "424242" {
t.Errorf("item tmdb id = %q, want 424242", item.TmdbID)
}
if item.Overview != "From TMDB." {
t.Errorf("item overview = %q, want remote enrichment applied", item.Overview)
}
}
// When a title-only NFO coexists with a successful remote search, the remote
// match must win: no downgrade of a found movie to an unenriched local: item,
// and the winning candidate keeps the remote provider IDs.
func TestInitialMatch_TitleOnlyNFO_RemoteSearchWins(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
nfo := &localHintStubProvider{
searchResults: []SearchResult{{Name: "Inception", Year: 2010, Provider: "nfo"}},
metadata: &MetadataResult{HasMetadata: true, Title: "Inception", Year: 2010},
}
remote := &remoteStubProvider{
slug: "tmdb",
searchResults: []SearchResult{{Name: "Inception", Year: 2010, ProviderIDs: map[string]string{"tmdb": "27205"}, Provider: "tmdb"}},
metadata: &MetadataResult{HasMetadata: true, Title: "Inception", Year: 2010, ProviderIDs: map[string]string{"tmdb": "27205"}},
}
// NFO deliberately first in the chain: chain priority must not let the
// ID-less local candidate beat the ID-bearing remote candidate.
result, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
Hints: &MatchHints{
Title: "Inception",
Year: 2010,
Type: "movie",
},
Language: "en",
Mode: ModeInitialMatch,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
item, err := h.itemRepo.GetByID(ctx, result.ContentID)
if err != nil {
t.Fatalf("load matched item: %v", err)
}
if item.TmdbID != "27205" {
t.Errorf("item tmdb id = %q, want 27205 (remote match must not be downgraded to local:)", item.TmdbID)
}
}
// Initial match: NFO hints beat folder-name-derived hints on conflict.
func TestInitialMatch_NFOHintBeatsFolderNameHint(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
nfo := &localHintStubProvider{
hints: map[string]string{"tmdb": "222"},
metadata: &MetadataResult{HasMetadata: true, Title: "Right Movie"},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Right Movie", ProviderIDs: map[string]string{"tmdb": "222"}},
}
_, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
Hints: &MatchHints{
Title: "Right Movie",
Year: 2020,
Type: "movie",
TmdbID: "111", // folder-name-derived hint
},
Language: "en",
Mode: ModeInitialMatch,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "222" {
t.Errorf("remote GetMetadata tmdb id = %q, want 222 (NFO hint beats folder-name hint)", got)
}
}
// Scheduled refresh: stored durable IDs beat NFO hints (no background
// identity flips).
func TestScheduledRefresh_StoredIDBeatsNFOHint(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
seedMovieItem(t, h, "movie:tmdb:100", "Stable Movie", 2018)
h.itemRepo.items["movie:tmdb:100"].TmdbID = "100"
nfo := &localHintStubProvider{
hints: map[string]string{"tmdb": "999"},
metadata: &MetadataResult{HasMetadata: true, Title: "Stable Movie"},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Stable Movie", ProviderIDs: map[string]string{"tmdb": "100"}},
}
_, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "movie:tmdb:100",
Language: "en",
Mode: ModeScheduledRefresh,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "100" {
t.Errorf("scheduled refresh tmdb id = %q, want 100 (stored id beats NFO hint)", got)
}
}
// Manual refresh: NFO hints beat stored IDs — fixing a wrong NFO id and
// refreshing re-anchors the item (recovery path).
func TestManualRefresh_NFOHintBeatsStoredID(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
seedMovieItem(t, h, "movie:tmdb:100", "Misidentified Movie", 2018)
h.itemRepo.items["movie:tmdb:100"].TmdbID = "100"
nfo := &localHintStubProvider{
hints: map[string]string{"tmdb": "200"},
metadata: &MetadataResult{HasMetadata: true, Title: "Corrected Movie"},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Corrected Movie", ProviderIDs: map[string]string{"tmdb": "200"}},
}
_, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "movie:tmdb:100",
Language: "en",
Mode: ModeManualRefresh,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "200" {
t.Errorf("manual refresh tmdb id = %q, want 200 (NFO hint beats stored id)", got)
}
}
// Manual Identify: the user's explicit identification wins and the NFO
// provider is skipped entirely — no hints, no metadata overlay.
func TestIdentify_SkipsNFOProviderEntirely(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
seedMovieItem(t, h, "movie:tmdb:100", "Old Title", 2018)
nfo := &localHintStubProvider{
hints: map[string]string{"tmdb": "999"},
metadata: &MetadataResult{HasMetadata: true, Title: "Stale NFO Title"},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "User Chosen Title", ProviderIDs: map[string]string{"tmdb": "555"}},
}
result, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "movie:tmdb:100",
ProviderIDs: map[string]string{"tmdb": "555"},
Language: "en",
Mode: ModeIdentify,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
if nfo.hintCalls != 0 || nfo.searchCalls != 0 || nfo.metadataCalls != 0 {
t.Errorf("NFO provider consulted during identify: hints=%d search=%d metadata=%d, want all 0",
nfo.hintCalls, nfo.searchCalls, nfo.metadataCalls)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "555" {
t.Errorf("identify tmdb id = %q, want 555 (user identification wins)", got)
}
item, err := h.itemRepo.GetByID(ctx, "movie:tmdb:100")
if err != nil {
t.Fatalf("load item: %v", err)
}
if item.Title != "User Chosen Title" {
t.Errorf("item title = %q, want User Chosen Title (stale NFO title must not overlay identify)", item.Title)
}
}
// Phase-2: NFO GetMetadata results merge fields but never inject provider-id
// keys that conflict with or extend a remotely-anchored identity.
func TestPhase2_NFOMetadataDoesNotInjectProviderIDs(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
nfo := &localHintStubProvider{
metadata: &MetadataResult{
HasMetadata: true,
Title: "Anchored Movie",
ProviderIDs: map[string]string{"tmdb": "999", "imdb": "tt0000001"},
},
}
remote := &remoteStubProvider{
slug: "tmdb",
searchResults: []SearchResult{{Name: "Anchored Movie", Year: 2015, ProviderIDs: map[string]string{"tmdb": "42"}, Provider: "tmdb"}},
metadata: &MetadataResult{HasMetadata: true, Title: "Anchored Movie", Year: 2015, ProviderIDs: map[string]string{"tmdb": "42"}},
}
result, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
Hints: &MatchHints{
Title: "Anchored Movie",
Year: 2015,
Type: "movie",
},
Language: "en",
Mode: ModeInitialMatch,
}, []Provider{remote, nfo})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
item, err := h.itemRepo.GetByID(ctx, result.ContentID)
if err != nil {
t.Fatalf("load item: %v", err)
}
if item.TmdbID != "42" {
t.Errorf("item tmdb id = %q, want 42 (NFO metadata ids must not overwrite the anchor)", item.TmdbID)
}
if item.ImdbID != "" {
t.Errorf("item imdb id = %q, want empty (NFO must not extend a remotely-anchored identity)", item.ImdbID)
}
}
internal/metadata/nfo_merge_semantics_test.go
+188
View File
@@ -0,0 +1,188 @@
package metadata
import (
"context"
"reflect"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// Phase-2 accumulation runs NFO (priority 1) first, then remote providers,
// always with MergeFillEmpty: the NFO wins every field it populates and the
// remote result backfills the rest.
func TestMerge_NFOFirstFillEmpty_NFOWinsRemoteBackfills(t *testing.T) {
t.Parallel()
nfoResult := &MetadataResult{
HasMetadata: true,
Title: "Curated Title",
Overview: "Curated overview.",
Genres: []string{"Documentary"},
Ratings: Ratings{IMDB: 8.1},
}
remoteResult := &MetadataResult{
HasMetadata: true,
Title: "Remote Title",
Overview: "Remote overview.",
Tagline: "Remote tagline",
Runtime: 117,
Genres: []string{"Action", "Thriller"},
Studios: []string{"Remote Studio"},
Ratings: Ratings{IMDB: 7.0, TMDB: 7.5},
People: []models.ItemPerson{{Person: models.Person{Name: "Remote Actor"}, Kind: models.PersonKindActor}},
ContentRating: "R",
}
accumulator := &MetadataResult{}
MergeMetadata(nfoResult, accumulator, nil, MergeFillEmpty)
MergeMetadata(remoteResult, accumulator, nil, MergeFillEmpty)
// NFO wins the fields it populated.
if accumulator.Title != "Curated Title" {
t.Errorf("Title = %q, want Curated Title", accumulator.Title)
}
if accumulator.Overview != "Curated overview." {
t.Errorf("Overview = %q, want the NFO overview", accumulator.Overview)
}
if accumulator.Ratings.IMDB != 8.1 {
t.Errorf("Ratings.IMDB = %v, want 8.1 (NFO wins)", accumulator.Ratings.IMDB)
}
// Genres are first-provider-wins as a whole list: no remote union.
if want := []string{"Documentary"}; !reflect.DeepEqual(accumulator.Genres, want) {
t.Errorf("Genres = %#v, want %#v (whole-list first-provider-wins)", accumulator.Genres, want)
}
// Remote backfills what the NFO left empty.
if accumulator.Tagline != "Remote tagline" || accumulator.Runtime != 117 || accumulator.ContentRating != "R" {
t.Errorf("backfill = tagline %q runtime %d rating %q", accumulator.Tagline, accumulator.Runtime, accumulator.ContentRating)
}
if accumulator.Ratings.TMDB != 7.5 {
t.Errorf("Ratings.TMDB = %v, want 7.5 (remote backfills)", accumulator.Ratings.TMDB)
}
if len(accumulator.Studios) != 1 || len(accumulator.People) != 1 {
t.Errorf("Studios = %#v People = %#v, want remote backfill", accumulator.Studios, accumulator.People)
}
}
// An NFO that declares no <genre> must leave Genres untouched so the remote
// provider's list applies (the provider emits nil, never an empty slice).
func TestMerge_NFOWithoutGenres_RemoteGenresApply(t *testing.T) {
t.Parallel()
nfoResult := &MetadataResult{HasMetadata: true, Title: "Curated Title"}
remoteResult := &MetadataResult{HasMetadata: true, Genres: []string{"Action", "Thriller"}}
accumulator := &MetadataResult{}
MergeMetadata(nfoResult, accumulator, nil, MergeFillEmpty)
MergeMetadata(remoteResult, accumulator, nil, MergeFillEmpty)
if want := []string{"Action", "Thriller"}; !reflect.DeepEqual(accumulator.Genres, want) {
t.Errorf("Genres = %#v, want %#v", accumulator.Genres, want)
}
}
// FieldReleaseDates locks Year and the release/air dates against refreshes.
func TestMerge_ReleaseDatesLockBlocksReplace(t *testing.T) {
t.Parallel()
source := &MetadataResult{
HasMetadata: true,
Year: 1982,
ReleaseDate: "1982-06-25",
FirstAirDate: "1982-06-25",
LastAirDate: "1983-01-01",
}
target := &MetadataResult{
Year: 1990,
ReleaseDate: "1990-01-01",
FirstAirDate: "1990-01-01",
LastAirDate: "1991-01-01",
}
MergeMetadata(source, target, []MetadataField{FieldReleaseDates}, MergeReplaceUnlocked)
if target.Year != 1990 || target.ReleaseDate != "1990-01-01" ||
target.FirstAirDate != "1990-01-01" || target.LastAirDate != "1991-01-01" {
t.Errorf("locked dates overwritten: %+v", target)
}
// Unlocked, replace mode applies the source values.
MergeMetadata(source, target, nil, MergeReplaceUnlocked)
if target.Year != 1982 || target.ReleaseDate != "1982-06-25" {
t.Errorf("unlocked replace did not apply: %+v", target)
}
}
// Refresh contract, scheduled side: MergeFillEmpty seeds the accumulator from
// the existing item, so an edited NFO title does NOT propagate on the 6h cycle.
func TestScheduledRefresh_DoesNotPropagateNFOEdits(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
seedMovieItem(t, h, "movie:tmdb:100", "Original Title", 2018)
h.itemRepo.items["movie:tmdb:100"].TmdbID = "100"
h.itemRepo.items["movie:tmdb:100"].Overview = "Original overview."
nfo := &localHintStubProvider{
metadata: &MetadataResult{HasMetadata: true, Title: "Edited via NFO", Overview: "Edited overview."},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Original Title", Year: 2018, ProviderIDs: map[string]string{"tmdb": "100"}},
}
_, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "movie:tmdb:100",
Language: "en",
Mode: ModeScheduledRefresh,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
item, err := h.itemRepo.GetByID(ctx, "movie:tmdb:100")
if err != nil {
t.Fatalf("load item: %v", err)
}
if item.Title != "Original Title" {
t.Errorf("scheduled refresh title = %q, want Original Title (NFO edits must not propagate)", item.Title)
}
if item.Overview != "Original overview." {
t.Errorf("scheduled refresh overview = %q, want the original", item.Overview)
}
}
// Refresh contract, manual side: MergeReplaceUnlocked propagates NFO edits
// ("edit the NFO, then Refresh metadata").
func TestManualRefresh_PropagatesNFOEdits(t *testing.T) {
h := newTestHarness()
ctx := context.Background()
seedMovieItem(t, h, "movie:tmdb:100", "Original Title", 2018)
h.itemRepo.items["movie:tmdb:100"].TmdbID = "100"
h.itemRepo.items["movie:tmdb:100"].Overview = "Original overview."
nfo := &localHintStubProvider{
metadata: &MetadataResult{HasMetadata: true, Title: "Edited via NFO", Overview: "Edited overview."},
}
remote := &remoteStubProvider{
slug: "tmdb",
metadata: &MetadataResult{HasMetadata: true, Title: "Original Title", Year: 2018, ProviderIDs: map[string]string{"tmdb": "100"}},
}
_, err := h.service.ProcessWithProviders(ctx, ProcessRequest{
ContentID: "movie:tmdb:100",
Language: "en",
Mode: ModeManualRefresh,
}, []Provider{nfo, remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
item, err := h.itemRepo.GetByID(ctx, "movie:tmdb:100")
if err != nil {
t.Fatalf("load item: %v", err)
}
if item.Title != "Edited via NFO" {
t.Errorf("manual refresh title = %q, want Edited via NFO", item.Title)
}
if item.Overview != "Edited overview." {
t.Errorf("manual refresh overview = %q, want the NFO edit", item.Overview)
}
}
internal/metadata/nfo_mixed_library_test.go
+394
View File
@@ -0,0 +1,394 @@
package metadata_test
// Mixed sports library tests (plan §2a win 3): one library of type "mixed"
// holding WWE pay-per-view events as movies alongside "WWE SmackDown" as a
// show, metadata driven mainly by NFO (sports content often has no TVDB/TMDB
// presence). The classification contract: naming decides movie-vs-series per
// file before any metadata provider runs; NFOs supply metadata and identity,
// never type (the provider's ContentType guard ignores a tvshow.nfo next to
// a movie-classified file). These tests reuse the NFOSeriesHarness bridge
// and the real built-in NFO provider, mirroring the fitness-library tests.
import (
"context"
"path/filepath"
"strings"
"sync"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
"github.com/Silo-Server/silo-server/internal/metadata/nfo"
"github.com/Silo-Server/silo-server/internal/naming"
)
// ---------------------------------------------------------------------------
// (a) Classification pin: naming decides type in a mixed library.
// ---------------------------------------------------------------------------
// Sports-flavored classification cases for the mixed default branch
// (internal/naming/filename.go): SxxEyy/season-folder routes to the series
// lane, everything else to the movie lane. NFO contents never participate.
func TestMixedLibrary_NamingDecidesType(t *testing.T) {
t.Parallel()
cases := []struct {
name string
path string
wantType string
wantSeason int
wantEpisode int
}{
{
name: "PPV event in its own movie folder",
path: "/sports/WWE/WrestleMania 41 (2025)/WrestleMania 41 (2025).mkv",
wantType: "movie",
},
{
name: "bare PPV event file defaults to the movie lane",
path: "/sports/WWE/Royal Rumble 2025.mkv",
wantType: "movie",
},
{
name: "weekly show with season folder and SxxEyy",
path: "/sports/WWE/WWE SmackDown/Season 27/WWE SmackDown S27E15.mkv",
wantType: "series",
wantSeason: 27,
wantEpisode: 15,
},
{
name: "SxxEyy without a season folder still routes to series",
path: "/sports/WWE/WWE SmackDown/WWE SmackDown S27E15.mkv",
wantType: "series",
wantSeason: 27,
wantEpisode: 15,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
ctx := naming.ResolvePathContext(tc.path, "mixed")
if ctx.Type != tc.wantType {
t.Fatalf("ResolvePathContext(%q, mixed).Type = %q, want %q", tc.path, ctx.Type, tc.wantType)
}
if ctx.SeasonNum != tc.wantSeason || ctx.EpisodeNum != tc.wantEpisode {
t.Errorf("season/episode = S%dE%d, want S%dE%d",
ctx.SeasonNum, ctx.EpisodeNum, tc.wantSeason, tc.wantEpisode)
}
})
}
}
// ---------------------------------------------------------------------------
// (b) Movie lane: an NFO-only PPV event.
// ---------------------------------------------------------------------------
func TestMixedLibrary_MovieLane_NFOOnlyPPV(t *testing.T) {
root := filepath.Join(t.TempDir(), "WWE", "WrestleMania 41 (2025)")
movieFile := filepath.Join(root, "WrestleMania 41 (2025).mkv")
writeFixtureFile(t, movieFile, "video")
writeFixtureFile(t, filepath.Join(root, "movie.nfo"),
`<movie><title>WrestleMania 41</title><plot>Two nights from Las Vegas.</plot><year>2025</year></movie>`)
writeFixtureFile(t, filepath.Join(root, "poster.jpg"), "ppv-poster")
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const movieID = "local-wm41"
if err := x.SeedMovieSkeleton(movieID, "WrestleMania 41"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: movieID,
Hints: &metadata.MatchHints{
Title: "WrestleMania 41",
Year: 2025,
Type: "movie",
FilePath: movieFile,
RepresentativeFilePath: movieFile,
AllGroupFilePaths: []string{movieFile},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
item, err := x.Item(movieID)
if err != nil {
t.Fatalf("PPV movie not found under local id: %v", err)
}
if item.Status != "matched" {
t.Errorf("item status = %q, want matched", item.Status)
}
if item.Type != "movie" {
t.Errorf("item type = %q, want movie (NFO must not flip type)", item.Type)
}
if item.Title != "WrestleMania 41" || item.Overview != "Two nights from Las Vegas." {
t.Errorf("item title/overview = %q/%q", item.Title, item.Overview)
}
if item.TmdbID != "" || item.TvdbID != "" || item.ImdbID != "" {
t.Errorf("title-only PPV must carry no remote ids, got tmdb=%q tvdb=%q imdb=%q",
item.TmdbID, item.TvdbID, item.ImdbID)
}
if want := "file://" + filepath.Join(root, "poster.jpg"); item.PosterSourcePath != want {
t.Errorf("item PosterSourcePath = %q, want %q", item.PosterSourcePath, want)
}
jobs := x.EnqueuedImageJobs()
found := false
for _, job := range jobs {
if job.SourcePath == "file://"+filepath.Join(root, "poster.jpg") {
found = true
if job.TargetType != metadata.ImageCacheTargetItem || job.ProviderID != "local" {
t.Errorf("poster job = target %q provider %q, want item/local", job.TargetType, job.ProviderID)
}
}
if !strings.HasPrefix(job.SourcePath, "file://") {
t.Errorf("enqueued non-local source %q", job.SourcePath)
}
}
if !found {
t.Errorf("PPV poster not enqueued as a file:// local source (jobs: %#v)", jobs)
}
}
// ---------------------------------------------------------------------------
// (c) Series lane: the weekly show next to the PPVs, full NFO depth.
// ---------------------------------------------------------------------------
func TestMixedLibrary_SeriesLane_WeeklyShow(t *testing.T) {
root := filepath.Join(t.TempDir(), "WWE", "WWE SmackDown")
writeFixtureFile(t, filepath.Join(root, "tvshow.nfo"),
`<tvshow><title>WWE SmackDown</title><plot>Friday night wrestling.</plot></tvshow>`)
s27 := filepath.Join(root, "Season 27")
writeFixtureFile(t, filepath.Join(s27, "season.nfo"),
`<season><title>SmackDown 2025</title><seasonnumber>27</seasonnumber></season>`)
writeFixtureFile(t, filepath.Join(s27, "poster.jpg"), "s27-poster")
episodeFile := filepath.Join(s27, "WWE SmackDown S27E15.mkv")
writeFixtureFile(t, episodeFile, "video")
writeFixtureFile(t, filepath.Join(s27, "WWE SmackDown S27E15.nfo"),
`<episodedetails><title>SmackDown: April 11, 2025</title><plot>Go-home show before Mania.</plot></episodedetails>`)
writeFixtureFile(t, filepath.Join(s27, "WWE SmackDown S27E15-thumb.jpg"), "e15-thumb")
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const seriesID = "local-smackdown"
if err := x.SeedSeriesSkeleton(seriesID, "WWE SmackDown"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Hints: &metadata.MatchHints{
Title: "WWE SmackDown",
Type: "series",
FilePath: episodeFile,
RepresentativeFilePath: episodeFile,
AllGroupFilePaths: []string{episodeFile},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
item, err := x.Item(seriesID)
if err != nil {
t.Fatalf("show not found under local id: %v", err)
}
if item.Type != "series" || item.Title != "WWE SmackDown" {
t.Errorf("item = type %q title %q, want series / WWE SmackDown", item.Type, item.Title)
}
seasons := x.Seasons(seriesID)
if len(seasons) != 1 || seasons[0].SeasonNumber != 27 || seasons[0].Title != "SmackDown 2025" {
t.Fatalf("seasons = %#v, want season 27 titled SmackDown 2025", seasons)
}
if want := "file://" + filepath.Join(s27, "poster.jpg"); seasons[0].PosterSourcePath != want {
t.Errorf("season PosterSourcePath = %q, want %q", seasons[0].PosterSourcePath, want)
}
episodes := x.Episodes(seriesID)
if len(episodes) != 1 || episodes[0].SeasonNumber != 27 || episodes[0].EpisodeNumber != 15 {
t.Fatalf("episodes = %#v, want S27E15", episodes)
}
if episodes[0].Title != "SmackDown: April 11, 2025" || episodes[0].Overview != "Go-home show before Mania." {
t.Errorf("episode title/plot = %q/%q", episodes[0].Title, episodes[0].Overview)
}
if want := "file://" + filepath.Join(s27, "WWE SmackDown S27E15-thumb.jpg"); episodes[0].StillSourcePath != want {
t.Errorf("episode StillSourcePath = %q, want %q", episodes[0].StillSourcePath, want)
}
}
// ---------------------------------------------------------------------------
// (d) Cross-type guard: a tvshow.nfo next to a movie-classified file.
// ---------------------------------------------------------------------------
// A tvshow.nfo adjacent to a movie-classified PPV file must be ignored by
// the ContentType guard: no series metadata is injected into the movie, and
// with nothing else to match against the item stays an unmatched skeleton
// (pipeline-level mirror of the Phase B provider guard tests).
func TestMixedLibrary_TVShowNFONextToMovieIgnored(t *testing.T) {
root := filepath.Join(t.TempDir(), "WWE", "WrestleMania 41 (2025)")
movieFile := filepath.Join(root, "WrestleMania 41 (2025).mkv")
writeFixtureFile(t, movieFile, "video")
// Only a series-shaped NFO — e.g. an export tool misplaced it.
writeFixtureFile(t, filepath.Join(root, "tvshow.nfo"),
`<tvshow><title>WWE SmackDown</title><plot>Wrong sidecar.</plot></tvshow>`)
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const movieID = "local-wm41-guard"
if err := x.SeedMovieSkeleton(movieID, "WrestleMania 41"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: movieID,
Hints: &metadata.MatchHints{
Title: "WrestleMania 41",
Year: 2025,
Type: "movie",
FilePath: movieFile,
RepresentativeFilePath: movieFile,
AllGroupFilePaths: []string{movieFile},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result != nil && result.Updated {
t.Fatalf("result = %#v, want Updated=false (mismatched NFO must not produce a match)", result)
}
item, err := x.Item(movieID)
if err != nil {
t.Fatalf("load skeleton: %v", err)
}
if item.Type != "movie" {
t.Errorf("item type = %q, want movie (tvshow.nfo must not flip type)", item.Type)
}
if item.Title != "WrestleMania 41" {
t.Errorf("item title = %q, want untouched WrestleMania 41 (no series title injection)", item.Title)
}
if item.Overview != "" {
t.Errorf("item overview = %q, want empty (series plot must not be injected)", item.Overview)
}
if item.Status != "pending_match" {
t.Errorf("item status = %q, want pending_match (skeleton unchanged)", item.Status)
}
}
// ---------------------------------------------------------------------------
// (e) Mixed identity: an ID-bearing PPV next to NFO-only ones.
// ---------------------------------------------------------------------------
// remoteMovieStubProvider is a canned remote movie provider that records the
// provider IDs it was asked to fetch metadata for (Phase A harness shape).
type remoteMovieStubProvider struct {
mu sync.Mutex
slug string
metadata *metadata.MetadataResult
lastMetaIDs map[string]string
}
func (p *remoteMovieStubProvider) Slug() string { return p.slug }
func (p *remoteMovieStubProvider) Name() string { return p.slug }
func (p *remoteMovieStubProvider) ForTypes() []string { return []string{"movie", "series"} }
func (p *remoteMovieStubProvider) Search(_ context.Context, _ metadata.SearchQuery) ([]metadata.SearchResult, error) {
return nil, nil // remote search knows nothing about obscure sports events
}
func (p *remoteMovieStubProvider) GetMetadata(_ context.Context, req metadata.MetadataRequest) (*metadata.MetadataResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.lastMetaIDs = make(map[string]string, len(req.ProviderIDs))
for k, v := range req.ProviderIDs {
p.lastMetaIDs[k] = v
}
if p.metadata != nil {
cp := *p.metadata
return &cp, nil
}
return &metadata.MetadataResult{}, nil
}
func (p *remoteMovieStubProvider) lastMetadataIDs() map[string]string {
p.mu.Lock()
defer p.mu.Unlock()
return p.lastMetaIDs
}
// A PPV whose movie.nfo carries a tmdb <uniqueid> anchors a remote match via
// the trusted-hint phase even when remote search-by-title finds nothing —
// remote-capable events enrich fully while NFO-only events (case b) fall
// back locally, side by side in the same mixed library.
func TestMixedLibrary_PPVWithUniqueIDAnchorsRemoteMatch(t *testing.T) {
root := filepath.Join(t.TempDir(), "WWE", "WrestleMania 41 (2025)")
movieFile := filepath.Join(root, "WrestleMania 41 (2025).mkv")
writeFixtureFile(t, movieFile, "video")
writeFixtureFile(t, filepath.Join(root, "movie.nfo"),
`<movie><title>WrestleMania 41</title><year>2025</year><uniqueid type="tmdb">424242</uniqueid></movie>`)
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
remote := &remoteMovieStubProvider{
slug: "tmdb",
metadata: &metadata.MetadataResult{
HasMetadata: true,
Title: "WrestleMania 41",
Year: 2025,
Overview: "From TMDB.",
ProviderIDs: map[string]string{"tmdb": "424242"},
},
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
Hints: &metadata.MatchHints{
Title: "WrestleMania 41",
Year: 2025,
Type: "movie",
FilePath: movieFile,
RepresentativeFilePath: movieFile,
AllGroupFilePaths: []string{movieFile},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider(), remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
if got := remote.lastMetadataIDs()["tmdb"]; got != "424242" {
t.Errorf("remote GetMetadata tmdb id = %q, want 424242 (NFO uniqueid must anchor identity)", got)
}
item, err := x.Item(result.ContentID)
if err != nil {
t.Fatalf("load matched item: %v", err)
}
if item.TmdbID != "424242" {
t.Errorf("item tmdb id = %q, want 424242", item.TmdbID)
}
if item.Overview != "From TMDB." {
t.Errorf("item overview = %q, want remote enrichment applied", item.Overview)
}
}
internal/metadata/nfo_series_depth_test.go
+412
View File
@@ -0,0 +1,412 @@
package metadata_test
// Phase D integration tests: an NFO-only series library scans into a fully
// presented tree using the real built-in NFO provider — series metadata and
// root art (Phases A+C), season names/posters from season.nfo, and episode
// titles/plots/thumbs from <basename>.nfo (Phase D). These live in an
// external test package so they can import internal/metadata/nfo (which
// imports internal/metadata) without an import cycle; the in-package
// fakes are exposed through NFOSeriesHarness (export_nfo_test.go).
import (
"context"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"github.com/Silo-Server/silo-server/internal/metadata"
"github.com/Silo-Server/silo-server/internal/metadata/nfo"
)
func writeFixtureFile(t *testing.T, path, contents string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatalf("MkdirAll(%s): %v", path, err)
}
if err := os.WriteFile(path, []byte(contents), 0o644); err != nil {
t.Fatalf("WriteFile(%s): %v", path, err)
}
}
// buildFitnessFixture lays out the P90X fitness library from the Phase D spec
// and returns (seriesRoot, episode file paths).
func buildFitnessFixture(t *testing.T) (string, []string) {
t.Helper()
root := filepath.Join(t.TempDir(), "Fitness", "P90X")
writeFixtureFile(t, filepath.Join(root, "tvshow.nfo"),
`<tvshow><title>P90X</title><plot>90-day home fitness program.</plot></tvshow>`)
writeFixtureFile(t, filepath.Join(root, "poster.jpg"), "series-poster")
writeFixtureFile(t, filepath.Join(root, "fanart.jpg"), "series-fanart")
s1 := filepath.Join(root, "Season 01")
writeFixtureFile(t, filepath.Join(s1, "season.nfo"),
`<season><title>Course A: Classic</title><plot>The classic schedule.</plot><seasonnumber>1</seasonnumber></season>`)
writeFixtureFile(t, filepath.Join(s1, "poster.jpg"), "s1-poster")
e1 := filepath.Join(s1, "P90X S01E01 - Chest and Back.mkv")
writeFixtureFile(t, e1, "video")
writeFixtureFile(t, filepath.Join(s1, "P90X S01E01 - Chest and Back.nfo"),
`<episodedetails><title>Chest and Back</title><plot>Push and pull.</plot><season>1</season><episode>1</episode></episodedetails>`)
writeFixtureFile(t, filepath.Join(s1, "P90X S01E01 - Chest and Back-thumb.jpg"), "e1-thumb")
s2 := filepath.Join(root, "Season 02")
writeFixtureFile(t, filepath.Join(s2, "season.nfo"),
`<season><title>Course B</title><seasonnumber>2</seasonnumber></season>`)
writeFixtureFile(t, filepath.Join(s2, "poster.jpg"), "s2-poster")
e2 := filepath.Join(s2, "P90X S02E01 - Core.mkv")
writeFixtureFile(t, e2, "video")
writeFixtureFile(t, filepath.Join(s2, "P90X S02E01 - Core.nfo"),
`<episodedetails><title>Core</title><plot>Core synergistics.</plot></episodedetails>`)
writeFixtureFile(t, filepath.Join(s2, "P90X S02E01 - Core-thumb.jpg"), "e2-thumb")
return root, []string{e1, e2}
}
// The headline Phase D acceptance case: an NFO-only fitness library with zero
// remote IDs anywhere scans into a matched show under a local: content id,
// seasons carrying the NFO names and local posters, episodes carrying NFO
// titles/plots and thumbs, and every image enqueued as a file:// local source.
func TestNFOOnlyFitnessLibrary_FullSeriesDepth(t *testing.T) {
root, episodeFiles := buildFitnessFixture(t)
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const seriesID = "local-p90x"
if err := x.SeedSeriesSkeleton(seriesID, "P90X"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Hints: &metadata.MatchHints{
Title: "P90X",
Type: "series",
FilePath: episodeFiles[0],
RepresentativeFilePath: episodeFiles[0],
AllGroupFilePaths: episodeFiles,
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
// Show: matched under the local: content id with NFO metadata, no remote IDs.
item, err := x.Item(seriesID)
if err != nil {
t.Fatalf("item not found under local id: %v", err)
}
if item.Status != "matched" {
t.Errorf("item status = %q, want matched", item.Status)
}
if item.Title != "P90X" || item.Overview != "90-day home fitness program." {
t.Errorf("item title/overview = %q/%q", item.Title, item.Overview)
}
if item.TmdbID != "" || item.TvdbID != "" || item.ImdbID != "" {
t.Errorf("NFO-only series must carry no remote ids, got tmdb=%q tvdb=%q imdb=%q",
item.TmdbID, item.TvdbID, item.ImdbID)
}
if want := "file://" + filepath.Join(root, "poster.jpg"); item.PosterSourcePath != want {
t.Errorf("item PosterSourcePath = %q, want %q", item.PosterSourcePath, want)
}
if want := "file://" + filepath.Join(root, "fanart.jpg"); item.BackdropSourcePath != want {
t.Errorf("item BackdropSourcePath = %q, want %q", item.BackdropSourcePath, want)
}
// Seasons: NFO names and local posters.
seasons := x.Seasons(seriesID)
if len(seasons) != 2 {
t.Fatalf("len(seasons) = %d, want 2 (%#v)", len(seasons), seasons)
}
if seasons[0].SeasonNumber != 1 || seasons[0].Title != "Course A: Classic" {
t.Errorf("season 1 = number %d title %q, want 1 / Course A: Classic",
seasons[0].SeasonNumber, seasons[0].Title)
}
if seasons[0].Overview != "The classic schedule." {
t.Errorf("season 1 overview = %q", seasons[0].Overview)
}
if want := "file://" + filepath.Join(root, "Season 01", "poster.jpg"); seasons[0].PosterSourcePath != want {
t.Errorf("season 1 PosterSourcePath = %q, want %q", seasons[0].PosterSourcePath, want)
}
if seasons[0].PosterPath != "" {
t.Errorf("season 1 PosterPath = %q, want empty until the cache job lands", seasons[0].PosterPath)
}
if seasons[1].SeasonNumber != 2 || seasons[1].Title != "Course B" {
t.Errorf("season 2 = number %d title %q, want 2 / Course B",
seasons[1].SeasonNumber, seasons[1].Title)
}
if want := "file://" + filepath.Join(root, "Season 02", "poster.jpg"); seasons[1].PosterSourcePath != want {
t.Errorf("season 2 PosterSourcePath = %q, want %q", seasons[1].PosterSourcePath, want)
}
// Episodes: NFO titles/plots and thumbs.
episodes := x.Episodes(seriesID)
if len(episodes) != 2 {
t.Fatalf("len(episodes) = %d, want 2 (%#v)", len(episodes), episodes)
}
if episodes[0].SeasonNumber != 1 || episodes[0].EpisodeNumber != 1 ||
episodes[0].Title != "Chest and Back" || episodes[0].Overview != "Push and pull." {
t.Errorf("episode S1E1 = %#v", episodes[0])
}
if want := "file://" + filepath.Join(root, "Season 01", "P90X S01E01 - Chest and Back-thumb.jpg"); episodes[0].StillSourcePath != want {
t.Errorf("episode S1E1 StillSourcePath = %q, want %q", episodes[0].StillSourcePath, want)
}
if episodes[1].SeasonNumber != 2 || episodes[1].EpisodeNumber != 1 || episodes[1].Title != "Core" {
t.Errorf("episode S2E1 = %#v", episodes[1])
}
// Every image is enqueued as a file:// local source.
jobs := x.EnqueuedImageJobs()
wantSources := map[string]string{
"file://" + filepath.Join(root, "poster.jpg"): metadata.ImageCacheTargetItem,
"file://" + filepath.Join(root, "fanart.jpg"): metadata.ImageCacheTargetItem,
"file://" + filepath.Join(root, "Season 01", "poster.jpg"): metadata.ImageCacheTargetSeason,
"file://" + filepath.Join(root, "Season 02", "poster.jpg"): metadata.ImageCacheTargetSeason,
"file://" + filepath.Join(root, "Season 01", "P90X S01E01 - Chest and Back-thumb.jpg"): metadata.ImageCacheTargetEpisode,
"file://" + filepath.Join(root, "Season 02", "P90X S02E01 - Core-thumb.jpg"): metadata.ImageCacheTargetEpisode,
}
got := make(map[string]string, len(jobs))
for _, job := range jobs {
got[job.SourcePath] = job.TargetType
if !strings.HasPrefix(job.SourcePath, "file://") {
t.Errorf("enqueued non-local source %q", job.SourcePath)
}
if job.ProviderID != "local" {
t.Errorf("job for %q attributed to provider %q, want local", job.SourcePath, job.ProviderID)
}
}
for source, target := range wantSources {
if got[source] != target {
t.Errorf("image job for %q = target %q, want %q (all jobs: %#v)", source, got[source], target, got)
}
}
}
// remoteEpisodeStubProvider is a canned remote series provider with
// season/episode support, standing in for TVDB in the mixed case.
type remoteEpisodeStubProvider struct {
mu sync.Mutex
slug string
metadata *metadata.MetadataResult
seasons []metadata.SeasonResult
episodes map[int][]metadata.EpisodeResult
}
func (p *remoteEpisodeStubProvider) Slug() string { return p.slug }
func (p *remoteEpisodeStubProvider) Name() string { return p.slug }
func (p *remoteEpisodeStubProvider) ForTypes() []string { return []string{"movie", "series"} }
func (p *remoteEpisodeStubProvider) GetMetadata(_ context.Context, _ metadata.MetadataRequest) (*metadata.MetadataResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
if p.metadata != nil {
cp := *p.metadata
return &cp, nil
}
return &metadata.MetadataResult{}, nil
}
func (p *remoteEpisodeStubProvider) GetSeasons(_ context.Context, _ metadata.SeasonsRequest) ([]metadata.SeasonResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
out := make([]metadata.SeasonResult, len(p.seasons))
copy(out, p.seasons)
return out, nil
}
func (p *remoteEpisodeStubProvider) GetEpisodes(_ context.Context, req metadata.EpisodesRequest) ([]metadata.EpisodeResult, error) {
p.mu.Lock()
defer p.mu.Unlock()
out := make([]metadata.EpisodeResult, len(p.episodes[req.SeasonNumber]))
copy(out, p.episodes[req.SeasonNumber])
return out, nil
}
// Mixed case: a real remote show with a curated season.nfo. With the NFO
// provider at chain priority 1, the local season name wins and the remote
// provider backfills everything the NFO does not supply (air date, episodes).
func TestMixedRemoteShow_CuratedSeasonNFOWins(t *testing.T) {
root := filepath.Join(t.TempDir(), "TV", "Real Show")
s1 := filepath.Join(root, "Season 01")
writeFixtureFile(t, filepath.Join(s1, "season.nfo"),
`<season><title>The Curated Season</title></season>`)
e1 := filepath.Join(s1, "Real Show S01E01.mkv")
writeFixtureFile(t, e1, "video")
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const seriesID = "series:tvdb:81189"
if err := x.SeedSeriesSkeleton(seriesID, "Real Show"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
remote := &remoteEpisodeStubProvider{
slug: "tvdb",
metadata: &metadata.MetadataResult{HasMetadata: true, Title: "Real Show", Overview: "From TVDB."},
seasons: []metadata.SeasonResult{
{SeasonNumber: 1, Title: "Season 1", Overview: "Remote season overview.", AirDate: "2010-01-01"},
},
episodes: map[int][]metadata.EpisodeResult{
1: {{SeasonNumber: 1, EpisodeNumber: 1, Title: "Remote Episode", Overview: "From TVDB."}},
},
}
// NFO first in the chain = chain priority 1.
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Hints: &metadata.MatchHints{
Title: "Real Show",
Type: "series",
FilePath: e1,
RepresentativeFilePath: e1,
AllGroupFilePaths: []string{e1},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider(), remote})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
if result == nil || !result.Updated {
t.Fatalf("result = %#v, want Updated=true", result)
}
seasons := x.Seasons(seriesID)
if len(seasons) != 1 {
t.Fatalf("len(seasons) = %d, want 1 (%#v)", len(seasons), seasons)
}
if seasons[0].Title != "The Curated Season" {
t.Errorf("season title = %q, want The Curated Season (NFO at priority 1 wins)", seasons[0].Title)
}
if seasons[0].Overview != "Remote season overview." {
t.Errorf("season overview = %q, want remote backfill", seasons[0].Overview)
}
if seasons[0].AirDate == nil {
t.Error("season air date not backfilled from remote provider")
}
episodes := x.Episodes(seriesID)
if len(episodes) != 1 || episodes[0].Title != "Remote Episode" {
t.Fatalf("episodes = %#v, want the remote episode (no episode NFO present)", episodes)
}
}
// Season NFO edits propagate on manual refresh only: a scheduled refresh
// fills empty fields but never re-applies an edited season.nfo over an
// existing name; a manual refresh replaces unlocked fields with the merged
// provider results, NFO first (same contract as item fields).
func TestSeasonNFOEdit_ManualRefreshOnlyPropagation(t *testing.T) {
root, episodeFiles := buildFitnessFixture(t)
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const seriesID = "local-p90x"
if err := x.SeedSeriesSkeleton(seriesID, "P90X"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
for i, path := range episodeFiles {
x.SeedSeriesFile(100+i, 10, path, root, seriesID)
}
if _, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Hints: &metadata.MatchHints{
Title: "P90X",
Type: "series",
FilePath: episodeFiles[0],
RepresentativeFilePath: episodeFiles[0],
AllGroupFilePaths: episodeFiles,
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()}); err != nil {
t.Fatalf("initial match: %v", err)
}
if seasons := x.Seasons(seriesID); len(seasons) == 0 || seasons[0].Title != "Course A: Classic" {
t.Fatalf("initial seasons = %#v, want Course A: Classic first", seasons)
}
// Edit the season NFO after the initial match.
writeFixtureFile(t, filepath.Join(root, "Season 01", "season.nfo"),
`<season><title>Course A: Renamed</title><seasonnumber>1</seasonnumber></season>`)
// Scheduled refresh: fill-empty only, the edit must NOT propagate.
if _, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Language: "en",
Mode: metadata.ModeScheduledRefresh,
}, []metadata.Provider{nfo.NewProvider()}); err != nil {
t.Fatalf("scheduled refresh: %v", err)
}
if seasons := x.Seasons(seriesID); seasons[0].Title != "Course A: Classic" {
t.Errorf("after scheduled refresh season title = %q, want unchanged Course A: Classic", seasons[0].Title)
}
// Manual refresh: the edit propagates.
if _, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Language: "en",
Mode: metadata.ModeManualRefresh,
}, []metadata.Provider{nfo.NewProvider()}); err != nil {
t.Fatalf("manual refresh: %v", err)
}
if seasons := x.Seasons(seriesID); seasons[0].Title != "Course A: Renamed" {
t.Errorf("after manual refresh season title = %q, want Course A: Renamed", seasons[0].Title)
}
}
// Regression: a series with no season/episode NFOs must behave exactly as
// before — the provider contributes nothing at season/episode depth and the
// synthesized fallback path stays in charge.
func TestSeriesWithoutSeasonOrEpisodeNFOs_UnchangedFallback(t *testing.T) {
root := filepath.Join(t.TempDir(), "TV", "Plain Show")
s1 := filepath.Join(root, "Season 01")
e1 := filepath.Join(s1, "Plain Show S01E01.mkv")
writeFixtureFile(t, e1, "video")
x := metadata.NewNFOSeriesHarness()
ctx := context.Background()
const seriesID = "local-plain"
if err := x.SeedSeriesSkeleton(seriesID, "Plain Show"); err != nil {
t.Fatalf("seed skeleton: %v", err)
}
result, err := x.Service().ProcessWithProviders(ctx, metadata.ProcessRequest{
ContentID: seriesID,
Hints: &metadata.MatchHints{
Title: "Plain Show",
Type: "series",
FilePath: e1,
RepresentativeFilePath: e1,
AllGroupFilePaths: []string{e1},
PrimarySidecarSearchPaths: []string{root},
},
Language: "en",
Mode: metadata.ModeInitialMatch,
}, []metadata.Provider{nfo.NewProvider()})
if err != nil {
t.Fatalf("ProcessWithProviders: %v", err)
}
// With no NFOs at all the provider finds nothing: the pipeline reports
// no update (the worker then synthesizes fallback structure), exactly as
// before Phase D.
if result == nil || result.Updated {
t.Fatalf("result = %#v, want Updated=false when the provider finds nothing", result)
}
if seasons := x.Seasons(seriesID); len(seasons) != 0 {
t.Errorf("seasons = %#v, want none from the provider path (fallback synthesis owns them)", seasons)
}
if episodes := x.Episodes(seriesID); len(episodes) != 0 {
t.Errorf("episodes = %#v, want none from the provider path", episodes)
}
if jobs := x.EnqueuedImageJobs(); len(jobs) != 0 {
t.Errorf("image jobs = %#v, want none without sidecar art", jobs)
}
}
internal/metadata/service.go
+359 -44
View File
@@ -7,6 +7,7 @@ import (
"log/slog"
"maps"
"path/filepath"
"slices"
"sort"
"strconv"
"strings"
@@ -893,6 +894,52 @@ func (s *MetadataService) localProviderContextForContent(ctx context.Context, co
}
}
// seriesChildLocalContext carries the local-sidecar context for season- and
// episode-level provider fetches: the series root sidecar directories plus
// the per-season directories and per-episode media file paths derived from
// filename parsing (naming owns structure; NFOs never create it).
type seriesChildLocalContext struct {
seriesRootPaths []string
seasonDirectoryPaths map[int][]string
episodeFilePaths map[int]map[int][]string // season -> episode -> media files
}
// buildSeriesChildLocalContext derives the season/episode sidecar context
// from the series' media file paths. Files without a filename-parseable
// episode number contribute nothing — exactly the set fallback synthesis
// also skips.
func buildSeriesChildLocalContext(seriesRootPaths []string, filePaths []string) seriesChildLocalContext {
childCtx := seriesChildLocalContext{
seriesRootPaths: compactUniqueFilePaths(seriesRootPaths),
seasonDirectoryPaths: make(map[int][]string),
episodeFilePaths: make(map[int]map[int][]string),
}
for _, path := range compactUniqueFilePaths(filePaths) {
hints := naming.ParseFilename(path, "series")
if hints == nil || hints.EpisodeNum == 0 {
continue
}
seasonNum, episodeNum := hints.SeasonNum, hints.EpisodeNum
dir := filepath.Dir(path)
if !slices.Contains(childCtx.seasonDirectoryPaths[seasonNum], dir) {
childCtx.seasonDirectoryPaths[seasonNum] = append(childCtx.seasonDirectoryPaths[seasonNum], dir)
}
if childCtx.episodeFilePaths[seasonNum] == nil {
childCtx.episodeFilePaths[seasonNum] = make(map[int][]string)
}
childCtx.episodeFilePaths[seasonNum][episodeNum] = append(childCtx.episodeFilePaths[seasonNum][episodeNum], path)
}
return childCtx
}
// seriesChildLocalContextForContent builds the season/episode sidecar context
// from a series' persisted media files (refresh paths, where no scan hints
// are available).
func (s *MetadataService) seriesChildLocalContextForContent(ctx context.Context, contentID string, folderID int) seriesChildLocalContext {
localCtx := s.localProviderContextForContent(ctx, contentID, folderID)
return buildSeriesChildLocalContext(localCtx.primarySidecarSearchPaths, localCtx.allGroupFilePaths)
}
func (s *MetadataService) directorySidecarSearchPathsForFiles(ctx context.Context, files []*models.MediaFile) []string {
if len(files) == 0 {
return nil
@@ -1145,6 +1192,20 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
AllGroupFilePaths: append([]string(nil), req.Hints.AllGroupFilePaths...),
PrimarySidecarSearchPaths: append([]string(nil), req.Hints.PrimarySidecarSearchPaths...),
}
// Seed curated identity hints (NFO <uniqueid>) before Phase-1 search.
// At initial match NFO hints beat folder-name-derived hints, but stored
// durable IDs (injected into req.ProviderIDs) beat NFO hints.
protectedKeys := make(map[string]bool, len(req.ProviderIDs))
for key := range req.ProviderIDs {
protectedKeys[key] = true
}
wonHints := applyBuiltinIdentityHints(ctx, itemChain, searchQuery, accumulatedIDs, protectedKeys)
selectionHints := req.Hints
if len(wonHints) > 0 {
hintsCopy := *req.Hints
overrideHintIDs(&hintsCopy, wonHints)
selectionHints = &hintsCopy
}
searchQuery = suppressTitleYearFallbackForTrustedIDs(searchQuery)
allResults := make([]SearchResult, 0)
for _, p := range itemChain {
@@ -1196,7 +1257,7 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
for _, p := range itemChain {
providerPriority = append(providerPriority, p.Slug())
}
if winner, ok := selectInitialMatchCandidate(req.Hints, candidates, providerPriority); ok && winner != nil {
if winner, ok := selectInitialMatchCandidate(selectionHints, candidates, providerPriority); ok && winner != nil {
for k, v := range winner.ProviderIDs {
if v != "" {
accumulatedIDs[k] = v
@@ -1262,6 +1323,18 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
searchQuery.ObservedRootPath = localCtx.observedRootPath
searchQuery.AllGroupFilePaths = append([]string(nil), localCtx.allGroupFilePaths...)
searchQuery.PrimarySidecarSearchPaths = append([]string(nil), localCtx.primarySidecarSearchPaths...)
// Seed curated identity hints (NFO <uniqueid>) before the refresh
// search. On a scheduled refresh stored durable IDs win over NFO hints
// (no background identity flips); on a manual refresh NFO hints win,
// which is the recovery path for a corrected NFO.
var protectedKeys map[string]bool
if req.Mode == ModeScheduledRefresh {
protectedKeys = make(map[string]bool, len(accumulatedIDs))
for key := range accumulatedIDs {
protectedKeys[key] = true
}
}
wonHints := applyBuiltinIdentityHints(ctx, itemChain, searchQuery, accumulatedIDs, protectedKeys)
searchQuery = suppressTitleYearFallbackForTrustedIDs(searchQuery)
allResults := make([]SearchResult, 0)
for _, p := range itemChain {
@@ -1288,7 +1361,7 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
}
}
candidates := NormalizeCandidates(allResults, contentType)
if winner, ok := selectRefreshMatchCandidate(existing, candidates); ok && winner != nil {
if winner, ok := selectRefreshMatchCandidate(existing, wonHints, candidates); ok && winner != nil {
for k, v := range winner.ProviderIDs {
if v != "" {
accumulatedIDs[k] = v
@@ -1333,6 +1406,13 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
if !ok {
continue
}
_, isIdentityHinter := p.(IdentityHintProvider)
// A manual Identify is the user's explicit identification: the local
// hint provider (NFO) is skipped entirely so a stale sidecar cannot
// re-apply its title/ids over the user's choice in the same operation.
if isIdentityHinter && req.Mode == ModeIdentify {
continue
}
result, err := mp.GetMetadata(ctx, MetadataRequest{
ProviderIDs: accumulatedIDs,
ContentType: contentType,
@@ -1358,6 +1438,13 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
if result == nil || !result.HasMetadata {
continue
}
// Identity-hint providers contribute IDs exclusively through the
// trusted-hint phase: their Phase-2 results merge metadata fields but
// never inject provider-id keys that conflict with or extend the
// established identity (kills chimeric ID sets).
if isIdentityHinter {
result.ProviderIDs = nil
}
// Bootstrap: feed new IDs to subsequent providers.
mergeProviderIDs(accumulator, result)
accumulatedIDs = accumulator.ProviderIDs
@@ -1371,10 +1458,19 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
if !ok {
continue
}
// A manual Identify skips the local hint provider (NFO) across every
// phase: its sidecar art is resolved by file path, not by the chosen id,
// so a stale poster/fanart must not be re-applied over the user's choice.
if _, isIdentityHinter := p.(IdentityHintProvider); isIdentityHinter && req.Mode == ModeIdentify {
continue
}
images, err := ip.GetImages(ctx, ImageRequest{
ProviderIDs: accumulatedIDs,
ContentType: contentType,
Language: req.Language,
ProviderIDs: accumulatedIDs,
ContentType: contentType,
Language: req.Language,
RepresentativeFilePath: representativeFilePath,
AllGroupFilePaths: allGroupFilePaths,
PrimarySidecarSearchPaths: primarySidecarSearchPaths,
})
if err != nil {
if handleProvider404(provider404s, accumulatedIDs, p.Slug(), err,
@@ -1393,6 +1489,10 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
var allSeasons []SeasonResult
var allEpisodes []EpisodeResult
if contentType == "series" {
childCtx := buildSeriesChildLocalContext(
primarySidecarSearchPaths,
append(append([]string(nil), representativeFilePath), allGroupFilePaths...),
)
seasonChain, err := resolveChain("season")
if err != nil {
return nil, err
@@ -1403,10 +1503,17 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
if !ok {
continue
}
// Identify skips the local hint provider (NFO): a stale season.nfo
// must not overlay season names on the user's chosen identity.
if _, isIdentityHinter := p.(IdentityHintProvider); isIdentityHinter && req.Mode == ModeIdentify {
continue
}
seasons, err := ep.GetSeasons(ctx, SeasonsRequest{
ProviderIDs: accumulatedIDs,
ContentType: contentType,
Language: req.Language,
ProviderIDs: accumulatedIDs,
ContentType: contentType,
Language: req.Language,
SeriesRootPaths: childCtx.seriesRootPaths,
SeasonDirectoryPaths: childCtx.seasonDirectoryPaths,
})
if err != nil {
// Pass nil for provider404s so this refresh can drop the
@@ -1426,33 +1533,54 @@ func (s *MetadataService) processInternal(ctx context.Context, req ProcessReques
}
allSeasons = flattenSeasonResults(seasonResults)
// Phase 4b: Episodes — resolve with "episode" content level.
if len(allSeasons) > 0 {
// Phase 4b: Episodes — resolve with "episode" content level. The
// season set is the provider-returned seasons plus any filename-
// derived on-disk seasons, so local episode NFOs are read even when
// no provider returned their season (persist then creates the
// implicit "Season N" rows).
episodeSeasonNumbers := make([]int, 0, len(allSeasons)+len(childCtx.episodeFilePaths))
for _, season := range allSeasons {
episodeSeasonNumbers = append(episodeSeasonNumbers, season.SeasonNumber)
}
for seasonNumber := range childCtx.episodeFilePaths {
if !slices.Contains(episodeSeasonNumbers, seasonNumber) {
episodeSeasonNumbers = append(episodeSeasonNumbers, seasonNumber)
}
}
sort.Ints(episodeSeasonNumbers)
if len(episodeSeasonNumbers) > 0 {
episodeChain, err := resolveChain("episode")
if err != nil {
return nil, err
}
episodeResults := make(map[episodeResultKey]*EpisodeResult)
for _, season := range allSeasons {
for _, seasonNumber := range episodeSeasonNumbers {
for _, p := range episodeChain {
ep, ok := p.(EpisodeProvider)
if !ok {
continue
}
// Identify skips the local hint provider (NFO): stale episode
// NFOs/thumbs must not overlay the user's chosen identity.
if _, isIdentityHinter := p.(IdentityHintProvider); isIdentityHinter && req.Mode == ModeIdentify {
continue
}
episodes, err := ep.GetEpisodes(ctx, EpisodesRequest{
ProviderIDs: accumulatedIDs,
SeasonNumber: season.SeasonNumber,
Language: req.Language,
ProviderIDs: accumulatedIDs,
SeasonNumber: seasonNumber,
Language: req.Language,
SeriesRootPaths: childCtx.seriesRootPaths,
EpisodeFilePaths: childCtx.episodeFilePaths[seasonNumber],
})
if err != nil {
if handleChildProvider404(p.Slug(), accumulatedIDs, err,
"content_id", req.ContentID,
"season", season.SeasonNumber,
"season", seasonNumber,
) {
continue
}
slog.WarnContext(ctx, "metadata: episode provider error", "component", "metadata",
"provider", p.Slug(), "season", season.SeasonNumber, "error", err)
"provider", p.Slug(), "season", seasonNumber, "error", err)
continue
}
accumulateEpisodeResults(episodeResults, episodes)
@@ -1645,6 +1773,33 @@ func (s *MetadataService) mergeAndPersist(
}
}
// Re-anchor an already provider-anchored item whose corrected identity now
// derives a different anchor — the recovery path when an admin fixes a wrong
// <uniqueid> in an NFO. Manual refresh only: the accumulator's IDs are seeded
// from the item's stored IDs and only a trusted NFO hint (which wins on
// manual refresh) can change them here, so a scheduled/background refresh
// never flips a stored identity. Reuses the local-promotion machinery under
// the provider-dedup lock; a no-op when the derived anchor is unchanged.
if !isNew && req.Mode == ModeManualRefresh && contentid.IsProviderAnchored(contentID) {
reanchored, err := s.reanchorContentID(
ctx, contentID, providerIDsStruct(accumulator.ProviderIDs), contentType)
if err != nil {
return nil, fmt.Errorf("reanchor provider content id: %w", err)
}
if reanchored != contentID {
contentID = reanchored
existingItem, err = s.itemRepo.GetByID(ctx, contentID)
if err != nil {
return nil, fmt.Errorf("loading reanchored item: %w", err)
}
locked = intSliceToFields(existingItem.LockedFields)
durableIDs, err = s.loadDurableProviderIDs(ctx, contentID)
if err != nil {
return nil, err
}
}
}
if len(durableIDs) > 0 {
if accumulator.ProviderIDs == nil {
accumulator.ProviderIDs = make(map[string]string, len(durableIDs))
@@ -1827,11 +1982,16 @@ func (s *MetadataService) mergeAndPersist(
}
}
// Persist seasons and episodes for series.
// Persist seasons and episodes for series. Episode-only results (e.g.
// local episode NFOs without a season.nfo) persist too — the persist
// path creates their implicit "Season N" rows. Fallback synthesis then
// covers any files the providers left unlinked (episodes with no NFO
// and no remote row); it is a no-op when every file is linked.
if contentType == "series" {
if len(seasons) > 0 {
if len(seasons) > 0 || len(episodes) > 0 {
s.persistSeasonsAndEpisodes(ctx, item, accumulator.ProviderIDs, canonicalLanguage, req.Language, seasons, episodes, mergeMode)
} else if err := s.SynthesizeFallbackEpisodes(ctx, contentID); err != nil {
}
if err := s.SynthesizeFallbackEpisodes(ctx, contentID); err != nil {
slog.WarnContext(ctx, "metadata: failed to synthesize fallback series structure", "component", "metadata",
"content_id", contentID, "error", err)
}
@@ -2536,6 +2696,7 @@ func (s *MetadataService) refreshSeriesChildTarget(
}
updated := false
childCtx := s.seriesChildLocalContextForContent(ctx, seriesID, folderID)
for _, language := range languages {
canonicalLanguage := strings.TrimSpace(series.DefaultMetadataLanguage)
if canonicalLanguage == "" {
@@ -2549,11 +2710,11 @@ func (s *MetadataService) refreshSeriesChildTarget(
if err != nil {
return err
}
seasons, err := s.fetchTargetSeasonResults(ctx, providerIDs, folderID, language, seasonNumber)
seasons, err := s.fetchTargetSeasonResults(ctx, providerIDs, folderID, language, seasonNumber, childCtx)
if err != nil {
return err
}
episodes, err := s.fetchTargetEpisodeResults(ctx, providerIDs, folderID, language, seasonNumber, episodeNumber)
episodes, err := s.fetchTargetEpisodeResults(ctx, providerIDs, folderID, language, seasonNumber, episodeNumber, childCtx)
if err != nil {
return err
}
@@ -2637,7 +2798,7 @@ func (s *MetadataService) resolveSeriesRefreshProviderIDs(ctx context.Context, s
}
}
candidates := NormalizeCandidates(allResults, series.Type)
if winner, ok := selectRefreshMatchCandidate(series, candidates); ok && winner != nil {
if winner, ok := selectRefreshMatchCandidate(series, nil, candidates); ok && winner != nil {
for k, v := range winner.ProviderIDs {
if v != "" {
accumulatedIDs[k] = v
@@ -2650,7 +2811,7 @@ func (s *MetadataService) resolveSeriesRefreshProviderIDs(ctx context.Context, s
return accumulatedIDs, nil
}
func (s *MetadataService) fetchTargetSeasonResults(ctx context.Context, providerIDs map[string]string, folderID int, language string, seasonNumber int) ([]SeasonResult, error) {
func (s *MetadataService) fetchTargetSeasonResults(ctx context.Context, providerIDs map[string]string, folderID int, language string, seasonNumber int, childCtx seriesChildLocalContext) ([]SeasonResult, error) {
seasonChain, err := s.resolveChainCached(ctx, folderID, "season")
if err != nil {
return nil, fmt.Errorf("resolve season provider chain: %w", err)
@@ -2662,9 +2823,11 @@ func (s *MetadataService) fetchTargetSeasonResults(ctx context.Context, provider
continue
}
seasons, err := ep.GetSeasons(ctx, SeasonsRequest{
ProviderIDs: providerIDs,
ContentType: "series",
Language: language,
ProviderIDs: providerIDs,
ContentType: "series",
Language: language,
SeriesRootPaths: childCtx.seriesRootPaths,
SeasonDirectoryPaths: childCtx.seasonDirectoryPaths,
})
if err != nil {
if handleProvider404(nil, providerIDs, p.Slug(), err, "season", seasonNumber) {
@@ -2684,7 +2847,7 @@ func (s *MetadataService) fetchTargetSeasonResults(ctx context.Context, provider
return flattenSeasonResults(seasonResults), nil
}
func (s *MetadataService) fetchTargetEpisodeResults(ctx context.Context, providerIDs map[string]string, folderID int, language string, seasonNumber int, episodeNumber int) ([]EpisodeResult, error) {
func (s *MetadataService) fetchTargetEpisodeResults(ctx context.Context, providerIDs map[string]string, folderID int, language string, seasonNumber int, episodeNumber int, childCtx seriesChildLocalContext) ([]EpisodeResult, error) {
episodeChain, err := s.resolveChainCached(ctx, folderID, "episode")
if err != nil {
return nil, fmt.Errorf("resolve episode provider chain: %w", err)
@@ -2696,9 +2859,11 @@ func (s *MetadataService) fetchTargetEpisodeResults(ctx context.Context, provide
continue
}
episodes, err := ep.GetEpisodes(ctx, EpisodesRequest{
ProviderIDs: providerIDs,
SeasonNumber: seasonNumber,
Language: language,
ProviderIDs: providerIDs,
SeasonNumber: seasonNumber,
Language: language,
SeriesRootPaths: childCtx.seriesRootPaths,
EpisodeFilePaths: childCtx.episodeFilePaths[seasonNumber],
})
if err != nil {
if handleChildProvider404(p.Slug(), providerIDs, err, "season", seasonNumber) {
@@ -2771,7 +2936,8 @@ func isCachedImagePath(path string) bool {
path != "-" &&
!strings.HasPrefix(path, "http://") &&
!strings.HasPrefix(path, "https://") &&
!isProviderImagePath(path)
!isProviderImagePath(path) &&
!isLocalImageSourcePath(path)
}
func isRemoteImageSourcePath(path string) bool {
@@ -2783,6 +2949,20 @@ func isRemoteImageSourcePath(path string) bool {
!isNonProviderImageScheme(lower)
}
// isLocalImageSourcePath reports whether path is a local sidecar artwork
// source (file:// scheme, produced by the NFO provider). Local sources route
// into *_source_path like remote sources and are copied into the S3 image
// cache by the processor; they are never served directly.
func isLocalImageSourcePath(path string) bool {
return strings.HasPrefix(strings.ToLower(strings.TrimSpace(path)), "file://")
}
// isCacheableImageSourcePath gates the image-cache enqueue paths: remote
// provider URLs and local file:// sidecars both queue cache jobs.
func isCacheableImageSourcePath(path string) bool {
return isRemoteImageSourcePath(path) || isLocalImageSourcePath(path)
}
func isNonProviderImageScheme(lowerPath string) bool {
return strings.HasPrefix(lowerPath, "s3://") ||
strings.HasPrefix(lowerPath, "file://") ||
@@ -2799,7 +2979,29 @@ func providerImageSourcePath(path string) string {
return ""
}
// splitProviderImagePath separates a provider-returned image path into the
// served path and the cacheable source path. A local file:// sidecar must
// never land in the served column: it routes into *_source_path only (the
// cache job fills the served path later). Remote sources keep the existing
// contract: the URL occupies both columns until the cache job lands.
func splitProviderImagePath(path string) (servedPath, sourcePath string) {
if isLocalImageSourcePath(path) {
return "", strings.TrimSpace(path)
}
return path, providerImageSourcePath(path)
}
func preserveCachedArtwork(providerPath, providerThumbhash, existingCachedPath, existingSourcePath, existingThumbhash string) (string, string, string) {
// A local file:// source must never land in the served *_path column: it
// routes into *_source_path and, during the pre-cache window, the path
// keeps the prior cached key (or stays empty) until the job completes.
if isLocalImageSourcePath(providerPath) {
providerPath = strings.TrimSpace(providerPath)
if isCachedImagePath(existingCachedPath) {
return existingCachedPath, existingThumbhash, providerPath
}
return "", "", providerPath
}
if !isRemoteImageSourcePath(providerPath) {
if strings.TrimSpace(providerPath) == "" && isCachedImagePath(existingCachedPath) {
return existingCachedPath, existingThumbhash, existingSourcePath
@@ -2961,7 +3163,21 @@ func buildItemLocalizationRecord(
LogoPath: loc.LogoPath,
LogoSourcePath: loc.LogoSourcePath,
}
applyBestImages(locItem, images, mergeMode, preferredLanguage)
// Local sidecar art is language-neutral: it must not duplicate into every
// localization row, so local candidates only compete at the item level.
remoteImages := images
for _, img := range images {
if isLocalImageSourcePath(img.URL) {
remoteImages = make([]RemoteImage, 0, len(images))
for _, candidate := range images {
if !isLocalImageSourcePath(candidate.URL) {
remoteImages = append(remoteImages, candidate)
}
}
break
}
}
applyBestImages(locItem, remoteImages, mergeMode, preferredLanguage)
prepareItemImagesForQueue(locItem, existingLocItem)
loc.PosterPath = locItem.PosterPath
@@ -3208,6 +3424,11 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
imageJobs := make([]EnqueueImageCacheJobInput, 0, len(seasons)+len(episodes))
fallbackProvider := primaryProviderID(providerIDs)
keyAttribution := func(sourcePath string) (string, string) {
// Local sidecar sources attribute to the synthetic "local" provider,
// keyed by the series' own content ID (mirrors item attribution).
if isLocalImageSourcePath(sourcePath) {
return imageCacheLocalProviderID, seriesID
}
providerID := providerIDFromPluginURL(sourcePath)
if providerID == "" {
providerID = fallbackProvider
@@ -3215,7 +3436,7 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
return providerID, findContentID(series, providerID)
}
addSeasonImageJob := func(season *models.Season) {
if season == nil || !isRemoteImageSourcePath(season.PosterSourcePath) {
if season == nil || !isCacheableImageSourcePath(season.PosterSourcePath) {
return
}
providerID, providerContentID := keyAttribution(season.PosterSourcePath)
@@ -3233,7 +3454,7 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
})
}
addEpisodeImageJob := func(episode *models.Episode) {
if episode == nil || !isRemoteImageSourcePath(episode.StillSourcePath) {
if episode == nil || !isCacheableImageSourcePath(episode.StillSourcePath) {
return
}
providerID, providerContentID := keyAttribution(episode.StillSourcePath)
@@ -3253,7 +3474,7 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
})
}
addSeasonLocalizationImageJob := func(season *models.Season, loc *models.SeasonLocalization) {
if season == nil || loc == nil || !isRemoteImageSourcePath(loc.PosterSourcePath) {
if season == nil || loc == nil || !isCacheableImageSourcePath(loc.PosterSourcePath) {
return
}
providerID, providerContentID := keyAttribution(loc.PosterSourcePath)
@@ -3282,13 +3503,15 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
continue
}
providerSeason := season
providerSeason.PosterSourcePath = providerImageSourcePath(providerSeason.PosterPath)
providerSeason.PosterPath, providerSeason.PosterSourcePath = splitProviderImagePath(season.PosterPath)
if existingSeason != nil && isCanonicalWrite {
mergedSeason := seasonResultFromModel(existingSeason)
MergeSeasonResult(&providerSeason, &mergedSeason, mergeMode)
// preserveCachedArtwork sees the raw provider path so a local
// file:// source still routes into *_source_path here.
nextPath, nextThumbhash, nextSourcePath := preserveCachedArtwork(
providerSeason.PosterPath,
providerSeason.PosterThumbhash,
season.PosterPath,
season.PosterThumbhash,
existingSeason.PosterPath,
existingSeason.PosterSourcePath,
existingSeason.PosterThumbhash,
@@ -3333,6 +3556,13 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
dbSeason.AirDate = &t
}
}
// An artwork-only season (poster but no season.nfo) would otherwise
// persist a blank title and then be skipped by fallback synthesis
// (which only fills not-yet-existing rows), so default it to the same
// "Season N"/"Specials" label synthesis uses. Graceful degradation.
if dbSeason.Title == "" {
dbSeason.Title = fallbackSeasonTitle(dbSeason.SeasonNumber)
}
if err := s.seasonRepo.Upsert(ctx, dbSeason); err != nil {
slog.WarnContext(ctx, "metadata: failed to upsert season", "component", "metadata",
"series_id", seriesID, "season", season.SeasonNumber, "error", err)
@@ -3439,13 +3669,15 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
continue
}
providerEpisode := ep
providerEpisode.StillSourcePath = providerImageSourcePath(providerEpisode.StillPath)
providerEpisode.StillPath, providerEpisode.StillSourcePath = splitProviderImagePath(ep.StillPath)
if existingEpisode != nil && isCanonicalWrite {
mergedEpisode := episodeResultFromModel(existingEpisode)
MergeEpisodeResult(&providerEpisode, &mergedEpisode, mergeMode)
// preserveCachedArtwork sees the raw provider path so a local
// file:// source still routes into *_source_path here.
nextPath, nextThumbhash, nextSourcePath := preserveCachedArtwork(
providerEpisode.StillPath,
providerEpisode.StillThumbhash,
ep.StillPath,
ep.StillThumbhash,
existingEpisode.StillPath,
existingEpisode.StillSourcePath,
existingEpisode.StillThumbhash,
@@ -3505,6 +3737,13 @@ func (s *MetadataService) persistSeasonsAndEpisodes(
v := providerEpisode.Ratings.IMDB
dbEp.RatingIMDB = &v
}
// An artwork-only episode (a -thumb.jpg but no episode .nfo) would
// otherwise persist a blank title and then be skipped by fallback
// synthesis, so default it to the same "Episode N" label synthesis
// uses. Graceful degradation for partially curated libraries.
if dbEp.Title == "" {
dbEp.Title = fallbackEpisodeTitle(dbEp.EpisodeNumber)
}
if err := s.episodeRepo.Upsert(ctx, dbEp); err != nil {
slog.WarnContext(ctx, "metadata: failed to upsert episode", "component", "metadata",
"series_id", seriesID, "season", ep.SeasonNumber,
@@ -5593,7 +5832,11 @@ func applyBestImages(item *models.MediaItem, images []RemoteImage, mode MergeMod
if b.url == "" {
return
}
if *current == "" || mode == MergeReplaceUnlocked || b.rating > 0 {
// Local sidecar candidates always apply: they carry rating 0, so an
// already-matched item could otherwise never pick up local art, and a
// rated remote image could stickily displace it after a transient
// local read failure.
if *current == "" || mode == MergeReplaceUnlocked || b.rating > 0 || isLocalImageSourcePath(b.url) {
*current = b.url
}
}
@@ -5658,7 +5901,7 @@ func (s *MetadataService) enqueueItemImages(ctx context.Context, item *models.Me
inputs := make([]EnqueueImageCacheJobInput, 0, 3)
for _, field := range itemArtworkFields(item) {
sourcePath := strings.TrimSpace(*field.source)
if !isRemoteImageSourcePath(sourcePath) {
if !isCacheableImageSourcePath(sourcePath) {
continue
}
providerID, providerContentID := itemImageCacheAttribution(item, providerIDs, images, sourcePath)
@@ -5690,7 +5933,7 @@ func (s *MetadataService) enqueueItemLocalizationImages(ctx context.Context, ite
inputs := make([]EnqueueImageCacheJobInput, 0, 3)
for _, field := range itemArtworkFields(locItem) {
sourcePath := strings.TrimSpace(*field.source)
if !isRemoteImageSourcePath(sourcePath) {
if !isCacheableImageSourcePath(sourcePath) {
continue
}
providerID, providerContentID := itemImageCacheAttribution(item, providerIDs, images, sourcePath)
@@ -5710,6 +5953,16 @@ func (s *MetadataService) enqueueItemLocalizationImages(ctx context.Context, ite
}
func itemImageCacheAttribution(item *models.MediaItem, providerIDs map[string]string, images []RemoteImage, sourcePath string) (string, string) {
// Local sidecar sources are attributed to the synthetic "local" provider
// (the generic scheme parse below would derive "file") and keyed by the
// item's own content ID, matching the audiobook/ebook local/ precedent.
if isLocalImageSourcePath(sourcePath) {
contentID := ""
if item != nil {
contentID = item.ContentID
}
return imageCacheLocalProviderID, contentID
}
providerID := providerIDFromPluginURL(sourcePath)
if providerID == "" {
providerID = findProviderID(images, sourcePath)
@@ -6104,6 +6357,68 @@ func searchResultConflictsWithTrustedIDs(hintedIDs, candidateIDs map[string]stri
return false
}
// applyBuiltinIdentityHints consults the chain's IdentityHintProviders (the
// built-in NFO provider) and folds their curated tmdb/imdb/tvdb ids into
// accumulatedIDs ahead of Phase-1 search. Keys in protected (stored durable
// ids) are never overwritten — a conflict is logged and the stored value
// kept. The returned map holds the hint values that won a slot in
// accumulatedIDs, so candidate selection can anchor on them through the
// trusted-hint machinery.
func applyBuiltinIdentityHints(
ctx context.Context,
chain []Provider,
query SearchQuery,
accumulatedIDs map[string]string,
protected map[string]bool,
) map[string]string {
var won map[string]string
for _, p := range chain {
hinter, ok := p.(IdentityHintProvider)
if !ok {
continue
}
hints := hinter.IdentityHints(ctx, query)
for _, key := range trustedSearchIDKeys {
value := strings.TrimSpace(hints[key])
if value == "" {
continue
}
current := strings.TrimSpace(accumulatedIDs[key])
if current != "" && protected[key] {
if current != value {
slog.WarnContext(ctx, "metadata: local identity hint conflicts with stored id; keeping stored id", "component", "metadata",
"provider", p.Slug(), "key", key, "stored", current, "hint", value)
}
continue
}
if current != "" && current != value {
slog.WarnContext(ctx, "metadata: local identity hint overrides conflicting hint id", "component", "metadata",
"provider", p.Slug(), "key", key, "previous", current, "hint", value)
}
accumulatedIDs[key] = value
if won == nil {
won = make(map[string]string, len(trustedSearchIDKeys))
}
won[key] = value
}
}
return won
}
// overrideHintIDs applies won identity-hint values onto a MatchHints copy so
// trustedHintIDsPresent/candidateMatchesTrustedIDs anchor selection on them.
func overrideHintIDs(hints *MatchHints, won map[string]string) {
if v := won["tmdb"]; v != "" {
hints.TmdbID = v
}
if v := won["tvdb"]; v != "" {
hints.TvdbID = v
}
if v := won["imdb"]; v != "" {
hints.ImdbID = v
}
}
func suppressTitleYearFallbackForTrustedIDs(query SearchQuery) SearchQuery {
for _, key := range trustedSearchIDKeys {
if strings.TrimSpace(query.ProviderIDs[key]) != "" {
internal/metadata/types.go
+34
View File
@@ -41,6 +41,9 @@ const (
FieldImages
FieldAirSchedule
FieldVideos
// FieldReleaseDates locks Year, ReleaseDate, and First/LastAirDate so a
// refresh (or an NFO at priority 1) cannot overwrite manual corrections.
FieldReleaseDates
)
// RefreshPriority controls queue ordering.
@@ -229,6 +232,14 @@ type ImageRequest struct {
ProviderIDs map[string]string
ContentType string
Language string
// Local sidecar context (additive; empty for purely remote providers).
// RepresentativeFilePath is the group's representative media file,
// AllGroupFilePaths are every file in the content group, and
// PrimarySidecarSearchPaths are directories already validated as safe for
// directory-level sidecars (single-content observed roots).
RepresentativeFilePath string
AllGroupFilePaths []string
PrimarySidecarSearchPaths []string
}
// RemoteImage describes an available image from a provider.
@@ -266,6 +277,10 @@ type CacheImageRequest struct {
SeasonNumber *int
EpisodeNumber *int
Language string
// KeyDiscriminator, when set, is inserted into the S3 key between the
// content ID and the image type (e.g. the 8-hex content hash of a local
// sidecar file) so re-cached art rotates to a fresh key.
KeyDiscriminator string
}
// CacheImageResult is returned by ImageCacher on success.
@@ -285,6 +300,13 @@ type ImageCacher interface {
CacheImage(ctx context.Context, req CacheImageRequest) (*CacheImageResult, error)
}
// ImageByteCacher caches an image already read into memory (e.g. a local
// sidecar file) to object storage. Implemented by imagecache.Cacher alongside
// ImageCacher; the processor type-asserts it for file:// sources.
type ImageByteCacher interface {
CacheImageBytes(ctx context.Context, data []byte, req CacheImageRequest) (*CacheImageResult, error)
}
type ImageCacheJobEnqueuer interface {
Enqueue(ctx context.Context, in EnqueueImageCacheJobInput) error
EnqueueBatch(ctx context.Context, inputs []EnqueueImageCacheJobInput) (int, error)
@@ -295,6 +317,13 @@ type SeasonsRequest struct {
ProviderIDs map[string]string
ContentType string
Language string
// Local sidecar context (additive, internal — mirrors MetadataRequest).
// SeriesRootPaths are directories already validated as safe for
// directory-level sidecars (the series root); SeasonDirectoryPaths maps
// each directory-derived season number to its candidate directories.
// Both are empty for purely remote providers, which key off ProviderIDs.
SeriesRootPaths []string
SeasonDirectoryPaths map[int][]string
}
// EpisodesRequest is passed to EpisodeProvider.GetEpisodes().
@@ -302,6 +331,11 @@ type EpisodesRequest struct {
ProviderIDs map[string]string
SeasonNumber int
Language string
// Local sidecar context (additive, internal). SeriesRootPaths as on
// SeasonsRequest; EpisodeFilePaths maps each filename-derived episode
// number within SeasonNumber to that episode's media file paths.
SeriesRootPaths []string
EpisodeFilePaths map[int][]string
}
// SeasonResult carries season data from a provider.
internal/plugins/auto_update.go
+8
View File
@@ -188,6 +188,14 @@ func (s *AutoUpdateService) Check(ctx context.Context, opts AutoUpdateOptions) (
if existing == nil || existing.RepositoryID == nil {
continue
}
// The reserved builtin row must never be matched against catalog
// entries: a catalog plugin named after it could otherwise rewrite its
// version/install_path and convert it into a launchable plugin.
// update_policy='manual' on the row and the reserved-plugin-id install
// rejection are the other layers.
if existing.IsBuiltin() {
continue
}
entry, ok := latestByRepositoryPlugin[repositoryPluginKey{
RepositoryID: *existing.RepositoryID,
PluginID: existing.PluginID,
internal/plugins/builtin_guard_test.go
+111
View File
@@ -0,0 +1,111 @@
package plugins
import (
"context"
"errors"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestInstallationIsBuiltin(t *testing.T) {
if (&Installation{Kind: KindBuiltin}).IsBuiltin() != true {
t.Error("kind=builtin must report IsBuiltin")
}
if (&Installation{Kind: KindPlugin}).IsBuiltin() {
t.Error("kind=plugin must not report IsBuiltin")
}
if (&Installation{}).IsBuiltin() {
t.Error("zero-value kind must not report IsBuiltin")
}
}
// The reserved builtin plugin id must be rejected at install time so a
// malicious or accidental catalog entry cannot hijack the reserved row.
func TestReservedPluginIDRejected(t *testing.T) {
if !isReservedPluginID("silo.builtin") {
t.Error("silo.builtin must be reserved")
}
if isReservedPluginID("silo.tmdb") || isReservedPluginID("silo.tvdb") {
t.Error("first-party plugin ids must stay installable")
}
if isReservedPluginID("community.example") {
t.Error("community plugin ids must stay installable")
}
}
// PreloadEnabled must skip builtin installations explicitly: the sentinel
// install path has no manifest on disk, and without the skip preload would
// fail (its caller is log.Fatalf) or at best emit spurious warnings.
func TestPreloadEnabledSkipsBuiltinInstallation(t *testing.T) {
store := newFakeServiceInstallationStore(&Installation{
ID: 1,
PluginID: "silo.builtin",
Kind: KindBuiltin,
Enabled: true,
InstallPath: "/nonexistent/silo-builtin",
})
svc := &Service{installations: store}
if err := svc.PreloadEnabled(context.Background()); err != nil {
t.Fatalf("PreloadEnabled with builtin-only store = %v, want nil (builtin must be skipped)", err)
}
}
// InstallationStore.Delete must reject builtin rows at the store layer: the
// delete cascades through every NFO chain row and RemoveAlls the install dir.
func TestInstallationStoreDeleteRejectsBuiltin(t *testing.T) {
pool := builtinGuardTestPool(t)
ctx := context.Background()
id := seedBuiltinTestInstallation(t, pool, "test.builtin.delete")
store := NewInstallationStore(pool)
err := store.Delete(ctx, id)
if err == nil {
t.Fatal("Delete(builtin) = nil, want error")
}
if !errors.Is(err, ErrBuiltinInstallationImmutable) {
t.Fatalf("Delete(builtin) error = %v, want ErrBuiltinInstallationImmutable", err)
}
var count int
if qErr := pool.QueryRow(ctx, `SELECT COUNT(*) FROM plugin_installations WHERE id = $1`, id).Scan(&count); qErr != nil {
t.Fatalf("count row: %v", qErr)
}
if count != 1 {
t.Fatalf("builtin row count after rejected delete = %d, want 1", count)
}
}
func builtinGuardTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func seedBuiltinTestInstallation(t *testing.T, pool *pgxpool.Pool, pluginID string) int {
t.Helper()
var id int
err := pool.QueryRow(context.Background(),
`INSERT INTO plugin_installations (plugin_id, version, install_path, enabled, update_policy, kind)
VALUES ($1, '0', '/nonexistent/silo-builtin-test', true, 'manual', 'builtin')
RETURNING id`, pluginID+time.Now().UTC().Format("-20060102150405.000000000")).Scan(&id)
if err != nil {
t.Fatalf("seed builtin installation: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM plugin_installations WHERE id = $1`, id)
})
return id
}
internal/plugins/event_dispatcher.go
+5
View File
@@ -175,6 +175,11 @@ func (d *EventDispatcher) fanOut(ctx context.Context, env events.Envelope, paylo
var wg sync.WaitGroup
for _, installation := range installations {
// Builtin installations are metadata-only and never launchable;
// defense-in-depth alongside the capability-type filter below.
if installation.IsBuiltin() {
continue
}
if env.TargetPluginID != "" && installation.PluginID != env.TargetPluginID {
continue
}
internal/plugins/installation.go
+39 -5
View File
@@ -19,6 +19,20 @@ var ErrInstallationNotFound = errors.New("plugin installation not found")
var ErrArchiveNotFound = errors.New("plugin archive not found")
var ErrInstallationDisabled = errors.New("plugin installation is disabled")
// ErrBuiltinInstallationImmutable is returned when a caller tries to mutate
// the reserved builtin-host installation row (delete, update, config, ...).
var ErrBuiltinInstallationImmutable = errors.New("builtin installation cannot be modified")
// Installation kinds. A 'builtin' installation is the reserved row that
// anchors built-in host provider capabilities (silo.builtin); it has no
// archive, manifest, or binary and must never be launched, updated, or
// deleted. Generic reads must NOT filter builtins — the metadata chain's
// enabled-check depends on reading them.
const (
KindPlugin = "plugin"
KindBuiltin = "builtin"
)
type Installation struct {
ID int
RepositoryID *int
@@ -26,12 +40,18 @@ type Installation struct {
Version string
InstallPath string
Enabled bool
Kind string `json:"kind"`
UpdatePolicy string `json:"update_policy"`
AvailableVersion *string `json:"available_version,omitempty"`
CreatedAt time.Time
UpdatedAt time.Time
}
// IsBuiltin reports whether this is the reserved builtin-host installation.
func (i *Installation) IsBuiltin() bool {
return i != nil && i.Kind == KindBuiltin
}
type Capability struct {
InstallationID int
Type string
@@ -77,7 +97,7 @@ func NewInstallationStore(pool *pgxpool.Pool) *InstallationStore {
return &InstallationStore{pool: pool}
}
const installationColumns = `id, repository_id, plugin_id, version, install_path, enabled, update_policy, available_version, created_at, updated_at`
const installationColumns = `id, repository_id, plugin_id, version, install_path, enabled, kind, update_policy, available_version, created_at, updated_at`
const capabilityColumns = `plugin_installation_id, capability_type, capability_id, metadata, created_at, updated_at`
const archiveColumns = `plugin_installation_id, manifest_json, checksum, archive_bytes, created_at, updated_at`
@@ -91,6 +111,7 @@ func scanInstallation(row pgx.Row) (*Installation, error) {
&installation.Version,
&installation.InstallPath,
&installation.Enabled,
&installation.Kind,
&installation.UpdatePolicy,
&installation.AvailableVersion,
&installation.CreatedAt,
@@ -263,6 +284,17 @@ func (s *InstallationStore) ListByPluginID(ctx context.Context, pluginID string)
}
func (s *InstallationStore) Update(ctx context.Context, id int, input UpdateInstallationInput) error {
// Guard at the store, not only the HTTP handler: the reserved builtin row
// carries no archive/binary and must never have its version, path, enabled
// flag, or capabilities rewritten, or its chain participation breaks.
installation, err := s.GetByID(ctx, id)
if err != nil {
return err
}
if installation.IsBuiltin() {
return ErrBuiltinInstallationImmutable
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("begin update installation transaction: %w", err)
@@ -314,10 +346,6 @@ func (s *InstallationStore) Update(ctx context.Context, id int, input UpdateInst
if tag.RowsAffected() == 0 {
return ErrInstallationNotFound
}
} else {
if _, err := s.GetByID(ctx, id); err != nil {
return err
}
}
if input.Capabilities != nil {
@@ -391,6 +419,12 @@ func (s *InstallationStore) Delete(ctx context.Context, id int) error {
if err != nil {
return err
}
// Guard at the store, not only the HTTP handler: deleting the reserved
// builtin row would cascade through every builtin chain row and RemoveAll
// the (sentinel) install dir.
if installation.IsBuiltin() {
return ErrBuiltinInstallationImmutable
}
tag, err := s.pool.Exec(ctx, `DELETE FROM plugin_installations WHERE id = $1`, id)
if err != nil {
internal/plugins/manifest.go
+11
View File
@@ -61,10 +61,21 @@ func capabilityKey(kind, id string) string {
return kind + "\x00" + id
}
// isReservedPluginID reports whether a plugin id is reserved for the host and
// must never be installable. Only the exact builtin registration id is
// reserved: first-party plugins legitimately use the "silo." prefix
// (silo.tmdb, silo.tvdb).
func isReservedPluginID(pluginID string) bool {
return pluginID == "silo.builtin"
}
func validateManifestShared(manifest *pluginv1.PluginManifest) error {
if err := publicmanifest.Validate(manifest); err != nil {
return err
}
if isReservedPluginID(manifest.GetPluginId()) {
return fmt.Errorf("plugin id %q is reserved for built-in host providers", manifest.GetPluginId())
}
if manifest.GetSiloApiVersion() == "" {
return fmt.Errorf("plugin manifest silo_api_version is required")
}
internal/plugins/service.go
+26
View File
@@ -462,6 +462,12 @@ func (s *Service) PreloadEnabled(ctx context.Context) error {
if installation == nil {
continue
}
// Builtin installations have no archive or binary; skip them explicitly
// instead of leaning on the tolerated ErrArchiveNotFound branch below
// (any other load error here is fatal to startup).
if installation.IsBuiltin() {
continue
}
if _, err := s.ensureLoadedInstallation(ctx, installation); err != nil {
if errors.Is(err, ErrArchiveNotFound) {
slog.WarnContext(ctx,
@@ -806,6 +812,15 @@ func (s *Service) loadInstallation(ctx context.Context, installationID int, requ
if requireEnabled && !installation.Enabled {
return nil, ErrInstallationDisabled
}
// requireEnabled marks paths that intend to launch or serve the plugin
// (start, manifest routes/assets, gRPC clients, HTTP proxy). The reserved
// builtin row has no binary behind it and must never reach those paths;
// not-found gives the proxy and API a clean 4xx. Reads with
// requireEnabled=false (IsInstallationEnabled for the metadata chain,
// generic listings) still see the row.
if requireEnabled && installation.IsBuiltin() {
return nil, ErrInstallationNotFound
}
return installation, nil
}
@@ -869,6 +884,17 @@ func (s *Service) IsInstallationEnabled(ctx context.Context, installationID int)
return installation.Enabled, nil
}
// InstallationKind returns the installation's kind ("plugin" or "builtin") from
// the same in-memory cache IsInstallationEnabled reads, so metadata chain
// resolution can identify builtin rows without a per-capability DB query.
func (s *Service) InstallationKind(ctx context.Context, installationID int) (string, error) {
installation, err := s.loadInstallation(ctx, installationID, false)
if err != nil {
return "", err
}
return installation.Kind, nil
}
func (s *Service) ensureLoadedInstallation(
ctx context.Context,
installation *Installation,
internal/plugins/task_registry.go
+5
View File
@@ -69,6 +69,11 @@ func (r *TaskRegistry) Tasks(ctx context.Context) ([]taskmanager.Task, error) {
var tasks []taskmanager.Task
for _, installation := range installations {
// Builtin installations expose no scheduled tasks and cannot be
// launched; defense-in-depth alongside the capability-type filter.
if installation.IsBuiltin() {
continue
}
capabilities, err := r.installations.ListCapabilities(ctx, installation.ID)
if err != nil {
return nil, fmt.Errorf("list plugin capabilities for installation %d: %w", installation.ID, err)
migrations/sql/20260712100125_builtin_host_providers.sql
+52
View File
@@ -0,0 +1,52 @@
-- +goose Up
-- Register built-in host metadata providers as data (#216, Phase A).
--
-- Chain rows require a real plugin_installations id (FK + composite PK), so
-- built-ins are represented by one reserved installation row of kind
-- 'builtin' plus ordinary plugin_capabilities rows. The install_path is a
-- guaranteed-nonexistent sentinel: InstallationStore.Delete removes the
-- install directory on disk, so the sentinel must never resolve to a real
-- directory (the store additionally rejects deleting builtin rows).
--
-- Statement order matters: the partial unique index must exist before any
-- ON CONFLICT targets it, and the installation insert uses
-- INSERT ... SELECT ... WHERE NOT EXISTS because plugin_id had no unique
-- constraint before this migration.
ALTER TABLE plugin_installations
ADD COLUMN kind TEXT NOT NULL DEFAULT 'plugin'
CONSTRAINT plugin_installations_kind_check CHECK (kind IN ('plugin', 'builtin'));
CREATE UNIQUE INDEX idx_plugin_installations_builtin_plugin_id
ON plugin_installations (plugin_id)
WHERE kind = 'builtin';
INSERT INTO plugin_installations (plugin_id, version, install_path, enabled, update_policy, kind)
SELECT 'silo.builtin', '0', '/nonexistent/silo-builtin', true, 'manual', 'builtin'
WHERE NOT EXISTS (
SELECT 1 FROM plugin_installations WHERE plugin_id = 'silo.builtin'
);
-- The NFO provider: seeded disabled everywhere (default_enabled=false), at
-- declared priority 1 for movie/series when an admin enables it.
INSERT INTO plugin_capabilities (plugin_installation_id, capability_type, capability_id, metadata)
SELECT pi.id, 'metadata_provider.v1', 'nfo',
'{"display_name":"NFO Files","default_priority":{"movie":1,"series":1},"default_enabled":false}'::jsonb
FROM plugin_installations pi
WHERE pi.plugin_id = 'silo.builtin' AND pi.kind = 'builtin'
ON CONFLICT DO NOTHING;
-- +goose Down
-- Rolling back deletes the reserved installation row; library_provider_chains
-- rows referencing it cascade, so any admin-configured NFO chain placement is
-- lost on rollback (accepted and documented).
DELETE FROM plugin_capabilities pc
USING plugin_installations pi
WHERE pc.plugin_installation_id = pi.id
AND pi.plugin_id = 'silo.builtin' AND pi.kind = 'builtin'
AND pc.capability_type = 'metadata_provider.v1' AND pc.capability_id = 'nfo';
DELETE FROM plugin_installations WHERE plugin_id = 'silo.builtin' AND kind = 'builtin';
DROP INDEX IF EXISTS idx_plugin_installations_builtin_plugin_id;
ALTER TABLE plugin_installations DROP COLUMN kind;
migrations/sql/20260712124741_nfo_capability_season_episode_levels.sql
+32
View File
@@ -0,0 +1,32 @@
-- +goose Up
-- Declare season/episode support for the built-in NFO provider (#216,
-- Phase D). The capability's default_priority map enumerates the content
-- levels a provider participates in: without these keys the startup builtin
-- chain sync never appends the NFO provider to season/episode chains, so
-- season.nfo names and episode NFO titles could never resolve. Priority 1
-- matches the movie/series declaration (local curation wins when enabled);
-- the capability stays default_enabled=false so nothing activates on its own.
UPDATE plugin_capabilities pc
SET metadata = jsonb_set(
pc.metadata,
'{default_priority}',
COALESCE(pc.metadata->'default_priority', '{}'::jsonb)
|| '{"season": 1, "episode": 1}'::jsonb
)
FROM plugin_installations pi
WHERE pc.plugin_installation_id = pi.id
AND pi.plugin_id = 'silo.builtin' AND pi.kind = 'builtin'
AND pc.capability_type = 'metadata_provider.v1' AND pc.capability_id = 'nfo';
-- +goose Down
UPDATE plugin_capabilities pc
SET metadata = jsonb_set(
pc.metadata,
'{default_priority}',
(pc.metadata->'default_priority') - 'season' - 'episode'
)
FROM plugin_installations pi
WHERE pc.plugin_installation_id = pi.id
AND pi.plugin_id = 'silo.builtin' AND pi.kind = 'builtin'
AND pc.capability_type = 'metadata_provider.v1' AND pc.capability_id = 'nfo'
AND pc.metadata ? 'default_priority';
web/src/components/EditMetadataDialog.tsx
+14 -5
View File
@@ -23,9 +23,11 @@ const FIELD_OVERVIEW = 1;
const FIELD_GENRES = 2;
const FIELD_STUDIOS = 3;
const FIELD_RATING = 6;
const FIELD_TAGS = 8;
const FIELD_RUNTIME = 7;
const FIELD_CONTENT_RATING = 9;
const FIELD_AIR_SCHEDULE = 11;
const FIELD_RELEASE_DATES = 13;
const AIR_TIMEZONES = [
"America/New_York",
@@ -62,7 +64,7 @@ const FIELD_LOCK_MAP: Record<string, number> = {
genres: FIELD_GENRES,
studios: FIELD_STUDIOS,
networks: FIELD_STUDIOS,
countries: FIELD_STUDIOS,
countries: FIELD_TAGS,
rating_imdb: FIELD_RATING,
rating_tmdb: FIELD_RATING,
rating_rt_critic: FIELD_RATING,
@@ -71,6 +73,10 @@ const FIELD_LOCK_MAP: Record<string, number> = {
content_rating: FIELD_CONTENT_RATING,
air_time: FIELD_AIR_SCHEDULE,
air_timezone: FIELD_AIR_SCHEDULE,
year: FIELD_RELEASE_DATES,
release_date: FIELD_RELEASE_DATES,
first_air_date: FIELD_RELEASE_DATES,
last_air_date: FIELD_RELEASE_DATES,
};
interface EditMetadataDialogProps {
@@ -429,7 +435,7 @@ export default function EditMetadataDialog({ item, open, onOpenChange }: EditMet
<div className="space-y-4">
{(item.type === "movie" || item.type === "series") && (
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
<FieldRow label="Year">
<FieldRow label="Year" lockIcon={renderLockIcon("year")}>
<Input
type="number"
value={form.year || ""}
@@ -438,7 +444,7 @@ export default function EditMetadataDialog({ item, open, onOpenChange }: EditMet
</FieldRow>
{item.type === "movie" && (
<FieldRow label="Release Date">
<FieldRow label="Release Date" lockIcon={renderLockIcon("release_date")}>
<Input
type="date"
value={form.release_date}
@@ -448,7 +454,10 @@ export default function EditMetadataDialog({ item, open, onOpenChange }: EditMet
)}
{item.type === "series" && (
<FieldRow label="First Air Date">
<FieldRow
label="First Air Date"
lockIcon={renderLockIcon("first_air_date")}
>
<Input
type="date"
value={form.first_air_date}
@@ -461,7 +470,7 @@ export default function EditMetadataDialog({ item, open, onOpenChange }: EditMet
{item.type === "series" && (
<div className="grid grid-cols-1 gap-4 sm:grid-cols-3">
<FieldRow label="Last Air Date">
<FieldRow label="Last Air Date" lockIcon={renderLockIcon("last_air_date")}>
<Input
type="date"
value={form.last_air_date}
web/src/components/admin/libraries/LibraryForm.test.tsx
+16 -44
View File
@@ -1,11 +1,9 @@
import { describe, expect, it } from "vitest";
import type { PluginInstallation } from "@/api/types";
import type { LevelChainItem } from "./useLibraryForm";
import {
contentLevelsForType,
hasMetadataProviderCapability,
hasChainProviders,
levelChainsFromResponse,
mergeChainWithDefaults,
} from "./useLibraryForm";
@@ -100,48 +98,22 @@ describe("mergeChainWithDefaults", () => {
});
});
describe("hasMetadataProviderCapability", () => {
const installation = (over: Partial<PluginInstallation>): PluginInstallation =>
({
id: 1,
plugin_id: "silo.tmdb",
version: "1.0.0",
install_path: "/x",
enabled: true,
capabilities: [],
global_config_schema: [],
user_config_schema: [],
routes: [],
assets: [],
global_configs: [],
auth_bindings: [],
task_bindings: [],
update_policy: "manual",
...over,
}) as PluginInstallation;
describe("hasChainProviders", () => {
const item = (slug: string): LevelChainItem => ({
plugin_installation_id: 1,
capability_id: slug,
provider_slug: slug,
enabled: false,
});
it("ignores disabled installations and non-metadata capabilities", () => {
expect(
hasMetadataProviderCapability([
installation({
id: 1,
enabled: false,
capabilities: [{ type: "metadata_provider.v1", id: "tvdb", display_name: "TVDB" }],
}),
installation({
id: 2,
capabilities: [{ type: "request_router.v1", id: "seerr", display_name: "Seerr" }],
}),
]),
).toBe(false);
it("is true when the server chain/defaults expose any provider, even disabled builtins", () => {
// A plugin-less server still seeds the built-in NFO provider (disabled):
// the chain editor must render instead of the "install a plugin" empty state.
expect(hasChainProviders({ movie: [item("nfo")] })).toBe(true);
});
expect(
hasMetadataProviderCapability([
installation({
id: 3,
capabilities: [{ type: "metadata_provider.v1", id: "tmdb", display_name: "TMDB" }],
}),
]),
).toBe(true);
it("is false only when the resolved chains are empty", () => {
expect(hasChainProviders({})).toBe(false);
expect(hasChainProviders({ movie: [], series: [] })).toBe(false);
});
});
web/src/components/admin/libraries/useLibraryForm.ts
+10 -18
View File
@@ -1,11 +1,6 @@
import { useMemo, useState } from "react";
import type {
CreateLibraryRequest,
Library,
LibraryProviderChainResponse,
PluginInstallation,
} from "@/api/types";
import type { CreateLibraryRequest, Library, LibraryProviderChainResponse } from "@/api/types";
import {
useCreateLibrary,
useLibraryProviderDefaults,
@@ -13,7 +8,6 @@ import {
useSetLibraryProviders,
useUpdateLibrary,
} from "@/hooks/queries/admin/libraries";
import { useAdminPlugins } from "@/hooks/queries/admin/plugins";
import { PROVIDER_TRAILER_KINDS } from "@/lib/extraKinds";
export type LevelChainItem = {
@@ -23,15 +17,14 @@ export type LevelChainItem = {
enabled: boolean;
};
// hasMetadataProviderCapability reports whether any enabled plugin installation
// exposes a metadata provider — used only to decide between the chain editor
// and the "install a plugin" empty state. The chain contents themselves come
// from the server.
export function hasMetadataProviderCapability(installations: PluginInstallation[]): boolean {
return installations.some(
(inst) =>
inst.enabled && (inst.capabilities ?? []).some((cap) => cap.type === "metadata_provider.v1"),
);
// hasChainProviders reports whether the server's resolved provider chains (a
// library's saved chain merged with the server-computed defaults) expose any
// provider — used only to decide between the chain editor and the "install a
// plugin" empty state. Built-in host providers (NFO) appear in the server
// response without any plugin installation, so this gate must not depend on
// installed plugins.
export function hasChainProviders(chains: Record<string, LevelChainItem[]>): boolean {
return Object.values(chains).some((items) => items.length > 0);
}
export interface LibraryFormErrors {
@@ -158,7 +151,6 @@ export function useLibraryForm({
const createMutation = useCreateLibrary();
const updateMutation = useUpdateLibrary();
const setChainMutation = useSetLibraryProviders();
const { installations } = useAdminPlugins();
const { data: currentChain } = useLibraryProviders(library?.id ?? null);
// The server computes default chains (same logic that seeds them on create),
// so the form never re-derives defaults from plugin manifests client-side.
@@ -346,7 +338,7 @@ export function useLibraryForm({
chainLoading,
reorderLevel,
toggleLevelProvider,
hasMetadataProviders: hasMetadataProviderCapability(installations),
hasMetadataProviders: hasChainProviders(activeLevelChains),
errors,
isPending,
submit,