Files
silo-server/internal/metadata/types.go
T
91e1164090 feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing

Phases A and B of the #216 local-NFO work, implemented test-first.

Registration & hint-first identity (Phase A):
- Migration seeds a reserved kind='builtin' silo.builtin installation
  and an 'nfo' metadata capability (default_enabled=false, priority 1
  for movie/series) with a partial unique index and documented Down.
- In-process builtin provider registry (internal/metadata/builtin.go);
  buildProviders returns the registered provider for builtin rows.
- Guard rails keep the reserved row out of every plugin surface (user
  plugin-settings, installations list, image resolvers, preload,
  auto-update, store Delete, mutation handlers -> 409); silo.builtin is
  a reserved manifest id.
- Startup sync materializes legacy content_level='' chains per level,
  then appends builtin capabilities disabled via
  AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy
  priority now respects default_enabled=false.
- NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider
  with per-mode conflict policy (stored IDs win on scheduled refresh,
  NFO wins on manual refresh, Identify skips NFO); ID-less candidates
  are excluded from provider-priority tie-breaks and nfo never counts
  as corroboration.
- Web chain-editor empty-state gate is now server-derived so builtin
  providers are reachable on plugin-less servers.

Parser breadth & sidecar hardening (Phase B):
- Parser covers the practical Kodi/Jellyfin field set for <movie> and
  <tvshow>: original title, tagline, runtime, dates, content rating,
  genres/studios/countries/tags, multi-source ratings with scale
  normalization, cast with roles/order, director/credits. Empty
  collections stay nil so merge early-returns apply.
- findNFO parses candidates and falls through on read/parse failure or
  root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo;
  GetMetadata gains the same ContentType guard Search has.
- New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate
  in merge (Go) and the edit-metadata dialog (web), closing the gap
  where a manual refresh re-applied NFO dates over admin corrections.
- Merge-contract tests pin NFO fill semantics, genres whole-list
  first-provider-wins, and NFO edits propagating on manual refresh only.
- Docs: new admin wiki page (supported fields, merge semantics,
  naming-supplies-structure contract), index bullet, sidecar wording
  revision, v1-scope feature-detection note.

Zero behavior change while the provider is disabled (default); pinned
by CI-mode and DB-gated test suites.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* feat(metadata): ingest local sidecar artwork and read series-depth NFO

Phases C and D of the #216 local-NFO work, implemented test-first, plus
the mixed-library use-case pins. Together these deliver the headline
case: a series absent from every remote database (e.g. a fitness
library) scans into a fully presented show -> named seasons -> titled
episodes tree from NFO files and sidecar art alone.

Local sidecar artwork through the S3 image cache (Phase C):
- The NFO provider implements ImageProvider: poster/backdrop/logo
  sidecar discovery with a fixed precedence map, symlink/non-regular
  rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic
  filenames apply only via the sidecar search paths, so a shared
  folder.jpg in a flat multi-movie directory applies to none.
- file:// becomes a live local source scheme: routed into *_source_path
  (never *_path), accepted by every image enqueue gate, attributed as
  provider "local", excluded from cached-path detection.
- The image-cache processor caches local files with lexical-on-logical
  confinement to the library roots, open-handle reads with re-checks,
  the same variant widths as remote art, and stable (7-day) failure
  classification. Keys land under
  local/{contentType}/{contentID}/{hash8}/{imageType}; superseded
  prefixes are cleaned on re-cache and item deletion.
- applyIfBetter gains a local exemption so rating-0 local art can fill
  matched items without being stickily displaced; ImageRequest carries
  additive sidecar path context.

Series depth (Phase D):
- SeasonsRequest/EpisodesRequest carry additive local path context
  (series roots, per-season directories, per-episode file paths),
  derived from naming at match time and reconstructed on refresh.
- season.nfo supplies season name/plot; NFO season numbers are advisory
  (directory-derived number wins with a Warn - naming owns structure).
  <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles
  episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy
  wins over NFO numbers.
- Episode NFOs work without a season.nfo (provider seasons unioned with
  on-disk seasons); SynthesizeFallbackEpisodes always runs after persist
  so NFO-less episodes keep synthesized rows. Season/episode file:// art
  rides the Phase C pipeline unchanged.
- Migration adds season:1/episode:1 to the builtin NFO capability's
  default_priority (still default_enabled=false).

Mixed sports-library use case (tests only, no product change):
- Pins the classification contract for one library holding movie-shaped
  and show-shaped content (WWE PPV events as movies next to a "WWE
  SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming
  decides movie-vs-series per file before any provider runs; the NFO
  supplies metadata/identity but never flips type (ContentType guard);
  the per-root Type override is the correction path.
- NFO-driven type classification at scan time is recorded as an explicit
  deferred open question.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* docs(metadata): document local NFO metadata architecture

Add a single as-built architecture page
(docs/architecture/local-nfo-metadata.md) for the #216 local-NFO
feature: the builtin registration model, hint-first identity semantics,
the file:// -> S3 artwork pipeline and its deployment constraint, series
depth, the mixed-library classification contract, and known limitations.

This replaces the working implementation plan, the per-phase specs, and
the narrow sidecar-artwork note, which were planning drafts and are left
untracked; admin-facing behavior remains in the wiki.

Part of #216

AI-use disclosure: planned, drafted, and consolidated with Claude Code
(Fable 5) using multi-agent exploration and adversarial review.

* fix(metadata): address PR review findings on NFO builtin provider

Fold in the valid, low-risk fixes surfaced by automated review on #390:

- imagecache: extract validateCacheRequest so CacheBytes (the local
  sidecar season/episode path) enforces the same episode-requires-season
  guard as Cache, preventing distinct episodes' art from colliding under
  one S3 key.
- image_cache_processor: close the sidecar symlink-swap window by
  rejecting the opened handle unless os.SameFile matches the Lstat'd
  file, so a leaf swapped to a symlink can't pull an out-of-root target
  into the public cache.
- plugins: guard the reserved builtin installation row in the store's
  Update, matching Delete, so its version/enabled/capabilities can never
  be rewritten even if a mutation slips past the HTTP layer.
- cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck
  DB round-trip fails fast at startup instead of hanging.
- metadata: panic instead of silently no-op'ing on an invalid
  RegisterBuiltinProvider call (init-time programmer error).
- docs: correct the media-folder-and-naming NFO paragraph to state
  season/episode NFOs and sidecar artwork are actively read.

---------

Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
2026-07-16 17:55:36 -04:00

369 lines
12 KiB
Go

package metadata
import (
"context"
"github.com/Silo-Server/silo-server/internal/models"
)
// RefreshMode determines how the pipeline processes an item.
type RefreshMode int
const (
ModeInitialMatch RefreshMode = iota // New file discovered by scanner
ModeScheduledRefresh // Background periodic refresh
ModeManualRefresh // User-triggered refresh
ModeIdentify // User manually identified item
)
// MergeMode determines how provider results merge into existing items.
type MergeMode int
const (
MergeFillEmpty MergeMode = iota // Only fill zero/empty fields
MergeReplaceUnlocked // Replace all unlocked fields
)
// MetadataField identifies a lockable metadata field.
type MetadataField int
const (
FieldName MetadataField = iota
FieldOverview
FieldGenres
FieldStudios
FieldCast
FieldCrew
FieldRating
FieldRuntime
FieldTags
FieldContentRating
FieldImages
FieldAirSchedule
FieldVideos
// FieldReleaseDates locks Year, ReleaseDate, and First/LastAirDate so a
// refresh (or an NFO at priority 1) cannot overwrite manual corrections.
FieldReleaseDates
)
// RefreshPriority controls queue ordering.
type RefreshPriority int
const (
PriorityHigh RefreshPriority = 0 // Manual refresh, identify
PriorityNormal RefreshPriority = 1 // Newly matched items
PriorityLow RefreshPriority = 2 // Scheduled background refresh
)
// ProcessRequest is the unified input to MetadataService.Process().
type ProcessRequest struct {
ContentID string // Existing item (refresh/identify) or empty (new match)
Hints *MatchHints // From scanner (initial match only)
ProviderIDs map[string]string // From user selection (identify only)
FolderID string // Which library folder
Language string // ISO 639-1 metadata language (resolved from folder)
Mode RefreshMode
// AdoptLanguage makes Language the item's new canonical metadata
// language: the base row is rewritten in Language and
// default_metadata_language is restamped, instead of routing a
// non-canonical language to the localization tables. Set by
// folder-scoped manual refreshes when the library's configured
// language differs from the item's stamp, so changing a library's
// metadata language actually re-fetches titles/overviews.
AdoptLanguage bool
}
// ProcessResult is the output of MetadataService.Process().
type ProcessResult struct {
ContentID string
IsNew bool // True if this was an initial match (new item created)
Updated bool // True if any fields were changed
}
// MatchHints carries scanner-extracted data into the pipeline.
// Adapted from matcher.MatchHints with FilePath added for local providers.
type MatchHints struct {
ContentID string
FileHash string
TmdbID string
TvdbID string
ImdbID string
Title string
Year int
Type string // "movie" or "series"
SeasonNum int
EpisodeNum int
HintSource string // "xattr", "nfo", "folder", "filename"
FilePath string // Absolute path to media file (legacy local-provider field)
RepresentativeFilePath string
ObservedRootPath string
AllGroupFilePaths []string
PrimarySidecarSearchPaths []string
}
// SearchQuery is passed to SearchProvider.Search().
type SearchQuery struct {
Title string
Author string // optional creator hint (e.g. ebook author) folded into the search query
Year int
ContentType string // media_items.type value
ProviderIDs map[string]string // Accumulated IDs from prior providers
Language string // ISO 639-1 code from library preference
FilePath string
RepresentativeFilePath string
ObservedRootPath string
AllGroupFilePaths []string
PrimarySidecarSearchPaths []string
}
// SearchResult is returned from SearchProvider.Search().
type SearchResult struct {
Name string
Year int
ProviderIDs map[string]string
ImageURL string
Overview string
Provider string // Slug of the provider that returned this
}
// MetadataRequest is passed to MetadataProvider.GetMetadata().
type MetadataRequest struct {
ProviderIDs map[string]string
ContentType string
Language string
FilePath string // Populated for local providers, empty for remote
RepresentativeFilePath string
ObservedRootPath string
AllGroupFilePaths []string
PrimarySidecarSearchPaths []string
GroupTitle string
GroupYear int
}
// PersonDetailRequest is passed to person detail lookups.
type PersonDetailRequest struct {
ProviderIDs map[string]string
Language string
}
// PersonDetailResult carries person-level metadata from a provider.
type PersonDetailResult struct {
Name string
SortName string
Bio string
BirthDate string
DeathDate string
Birthplace string
Homepage string
PhotoPath string
PhotoSourcePath string
PhotoThumbhash string
ProviderIDs map[string]string
}
// MetadataResult carries structured metadata from a single provider.
type MetadataResult struct {
HasMetadata bool
ProviderIDs map[string]string
Title string
OriginalTitle string
SortTitle string
Overview string
Tagline string
Year int
Runtime int
Genres []string
Studios []string
Networks []string
Countries []string
Keywords []string
OriginalLanguage string
ContentRating string
Ratings Ratings
People []models.ItemPerson
// Images (S3 paths or URLs).
PosterPath string
PosterThumbhash string
BackdropPath string
BackdropThumbhash string
LogoPath string
// Series-specific
ReleaseDate string
SeasonCount int
FirstAirDate string
LastAirDate string
AirTime string
AirTimezone string
// ShowStatus is the publication/airing status ("Ongoing", "Completed",
// "Continuing", "Ended") when the provider reports one.
ShowStatus string
// Videos are remote promotional/supplemental videos (trailers, teasers,
// ...) hosted on external sites. Accumulated across providers with
// (Provider, ProviderKey) dedup, gated by FieldVideos and the library's
// trailer_kinds allow-list at persist time.
Videos []RemoteVideo
}
// RemoteVideo describes a provider-reported external video (YouTube trailer
// etc.). It mirrors the SDK's VideoRecord and persists as an item_videos row.
type RemoteVideo struct {
Provider string // provider slug that returned this video
ProviderKey string // provider-native video id (dedup key)
Kind models.ExtraKind
Site string // hosting site, e.g. "youtube"
SiteKey string // site-native video key (YouTube video id)
Name string
Language string // ISO 639-1, empty when unknown
IsOfficial bool
SizeHint int // vertical resolution hint (e.g. 1080), 0 when unknown
PublishedAt string // RFC 3339, empty when unknown
}
// Ratings holds ratings from multiple sources.
type Ratings struct {
IMDB float64
TMDB float64
RTCritic float64
RTAudience float64
}
// ImageRequest is passed to ImageProvider.GetImages().
type ImageRequest struct {
ProviderIDs map[string]string
ContentType string
Language string
// Local sidecar context (additive; empty for purely remote providers).
// RepresentativeFilePath is the group's representative media file,
// AllGroupFilePaths are every file in the content group, and
// PrimarySidecarSearchPaths are directories already validated as safe for
// directory-level sidecars (single-content observed roots).
RepresentativeFilePath string
AllGroupFilePaths []string
PrimarySidecarSearchPaths []string
}
// RemoteImage describes an available image from a provider.
type RemoteImage struct {
ProviderID string // Slug of the provider that returned this image
URL string
Type ImageType
Language string
Width int
Height int
Rating float64 // Vote average for ordering
}
// ImageType classifies image purpose.
type ImageType int
const (
ImagePoster ImageType = iota
ImageBackdrop
ImageLogo
ImageStill // Episode stills
ImageProfile
)
// CacheImageRequest describes an image to be cached. For season posters
// and episode stills, ContentID is the parent series's ID and the
// SeasonNumber / EpisodeNumber fields scope the S3 key so siblings do not
// collide. Both pointers are nil for item-level images.
type CacheImageRequest struct {
SourceURL string
ProviderID string
ContentType string // "movies" or "series"
ContentID string
ImageType ImageType
SeasonNumber *int
EpisodeNumber *int
Language string
// KeyDiscriminator, when set, is inserted into the S3 key between the
// content ID and the image type (e.g. the 8-hex content hash of a local
// sidecar file) so re-cached art rotates to a fresh key.
KeyDiscriminator string
}
// CacheImageResult is returned by ImageCacher on success.
type CacheImageResult struct {
BasePath string // image-type prefix retained for legacy callers
OriginalPath string // exact immutable original-variant object key
Revision string // content revision shared by all generated variants
Thumbhash string // base64-encoded
Ext string // encoded file extension including dot
UploadedVariants int
ExistingVariants int
}
// ImageCacher caches a remote image to object storage.
type ImageCacher interface {
CacheImage(ctx context.Context, req CacheImageRequest) (*CacheImageResult, error)
}
// ImageByteCacher caches an image already read into memory (e.g. a local
// sidecar file) to object storage. Implemented by imagecache.Cacher alongside
// ImageCacher; the processor type-asserts it for file:// sources.
type ImageByteCacher interface {
CacheImageBytes(ctx context.Context, data []byte, req CacheImageRequest) (*CacheImageResult, error)
}
type ImageCacheJobEnqueuer interface {
Enqueue(ctx context.Context, in EnqueueImageCacheJobInput) error
EnqueueBatch(ctx context.Context, inputs []EnqueueImageCacheJobInput) (int, error)
}
// SeasonsRequest is passed to EpisodeProvider.GetSeasons().
type SeasonsRequest struct {
ProviderIDs map[string]string
ContentType string
Language string
// Local sidecar context (additive, internal — mirrors MetadataRequest).
// SeriesRootPaths are directories already validated as safe for
// directory-level sidecars (the series root); SeasonDirectoryPaths maps
// each directory-derived season number to its candidate directories.
// Both are empty for purely remote providers, which key off ProviderIDs.
SeriesRootPaths []string
SeasonDirectoryPaths map[int][]string
}
// EpisodesRequest is passed to EpisodeProvider.GetEpisodes().
type EpisodesRequest struct {
ProviderIDs map[string]string
SeasonNumber int
Language string
// Local sidecar context (additive, internal). SeriesRootPaths as on
// SeasonsRequest; EpisodeFilePaths maps each filename-derived episode
// number within SeasonNumber to that episode's media file paths.
SeriesRootPaths []string
EpisodeFilePaths map[int][]string
}
// SeasonResult carries season data from a provider.
type SeasonResult struct {
ContentID string
SeasonNumber int
Title string
Overview string
AirDate string
PosterPath string
PosterSourcePath string
PosterThumbhash string
Episodes []EpisodeResult
}
// EpisodeResult carries episode data from a provider.
type EpisodeResult struct {
ContentID string
ProviderIDs map[string]string
SeasonNumber int
EpisodeNumber int
Title string
Overview string
AirDate string
Runtime int
Ratings Ratings
StillPath string
StillSourcePath string
StillThumbhash string
}