Files
silo-server/internal/metadata/merge.go
91e1164090 feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing

Phases A and B of the #216 local-NFO work, implemented test-first.

Registration & hint-first identity (Phase A):
- Migration seeds a reserved kind='builtin' silo.builtin installation
  and an 'nfo' metadata capability (default_enabled=false, priority 1
  for movie/series) with a partial unique index and documented Down.
- In-process builtin provider registry (internal/metadata/builtin.go);
  buildProviders returns the registered provider for builtin rows.
- Guard rails keep the reserved row out of every plugin surface (user
  plugin-settings, installations list, image resolvers, preload,
  auto-update, store Delete, mutation handlers -> 409); silo.builtin is
  a reserved manifest id.
- Startup sync materializes legacy content_level='' chains per level,
  then appends builtin capabilities disabled via
  AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy
  priority now respects default_enabled=false.
- NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider
  with per-mode conflict policy (stored IDs win on scheduled refresh,
  NFO wins on manual refresh, Identify skips NFO); ID-less candidates
  are excluded from provider-priority tie-breaks and nfo never counts
  as corroboration.
- Web chain-editor empty-state gate is now server-derived so builtin
  providers are reachable on plugin-less servers.

Parser breadth & sidecar hardening (Phase B):
- Parser covers the practical Kodi/Jellyfin field set for <movie> and
  <tvshow>: original title, tagline, runtime, dates, content rating,
  genres/studios/countries/tags, multi-source ratings with scale
  normalization, cast with roles/order, director/credits. Empty
  collections stay nil so merge early-returns apply.
- findNFO parses candidates and falls through on read/parse failure or
  root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo;
  GetMetadata gains the same ContentType guard Search has.
- New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate
  in merge (Go) and the edit-metadata dialog (web), closing the gap
  where a manual refresh re-applied NFO dates over admin corrections.
- Merge-contract tests pin NFO fill semantics, genres whole-list
  first-provider-wins, and NFO edits propagating on manual refresh only.
- Docs: new admin wiki page (supported fields, merge semantics,
  naming-supplies-structure contract), index bullet, sidecar wording
  revision, v1-scope feature-detection note.

Zero behavior change while the provider is disabled (default); pinned
by CI-mode and DB-gated test suites.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* feat(metadata): ingest local sidecar artwork and read series-depth NFO

Phases C and D of the #216 local-NFO work, implemented test-first, plus
the mixed-library use-case pins. Together these deliver the headline
case: a series absent from every remote database (e.g. a fitness
library) scans into a fully presented show -> named seasons -> titled
episodes tree from NFO files and sidecar art alone.

Local sidecar artwork through the S3 image cache (Phase C):
- The NFO provider implements ImageProvider: poster/backdrop/logo
  sidecar discovery with a fixed precedence map, symlink/non-regular
  rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic
  filenames apply only via the sidecar search paths, so a shared
  folder.jpg in a flat multi-movie directory applies to none.
- file:// becomes a live local source scheme: routed into *_source_path
  (never *_path), accepted by every image enqueue gate, attributed as
  provider "local", excluded from cached-path detection.
- The image-cache processor caches local files with lexical-on-logical
  confinement to the library roots, open-handle reads with re-checks,
  the same variant widths as remote art, and stable (7-day) failure
  classification. Keys land under
  local/{contentType}/{contentID}/{hash8}/{imageType}; superseded
  prefixes are cleaned on re-cache and item deletion.
- applyIfBetter gains a local exemption so rating-0 local art can fill
  matched items without being stickily displaced; ImageRequest carries
  additive sidecar path context.

Series depth (Phase D):
- SeasonsRequest/EpisodesRequest carry additive local path context
  (series roots, per-season directories, per-episode file paths),
  derived from naming at match time and reconstructed on refresh.
- season.nfo supplies season name/plot; NFO season numbers are advisory
  (directory-derived number wins with a Warn - naming owns structure).
  <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles
  episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy
  wins over NFO numbers.
- Episode NFOs work without a season.nfo (provider seasons unioned with
  on-disk seasons); SynthesizeFallbackEpisodes always runs after persist
  so NFO-less episodes keep synthesized rows. Season/episode file:// art
  rides the Phase C pipeline unchanged.
- Migration adds season:1/episode:1 to the builtin NFO capability's
  default_priority (still default_enabled=false).

Mixed sports-library use case (tests only, no product change):
- Pins the classification contract for one library holding movie-shaped
  and show-shaped content (WWE PPV events as movies next to a "WWE
  SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming
  decides movie-vs-series per file before any provider runs; the NFO
  supplies metadata/identity but never flips type (ContentType guard);
  the per-root Type override is the correction path.
- NFO-driven type classification at scan time is recorded as an explicit
  deferred open question.

Part of #216

AI-use disclosure: implemented with Claude Code (Fable 5) via
spec-driven TDD and agent-assisted implementation.

* docs(metadata): document local NFO metadata architecture

Add a single as-built architecture page
(docs/architecture/local-nfo-metadata.md) for the #216 local-NFO
feature: the builtin registration model, hint-first identity semantics,
the file:// -> S3 artwork pipeline and its deployment constraint, series
depth, the mixed-library classification contract, and known limitations.

This replaces the working implementation plan, the per-phase specs, and
the narrow sidecar-artwork note, which were planning drafts and are left
untracked; admin-facing behavior remains in the wiki.

Part of #216

AI-use disclosure: planned, drafted, and consolidated with Claude Code
(Fable 5) using multi-agent exploration and adversarial review.

* fix(metadata): address PR review findings on NFO builtin provider

Fold in the valid, low-risk fixes surfaced by automated review on #390:

- imagecache: extract validateCacheRequest so CacheBytes (the local
  sidecar season/episode path) enforces the same episode-requires-season
  guard as Cache, preventing distinct episodes' art from colliding under
  one S3 key.
- image_cache_processor: close the sidecar symlink-swap window by
  rejecting the opened handle unless os.SameFile matches the Lstat'd
  file, so a leaf swapped to a symlink can't pull an out-of-root target
  into the public cache.
- plugins: guard the reserved builtin installation row in the store's
  Update, matching Delete, so its version/enabled/capabilities can never
  be rewritten even if a mutation slips past the HTTP layer.
- cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck
  DB round-trip fails fast at startup instead of hanging.
- metadata: panic instead of silently no-op'ing on an invalid
  RegisterBuiltinProvider call (init-time programmer error).
- docs: correct the media-folder-and-naming NFO paragraph to state
  season/episode NFOs and sidecar artwork are actively read.

---------

Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
2026-07-16 17:55:36 -04:00

417 lines
13 KiB
Go

package metadata
import (
"slices"
"strconv"
"strings"
"github.com/Silo-Server/silo-server/internal/models"
)
// MergeMetadata merges source into target respecting locked fields and merge mode.
func MergeMetadata(source, target *MetadataResult, locked []MetadataField, mode MergeMode) {
if source == nil {
return
}
isLocked := func(f MetadataField) bool {
return slices.Contains(locked, f)
}
// Scalar fields
if !isLocked(FieldName) {
mergeScalar(&target.Title, source.Title, mode)
mergeScalar(&target.OriginalTitle, source.OriginalTitle, mode)
mergeScalar(&target.SortTitle, source.SortTitle, mode)
}
if !isLocked(FieldOverview) {
mergeScalar(&target.Overview, source.Overview, mode)
mergeScalar(&target.Tagline, source.Tagline, mode)
}
if !isLocked(FieldRuntime) {
mergeInt(&target.Runtime, source.Runtime, mode)
}
if !isLocked(FieldContentRating) {
mergeScalar(&target.ContentRating, source.ContentRating, mode)
}
if !isLocked(FieldRating) {
mergeFloat(&target.Ratings.IMDB, source.Ratings.IMDB, mode)
mergeFloat(&target.Ratings.TMDB, source.Ratings.TMDB, mode)
mergeFloat(&target.Ratings.RTCritic, source.Ratings.RTCritic, mode)
mergeFloat(&target.Ratings.RTAudience, source.Ratings.RTAudience, mode)
}
// Year and release/air dates lock together under FieldReleaseDates.
if !isLocked(FieldReleaseDates) {
mergeInt(&target.Year, source.Year, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
}
mergeScalar(&target.OriginalLanguage, source.OriginalLanguage, mode)
// Series fields (not lockable — structural)
mergeInt(&target.SeasonCount, source.SeasonCount, mode)
mergeScalar(&target.ShowStatus, source.ShowStatus, mode)
if !isLocked(FieldAirSchedule) {
mergeScalar(&target.AirTime, source.AirTime, mode)
mergeScalar(&target.AirTimezone, source.AirTimezone, mode)
}
// Genres follow provider priority during FillEmpty instead of unioning tags
// from later providers, which can create noisy hybrid classifications.
if !isLocked(FieldGenres) {
mergePrioritizedStringSlice(&target.Genres, source.Genres, mode)
}
// Accumulate other arrays
if !isLocked(FieldStudios) {
mergeStringSlice(&target.Studios, source.Studios, mode)
mergeStringSlice(&target.Networks, source.Networks, mode)
}
if !isLocked(FieldTags) {
mergeStringSlice(&target.Countries, source.Countries, mode)
mergeStringSlice(&target.Keywords, source.Keywords, mode)
}
// Smart merge people (unified cast/crew)
if !isLocked(FieldCast) || !isLocked(FieldCrew) {
mergePeople(&target.People, source.People, mode)
}
// Images
if !isLocked(FieldImages) {
mergeScalar(&target.PosterPath, source.PosterPath, mode)
mergeScalar(&target.PosterThumbhash, source.PosterThumbhash, mode)
mergeScalar(&target.BackdropPath, source.BackdropPath, mode)
mergeScalar(&target.BackdropThumbhash, source.BackdropThumbhash, mode)
mergeScalar(&target.LogoPath, source.LogoPath, mode)
}
// Remote videos accumulate across providers (TMDB trailers + another
// provider's clips can coexist), deduped by provider identity.
if !isLocked(FieldVideos) {
mergeVideos(&target.Videos, source.Videos)
}
// Provider IDs always accumulate, never overwrite
mergeProviderIDs(target, source)
}
// MergeGlobalMetadata merges only provider-invariant fields, preserving any
// language-specific presentation fields already stored on the target.
func MergeGlobalMetadata(source, target *MetadataResult, locked []MetadataField, mode MergeMode) {
if source == nil {
return
}
isLocked := func(f MetadataField) bool {
return slices.Contains(locked, f)
}
if !isLocked(FieldRuntime) {
mergeInt(&target.Runtime, source.Runtime, mode)
}
if !isLocked(FieldContentRating) {
mergeScalar(&target.ContentRating, source.ContentRating, mode)
}
if !isLocked(FieldRating) {
mergeFloat(&target.Ratings.IMDB, source.Ratings.IMDB, mode)
mergeFloat(&target.Ratings.TMDB, source.Ratings.TMDB, mode)
mergeFloat(&target.Ratings.RTCritic, source.Ratings.RTCritic, mode)
mergeFloat(&target.Ratings.RTAudience, source.Ratings.RTAudience, mode)
}
if !isLocked(FieldReleaseDates) {
mergeInt(&target.Year, source.Year, mode)
mergeScalar(&target.ReleaseDate, source.ReleaseDate, mode)
mergeScalar(&target.FirstAirDate, source.FirstAirDate, mode)
mergeScalar(&target.LastAirDate, source.LastAirDate, mode)
}
mergeScalar(&target.OriginalTitle, source.OriginalTitle, mode)
mergeScalar(&target.OriginalLanguage, source.OriginalLanguage, mode)
mergeInt(&target.SeasonCount, source.SeasonCount, mode)
mergeScalar(&target.ShowStatus, source.ShowStatus, mode)
if !isLocked(FieldAirSchedule) {
mergeScalar(&target.AirTime, source.AirTime, mode)
mergeScalar(&target.AirTimezone, source.AirTimezone, mode)
}
if !isLocked(FieldGenres) {
mergePrioritizedStringSlice(&target.Genres, source.Genres, mode)
}
if !isLocked(FieldStudios) {
mergeStringSlice(&target.Studios, source.Studios, mode)
mergeStringSlice(&target.Networks, source.Networks, mode)
}
if !isLocked(FieldTags) {
mergeStringSlice(&target.Countries, source.Countries, mode)
mergeStringSlice(&target.Keywords, source.Keywords, mode)
}
if !isLocked(FieldCast) || !isLocked(FieldCrew) {
mergePeople(&target.People, source.People, mode)
}
if !isLocked(FieldVideos) {
mergeVideos(&target.Videos, source.Videos)
}
mergeProviderIDs(target, source)
}
// mergeVideos accumulates source videos into target, skipping entries the
// target already has from the same provider (same provider video id) or that
// point at the same hosted video (same site/site_key). Unlike scalar fields
// this ignores MergeMode: the set is replaced wholesale at persist time each
// refresh, so within one refresh pass accumulation is always correct.
func mergeVideos(target *[]RemoteVideo, source []RemoteVideo) {
if len(source) == 0 {
return
}
seen := make(map[string]bool, len(*target)*2)
for _, v := range *target {
seen["p|"+v.Provider+"|"+v.ProviderKey] = true
seen["s|"+strings.ToLower(v.Site)+"|"+v.SiteKey] = true
}
for _, v := range source {
providerKey := "p|" + v.Provider + "|" + v.ProviderKey
siteKey := "s|" + strings.ToLower(v.Site) + "|" + v.SiteKey
if seen[providerKey] || (v.SiteKey != "" && seen[siteKey]) {
continue
}
seen[providerKey] = true
seen[siteKey] = true
*target = append(*target, v)
}
}
// MergeSeasonResult merges source into target using the standard metadata
// fallback contract for season-level fields.
func MergeSeasonResult(source, target *SeasonResult, mode MergeMode) {
if source == nil {
return
}
mergeScalar(&target.ContentID, source.ContentID, mode)
mergeInt(&target.SeasonNumber, source.SeasonNumber, mode)
mergeScalar(&target.Title, source.Title, mode)
mergeScalar(&target.Overview, source.Overview, mode)
mergeScalar(&target.AirDate, source.AirDate, mode)
mergeScalar(&target.PosterPath, source.PosterPath, mode)
mergeScalar(&target.PosterThumbhash, source.PosterThumbhash, mode)
}
// MergeEpisodeResult merges source into target using the standard metadata
// fallback contract for episode-level fields.
func MergeEpisodeResult(source, target *EpisodeResult, mode MergeMode) {
if source == nil {
return
}
mergeScalar(&target.ContentID, source.ContentID, mode)
mergeInt(&target.SeasonNumber, source.SeasonNumber, mode)
mergeInt(&target.EpisodeNumber, source.EpisodeNumber, mode)
mergeScalar(&target.Title, source.Title, mode)
mergeScalar(&target.Overview, source.Overview, mode)
mergeScalar(&target.AirDate, source.AirDate, mode)
mergeInt(&target.Runtime, source.Runtime, mode)
mergeFloat(&target.Ratings.IMDB, source.Ratings.IMDB, mode)
mergeFloat(&target.Ratings.TMDB, source.Ratings.TMDB, mode)
mergeFloat(&target.Ratings.RTCritic, source.Ratings.RTCritic, mode)
mergeFloat(&target.Ratings.RTAudience, source.Ratings.RTAudience, mode)
mergeScalar(&target.StillPath, source.StillPath, mode)
mergeScalar(&target.StillThumbhash, source.StillThumbhash, mode)
mergeProviderIDMap(&target.ProviderIDs, source.ProviderIDs)
}
// MergePersonDetail merges source into target using the standard metadata
// fallback contract for person-level fields.
func MergePersonDetail(source, target *PersonDetailResult, mode MergeMode) {
if source == nil {
return
}
mergeScalar(&target.Name, source.Name, mode)
mergeScalar(&target.SortName, source.SortName, mode)
mergeScalar(&target.Bio, source.Bio, mode)
mergeScalar(&target.BirthDate, source.BirthDate, mode)
mergeScalar(&target.DeathDate, source.DeathDate, mode)
mergeScalar(&target.Birthplace, source.Birthplace, mode)
mergeScalar(&target.Homepage, source.Homepage, mode)
mergeScalar(&target.PhotoPath, source.PhotoPath, mode)
mergeScalar(&target.PhotoSourcePath, source.PhotoSourcePath, mode)
mergeScalar(&target.PhotoThumbhash, source.PhotoThumbhash, mode)
mergeProviderIDMap(&target.ProviderIDs, source.ProviderIDs)
}
// mergeProviderIDs adds new IDs without overwriting existing ones.
func mergeProviderIDs(target, source *MetadataResult) {
if len(source.ProviderIDs) == 0 {
return
}
if target.ProviderIDs == nil {
target.ProviderIDs = make(map[string]string)
}
for k, v := range source.ProviderIDs {
if v == "" {
continue
}
if _, exists := target.ProviderIDs[k]; !exists {
target.ProviderIDs[k] = v
}
}
}
func mergeProviderIDMap(target *map[string]string, source map[string]string) {
if len(source) == 0 {
return
}
if *target == nil {
*target = make(map[string]string)
}
for key, value := range source {
if value == "" {
continue
}
if (*target)[key] == "" {
(*target)[key] = value
}
}
}
func mergeScalar(target *string, source string, mode MergeMode) {
if source == "" {
return
}
if mode == MergeReplaceUnlocked || *target == "" {
*target = source
}
}
func mergeInt(target *int, source int, mode MergeMode) {
if source == 0 {
return
}
if mode == MergeReplaceUnlocked || *target == 0 {
*target = source
}
}
func mergeFloat(target *float64, source float64, mode MergeMode) {
if source == 0 {
return
}
if mode == MergeReplaceUnlocked || *target == 0 {
*target = source
}
}
func mergeStringSlice(target *[]string, source []string, mode MergeMode) {
if len(source) == 0 {
return
}
if mode == MergeReplaceUnlocked {
*target = source
return
}
// FillEmpty: accumulate unique
if len(*target) == 0 {
*target = source
return
}
seen := make(map[string]bool, len(*target))
for _, s := range *target {
seen[strings.ToLower(s)] = true
}
for _, s := range source {
if !seen[strings.ToLower(s)] {
*target = append(*target, s)
seen[strings.ToLower(s)] = true
}
}
}
func mergePrioritizedStringSlice(target *[]string, source []string, mode MergeMode) {
if len(source) == 0 {
return
}
if mode == MergeReplaceUnlocked || len(*target) == 0 {
*target = source
}
}
func mergePeople(target *[]models.ItemPerson, source []models.ItemPerson, mode MergeMode) {
if len(source) == 0 {
return
}
if mode == MergeReplaceUnlocked {
*target = source
return
}
if len(*target) == 0 {
*target = source
return
}
byTmdbID := indexPeopleBy(*target, func(p models.ItemPerson) string { return p.TmdbID })
byTvdbID := indexPeopleBy(*target, func(p models.ItemPerson) string { return p.TvdbID })
byImdbID := indexPeopleBy(*target, func(p models.ItemPerson) string { return p.ImdbID })
byPlexGUID := indexPeopleBy(*target, func(p models.ItemPerson) string { return p.PlexGUID })
byNameKind := indexPeopleBy(*target, func(p models.ItemPerson) string {
return strings.ToLower(p.Name) + "|" + strconv.Itoa(int(p.Kind))
})
for _, sp := range source {
if idx, ok := findPersonMatch(sp, byTmdbID, byTvdbID, byImdbID, byPlexGUID, byNameKind); ok {
mergePersonFields(&(*target)[idx], sp)
} else {
*target = append(*target, sp)
}
}
}
func indexPeopleBy(people []models.ItemPerson, keyFn func(models.ItemPerson) string) map[string]int {
m := make(map[string]int, len(people))
for i, p := range people {
if key := keyFn(p); key != "" {
m[key] = i
}
}
return m
}
func findPersonMatch(sp models.ItemPerson, maps ...map[string]int) (int, bool) {
keys := []string{sp.TmdbID, sp.TvdbID, sp.ImdbID, sp.PlexGUID,
strings.ToLower(sp.Name) + "|" + strconv.Itoa(int(sp.Kind))}
for i, m := range maps {
if i < len(keys) {
if idx, ok := m[keys[i]]; ok && keys[i] != "" {
return idx, true
}
}
}
return 0, false
}
func mergePersonFields(dst *models.ItemPerson, src models.ItemPerson) {
if dst.TmdbID == "" {
dst.TmdbID = src.TmdbID
}
if dst.ImdbID == "" {
dst.ImdbID = src.ImdbID
}
if dst.TvdbID == "" {
dst.TvdbID = src.TvdbID
}
if dst.PlexGUID == "" {
dst.PlexGUID = src.PlexGUID
}
if dst.PhotoPath == "" {
dst.PhotoPath = src.PhotoPath
}
if dst.PhotoSourcePath == "" {
dst.PhotoSourcePath = src.PhotoSourcePath
}
if dst.PhotoThumbhash == "" {
dst.PhotoThumbhash = src.PhotoThumbhash
}
if dst.Character == "" {
dst.Character = src.Character
}
}