* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
369 lines
12 KiB
Go
369 lines
12 KiB
Go
package metadata
|
|
|
|
import (
|
|
"context"
|
|
|
|
"github.com/Silo-Server/silo-server/internal/models"
|
|
)
|
|
|
|
// RefreshMode determines how the pipeline processes an item.
|
|
type RefreshMode int
|
|
|
|
const (
|
|
ModeInitialMatch RefreshMode = iota // New file discovered by scanner
|
|
ModeScheduledRefresh // Background periodic refresh
|
|
ModeManualRefresh // User-triggered refresh
|
|
ModeIdentify // User manually identified item
|
|
)
|
|
|
|
// MergeMode determines how provider results merge into existing items.
|
|
type MergeMode int
|
|
|
|
const (
|
|
MergeFillEmpty MergeMode = iota // Only fill zero/empty fields
|
|
MergeReplaceUnlocked // Replace all unlocked fields
|
|
)
|
|
|
|
// MetadataField identifies a lockable metadata field.
|
|
type MetadataField int
|
|
|
|
const (
|
|
FieldName MetadataField = iota
|
|
FieldOverview
|
|
FieldGenres
|
|
FieldStudios
|
|
FieldCast
|
|
FieldCrew
|
|
FieldRating
|
|
FieldRuntime
|
|
FieldTags
|
|
FieldContentRating
|
|
FieldImages
|
|
FieldAirSchedule
|
|
FieldVideos
|
|
// FieldReleaseDates locks Year, ReleaseDate, and First/LastAirDate so a
|
|
// refresh (or an NFO at priority 1) cannot overwrite manual corrections.
|
|
FieldReleaseDates
|
|
)
|
|
|
|
// RefreshPriority controls queue ordering.
|
|
type RefreshPriority int
|
|
|
|
const (
|
|
PriorityHigh RefreshPriority = 0 // Manual refresh, identify
|
|
PriorityNormal RefreshPriority = 1 // Newly matched items
|
|
PriorityLow RefreshPriority = 2 // Scheduled background refresh
|
|
)
|
|
|
|
// ProcessRequest is the unified input to MetadataService.Process().
|
|
type ProcessRequest struct {
|
|
ContentID string // Existing item (refresh/identify) or empty (new match)
|
|
Hints *MatchHints // From scanner (initial match only)
|
|
ProviderIDs map[string]string // From user selection (identify only)
|
|
FolderID string // Which library folder
|
|
Language string // ISO 639-1 metadata language (resolved from folder)
|
|
Mode RefreshMode
|
|
// AdoptLanguage makes Language the item's new canonical metadata
|
|
// language: the base row is rewritten in Language and
|
|
// default_metadata_language is restamped, instead of routing a
|
|
// non-canonical language to the localization tables. Set by
|
|
// folder-scoped manual refreshes when the library's configured
|
|
// language differs from the item's stamp, so changing a library's
|
|
// metadata language actually re-fetches titles/overviews.
|
|
AdoptLanguage bool
|
|
}
|
|
|
|
// ProcessResult is the output of MetadataService.Process().
|
|
type ProcessResult struct {
|
|
ContentID string
|
|
IsNew bool // True if this was an initial match (new item created)
|
|
Updated bool // True if any fields were changed
|
|
}
|
|
|
|
// MatchHints carries scanner-extracted data into the pipeline.
|
|
// Adapted from matcher.MatchHints with FilePath added for local providers.
|
|
type MatchHints struct {
|
|
ContentID string
|
|
FileHash string
|
|
TmdbID string
|
|
TvdbID string
|
|
ImdbID string
|
|
Title string
|
|
Year int
|
|
Type string // "movie" or "series"
|
|
SeasonNum int
|
|
EpisodeNum int
|
|
HintSource string // "xattr", "nfo", "folder", "filename"
|
|
FilePath string // Absolute path to media file (legacy local-provider field)
|
|
RepresentativeFilePath string
|
|
ObservedRootPath string
|
|
AllGroupFilePaths []string
|
|
PrimarySidecarSearchPaths []string
|
|
}
|
|
|
|
// SearchQuery is passed to SearchProvider.Search().
|
|
type SearchQuery struct {
|
|
Title string
|
|
Author string // optional creator hint (e.g. ebook author) folded into the search query
|
|
Year int
|
|
ContentType string // media_items.type value
|
|
ProviderIDs map[string]string // Accumulated IDs from prior providers
|
|
Language string // ISO 639-1 code from library preference
|
|
FilePath string
|
|
RepresentativeFilePath string
|
|
ObservedRootPath string
|
|
AllGroupFilePaths []string
|
|
PrimarySidecarSearchPaths []string
|
|
}
|
|
|
|
// SearchResult is returned from SearchProvider.Search().
|
|
type SearchResult struct {
|
|
Name string
|
|
Year int
|
|
ProviderIDs map[string]string
|
|
ImageURL string
|
|
Overview string
|
|
Provider string // Slug of the provider that returned this
|
|
}
|
|
|
|
// MetadataRequest is passed to MetadataProvider.GetMetadata().
|
|
type MetadataRequest struct {
|
|
ProviderIDs map[string]string
|
|
ContentType string
|
|
Language string
|
|
FilePath string // Populated for local providers, empty for remote
|
|
RepresentativeFilePath string
|
|
ObservedRootPath string
|
|
AllGroupFilePaths []string
|
|
PrimarySidecarSearchPaths []string
|
|
GroupTitle string
|
|
GroupYear int
|
|
}
|
|
|
|
// PersonDetailRequest is passed to person detail lookups.
|
|
type PersonDetailRequest struct {
|
|
ProviderIDs map[string]string
|
|
Language string
|
|
}
|
|
|
|
// PersonDetailResult carries person-level metadata from a provider.
|
|
type PersonDetailResult struct {
|
|
Name string
|
|
SortName string
|
|
Bio string
|
|
BirthDate string
|
|
DeathDate string
|
|
Birthplace string
|
|
Homepage string
|
|
PhotoPath string
|
|
PhotoSourcePath string
|
|
PhotoThumbhash string
|
|
ProviderIDs map[string]string
|
|
}
|
|
|
|
// MetadataResult carries structured metadata from a single provider.
|
|
type MetadataResult struct {
|
|
HasMetadata bool
|
|
ProviderIDs map[string]string
|
|
Title string
|
|
OriginalTitle string
|
|
SortTitle string
|
|
Overview string
|
|
Tagline string
|
|
Year int
|
|
Runtime int
|
|
Genres []string
|
|
Studios []string
|
|
Networks []string
|
|
Countries []string
|
|
Keywords []string
|
|
OriginalLanguage string
|
|
ContentRating string
|
|
Ratings Ratings
|
|
People []models.ItemPerson
|
|
// Images (S3 paths or URLs).
|
|
PosterPath string
|
|
PosterThumbhash string
|
|
BackdropPath string
|
|
BackdropThumbhash string
|
|
LogoPath string
|
|
// Series-specific
|
|
ReleaseDate string
|
|
SeasonCount int
|
|
FirstAirDate string
|
|
LastAirDate string
|
|
AirTime string
|
|
AirTimezone string
|
|
// ShowStatus is the publication/airing status ("Ongoing", "Completed",
|
|
// "Continuing", "Ended") when the provider reports one.
|
|
ShowStatus string
|
|
// Videos are remote promotional/supplemental videos (trailers, teasers,
|
|
// ...) hosted on external sites. Accumulated across providers with
|
|
// (Provider, ProviderKey) dedup, gated by FieldVideos and the library's
|
|
// trailer_kinds allow-list at persist time.
|
|
Videos []RemoteVideo
|
|
}
|
|
|
|
// RemoteVideo describes a provider-reported external video (YouTube trailer
|
|
// etc.). It mirrors the SDK's VideoRecord and persists as an item_videos row.
|
|
type RemoteVideo struct {
|
|
Provider string // provider slug that returned this video
|
|
ProviderKey string // provider-native video id (dedup key)
|
|
Kind models.ExtraKind
|
|
Site string // hosting site, e.g. "youtube"
|
|
SiteKey string // site-native video key (YouTube video id)
|
|
Name string
|
|
Language string // ISO 639-1, empty when unknown
|
|
IsOfficial bool
|
|
SizeHint int // vertical resolution hint (e.g. 1080), 0 when unknown
|
|
PublishedAt string // RFC 3339, empty when unknown
|
|
}
|
|
|
|
// Ratings holds ratings from multiple sources.
|
|
type Ratings struct {
|
|
IMDB float64
|
|
TMDB float64
|
|
RTCritic float64
|
|
RTAudience float64
|
|
}
|
|
|
|
// ImageRequest is passed to ImageProvider.GetImages().
|
|
type ImageRequest struct {
|
|
ProviderIDs map[string]string
|
|
ContentType string
|
|
Language string
|
|
// Local sidecar context (additive; empty for purely remote providers).
|
|
// RepresentativeFilePath is the group's representative media file,
|
|
// AllGroupFilePaths are every file in the content group, and
|
|
// PrimarySidecarSearchPaths are directories already validated as safe for
|
|
// directory-level sidecars (single-content observed roots).
|
|
RepresentativeFilePath string
|
|
AllGroupFilePaths []string
|
|
PrimarySidecarSearchPaths []string
|
|
}
|
|
|
|
// RemoteImage describes an available image from a provider.
|
|
type RemoteImage struct {
|
|
ProviderID string // Slug of the provider that returned this image
|
|
URL string
|
|
Type ImageType
|
|
Language string
|
|
Width int
|
|
Height int
|
|
Rating float64 // Vote average for ordering
|
|
}
|
|
|
|
// ImageType classifies image purpose.
|
|
type ImageType int
|
|
|
|
const (
|
|
ImagePoster ImageType = iota
|
|
ImageBackdrop
|
|
ImageLogo
|
|
ImageStill // Episode stills
|
|
ImageProfile
|
|
)
|
|
|
|
// CacheImageRequest describes an image to be cached. For season posters
|
|
// and episode stills, ContentID is the parent series's ID and the
|
|
// SeasonNumber / EpisodeNumber fields scope the S3 key so siblings do not
|
|
// collide. Both pointers are nil for item-level images.
|
|
type CacheImageRequest struct {
|
|
SourceURL string
|
|
ProviderID string
|
|
ContentType string // "movies" or "series"
|
|
ContentID string
|
|
ImageType ImageType
|
|
SeasonNumber *int
|
|
EpisodeNumber *int
|
|
Language string
|
|
// KeyDiscriminator, when set, is inserted into the S3 key between the
|
|
// content ID and the image type (e.g. the 8-hex content hash of a local
|
|
// sidecar file) so re-cached art rotates to a fresh key.
|
|
KeyDiscriminator string
|
|
}
|
|
|
|
// CacheImageResult is returned by ImageCacher on success.
|
|
type CacheImageResult struct {
|
|
BasePath string // image-type prefix retained for legacy callers
|
|
OriginalPath string // exact immutable original-variant object key
|
|
Revision string // content revision shared by all generated variants
|
|
Thumbhash string // base64-encoded
|
|
Ext string // encoded file extension including dot
|
|
|
|
UploadedVariants int
|
|
ExistingVariants int
|
|
}
|
|
|
|
// ImageCacher caches a remote image to object storage.
|
|
type ImageCacher interface {
|
|
CacheImage(ctx context.Context, req CacheImageRequest) (*CacheImageResult, error)
|
|
}
|
|
|
|
// ImageByteCacher caches an image already read into memory (e.g. a local
|
|
// sidecar file) to object storage. Implemented by imagecache.Cacher alongside
|
|
// ImageCacher; the processor type-asserts it for file:// sources.
|
|
type ImageByteCacher interface {
|
|
CacheImageBytes(ctx context.Context, data []byte, req CacheImageRequest) (*CacheImageResult, error)
|
|
}
|
|
|
|
type ImageCacheJobEnqueuer interface {
|
|
Enqueue(ctx context.Context, in EnqueueImageCacheJobInput) error
|
|
EnqueueBatch(ctx context.Context, inputs []EnqueueImageCacheJobInput) (int, error)
|
|
}
|
|
|
|
// SeasonsRequest is passed to EpisodeProvider.GetSeasons().
|
|
type SeasonsRequest struct {
|
|
ProviderIDs map[string]string
|
|
ContentType string
|
|
Language string
|
|
// Local sidecar context (additive, internal — mirrors MetadataRequest).
|
|
// SeriesRootPaths are directories already validated as safe for
|
|
// directory-level sidecars (the series root); SeasonDirectoryPaths maps
|
|
// each directory-derived season number to its candidate directories.
|
|
// Both are empty for purely remote providers, which key off ProviderIDs.
|
|
SeriesRootPaths []string
|
|
SeasonDirectoryPaths map[int][]string
|
|
}
|
|
|
|
// EpisodesRequest is passed to EpisodeProvider.GetEpisodes().
|
|
type EpisodesRequest struct {
|
|
ProviderIDs map[string]string
|
|
SeasonNumber int
|
|
Language string
|
|
// Local sidecar context (additive, internal). SeriesRootPaths as on
|
|
// SeasonsRequest; EpisodeFilePaths maps each filename-derived episode
|
|
// number within SeasonNumber to that episode's media file paths.
|
|
SeriesRootPaths []string
|
|
EpisodeFilePaths map[int][]string
|
|
}
|
|
|
|
// SeasonResult carries season data from a provider.
|
|
type SeasonResult struct {
|
|
ContentID string
|
|
SeasonNumber int
|
|
Title string
|
|
Overview string
|
|
AirDate string
|
|
PosterPath string
|
|
PosterSourcePath string
|
|
PosterThumbhash string
|
|
Episodes []EpisodeResult
|
|
}
|
|
|
|
// EpisodeResult carries episode data from a provider.
|
|
type EpisodeResult struct {
|
|
ContentID string
|
|
ProviderIDs map[string]string
|
|
SeasonNumber int
|
|
EpisodeNumber int
|
|
Title string
|
|
Overview string
|
|
AirDate string
|
|
Runtime int
|
|
Ratings Ratings
|
|
StillPath string
|
|
StillSourcePath string
|
|
StillThumbhash string
|
|
}
|