* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
139 lines
5.5 KiB
Go
139 lines
5.5 KiB
Go
package metadata
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
|
|
"github.com/Silo-Server/silo-server/internal/catalog"
|
|
"github.com/Silo-Server/silo-server/internal/contentid"
|
|
)
|
|
|
|
// providerIDsStruct adapts the denormalized provider-id map carried on a
|
|
// MetadataResult to the contentid.ProviderIDs shape used for id derivation.
|
|
func providerIDsStruct(m map[string]string) contentid.ProviderIDs {
|
|
return contentid.ProviderIDs{
|
|
Tmdb: m["tmdb"],
|
|
Imdb: m["imdb"],
|
|
Tvdb: m["tvdb"],
|
|
}
|
|
}
|
|
|
|
// canonicalizeLocalContentID promotes a local skeleton to its deterministic,
|
|
// provider-anchored content_id once a confirmed match has supplied provider IDs
|
|
// (the untagged-then-matched re-ID). It returns the canonical id, which equals
|
|
// from when there is nothing to do.
|
|
//
|
|
// Tagged content and every refresh hit only the IsLocal guard, so the common
|
|
// path is free. When there is work to do, the promotion is one of:
|
|
//
|
|
// - target already taken: the two are the same logical item, so merge this
|
|
// skeleton onto the existing row using the shared rebind machinery; or
|
|
// - target free: rename in place. FK children move via ON UPDATE CASCADE (see
|
|
// migration 20260614120000_content_id_online_reid), so for a fresh skeleton
|
|
// this is a handful of rows, not the full-table remap the bulk migration does.
|
|
//
|
|
// It must run with the provider-dedup lock held (mergeAndPersist holds it), so a
|
|
// concurrent match of the same title cannot claim the target underneath us. The
|
|
// rename is self-healing: if it loses a rare race with skeleton creation and the
|
|
// unique constraint fires, the match returns an error, retries, and takes the
|
|
// merge branch on the next pass.
|
|
//
|
|
// Note: a series that already had season/episode rows before it matched keeps
|
|
// those children on their Sonyflake ids (ForSeason/ForEpisode need a series
|
|
// anchor). At first match the children usually do not exist yet, so this is
|
|
// rare; re-deriving them is a deferred follow-up (recomposeSeriesChildIDs).
|
|
func (s *MetadataService) canonicalizeLocalContentID(
|
|
ctx context.Context,
|
|
from string,
|
|
ids contentid.ProviderIDs,
|
|
itemType string,
|
|
) (string, error) {
|
|
if s == nil || !contentid.IsLocal(from) {
|
|
return from, nil
|
|
}
|
|
return s.reanchorContentID(ctx, from, ids, itemType)
|
|
}
|
|
|
|
// reanchorContentID moves `from` to the provider-anchored content_id derived
|
|
// from `ids` when the two differ, merging onto an existing target or renaming a
|
|
// free one. It is the shared core behind two callers: the local-skeleton
|
|
// promotion (canonicalizeLocalContentID) and the manual-refresh corrected-
|
|
// identity re-anchor (a fixed <uniqueid> in an NFO whose item was already
|
|
// anchored to the wrong provider id). Both must hold the provider-dedup lock so
|
|
// the target id cannot be claimed underneath us. When ids do not derive a
|
|
// provider-anchored id, or it equals `from`, this is a no-op.
|
|
func (s *MetadataService) reanchorContentID(
|
|
ctx context.Context,
|
|
from string,
|
|
ids contentid.ProviderIDs,
|
|
itemType string,
|
|
) (string, error) {
|
|
if s == nil {
|
|
return from, nil
|
|
}
|
|
|
|
// Derive with no path fallback: we only want a provider-anchored id here, not
|
|
// another local value.
|
|
target, err := deriveLogicalContentID(itemType, ids, "")
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
if !contentid.IsProviderAnchored(target) || target == from {
|
|
return from, nil
|
|
}
|
|
|
|
// Look up the target, distinguishing "free" (not-found) from a transient
|
|
// failure. Treating a real error as "target free" would fall through to the
|
|
// rename path and surface as a misleading unique-constraint conflict instead
|
|
// of a retryable error.
|
|
existing, err := s.itemRepo.GetByID(ctx, target)
|
|
switch {
|
|
case err == nil && existing != nil:
|
|
// A row already at the target id is the same logical item; merge onto it.
|
|
// allowMatchedSource: this also runs when refreshing an already-matched
|
|
// local item, so the source row may be 'matched'; we still consolidate it
|
|
// onto the canonical target rather than orphaning a duplicate. Safe under
|
|
// the provider-dedup lock the caller holds.
|
|
if err := s.rebindItemToExistingItem(ctx, from, target, true); err != nil {
|
|
return "", fmt.Errorf("merging local item %s into %s: %w", from, target, err)
|
|
}
|
|
return target, nil
|
|
case err != nil && !errors.Is(err, catalog.ErrItemNotFound):
|
|
return "", fmt.Errorf("looking up canonical target %s: %w", target, err)
|
|
}
|
|
|
|
// Target free: pure value-move.
|
|
if err := s.renameContentID(ctx, from, target); err != nil {
|
|
return "", err
|
|
}
|
|
return target, nil
|
|
}
|
|
|
|
// renameContentID moves a single content_id value (a media item, season or
|
|
// episode) to a new, currently-free id. FK children follow via ON UPDATE
|
|
// CASCADE; the silo_rename_content_id function also sweeps the unconstrained
|
|
// soft references. The function body runs as one statement, so the move is
|
|
// atomic.
|
|
func (s *MetadataService) renameContentID(ctx context.Context, from, to string) error {
|
|
if s.dbPool == nil {
|
|
return fmt.Errorf("rename content id requires database pool")
|
|
}
|
|
tx, err := s.dbPool.Begin(ctx)
|
|
if err != nil {
|
|
return fmt.Errorf("begin content_id rename transaction: %w", err)
|
|
}
|
|
defer func() { _ = tx.Rollback(ctx) }()
|
|
|
|
if _, err := tx.Exec(ctx, `SELECT silo_rename_content_id($1, $2)`, from, to); err != nil {
|
|
return fmt.Errorf("rename content_id %s -> %s: %w", from, to, err)
|
|
}
|
|
if err := catalog.EnqueueSearchIndexRename(ctx, tx, from, to); err != nil {
|
|
return fmt.Errorf("enqueue catalog search rename %s -> %s: %w", from, to, err)
|
|
}
|
|
if err := tx.Commit(ctx); err != nil {
|
|
return fmt.Errorf("commit content_id rename transaction: %w", err)
|
|
}
|
|
return nil
|
|
}
|