Files
silo-server/internal/models/media.go
881c96864b feat(playback): finalize platform-neutral protocol v3 (#567)
* docs(playback): add v3 neutral-contract finalization plan

Supersedes the wire-contract sections of the 2026-07-12 v3 plan: server-owned
attempt keys, delivery-keyed negotiation without Media3 engine names, tiered
capability evidence, neutral device/output context, track/quality replan
operations, audio-only planning, and coordinated no-back-compat rollout
across server, Android, Apple, and web.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): make v3 attempt keys server-owned and replace engines with deliveries

Contract core of the platform-neutral v3 finalization (plan sections 3.1
and 3.2), breaking on purpose — v3 is dark and all clients move together:

- Every PlanV3 now carries plan_attempt_key, an opaque server-computed
  token clients store and echo in attempted_plan_keys; ReplanRequestV3
  gains bounded local_mutations that the replan handler folds into the
  failed plan's key. Clients never hash anything.
- KotlinName() is deleted from DeliveryV3, StreamProtocolV3 and
  SubtitleModeV3; the attempt-key canonical string now uses lowercase
  wire tokens, and PlanRecipeVersionV3 bumps to v3.3 so no key or plan
  ID computed under the old canonicalization can collide.
- EngineV3 leaves the wire: ClientPlaybackContextV3.Engines (media3_*)
  becomes Deliveries keyed original_http|progressive|hls, with
  EngineCapabilityV3 renamed DeliveryCapabilityV3. PlanV3.Engine is
  removed; the planner, subtitle policy and quirk registry re-key on
  delivery class, and the media3_only feature token is deleted.
- Validated-claim strings drop the prefix: media3_h264_decode ->
  h264_decode, media3_audio_decode -> audio_decode.
- Golden fixtures in testdata/protocol_v3 are regenerated by Go and are
  now the cross-repo source of truth.

Part of the playback protocol v3 neutral-contract train (steps 2-3 of
docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): add v3 evidence tiers and neutral device/output context

Implement plan sections 3.3 and 3.4 of the v3 neutral-contract pass:

- ClientCodecCapabilitiesV3 gains required video_evidence and
  audio_evidence closed enums (exact | platform_attested | declared).
  Planner strictness follows the tier: exact keeps the strict decode-entry
  validation, platform_attested validates codec/resolution/bit-depth/
  frame-rate but skips profile/level matching, declared grants copy routes
  from the flat codec lists. Only exact audio evidence earns passthrough
  claims. The detailed_decode_capabilities feature token is deleted
  (subsumed by video_evidence=exact), and evidence-blocked direct routes
  carry the new evidence_insufficient_for_direct reason/warning.

- DeviceContextV3 is now platform/os_version/manufacturer/model plus a
  bounded platform_details map (<=16 entries, <=128 chars); the Android
  Build dump fields are gone. Fire TV quirks keep matching on
  manufacturer/model (brand fallback removed with the field).

- output_route_generation (int64, dual-location) becomes an optional
  opaque output_context_id string on the output context; the dual-location
  consistency validation is deleted. Attempt keys, plan invalidation,
  route events, and the planstore column follow (new Goose migration).

- Feature advertisement collapses to the top-level client_features list
  only; ClientPlaybackContextV3.Features is deleted and ReplanRequestV3
  gains an optional client_features refresh.

- PlanRecipeVersionV3 bumped v3.3 -> v3.4; fixtures re-keyed.

Part of the playback protocol v3 neutral-contract finalization plan
(docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): add v3 intent replans, quality menu, and audio-only routes

Protocol v3 could only replan after a failure, so changing the audio track
or the quality still required the legacy audio PATCH and the client-recipe
transcode start — the two endpoints v3 is meant to replace. Clients also had
to own a resolution ladder to render a quality menu, and a source with no
video track was terminaled by the video/HDR gates, keeping audiobooks on the
legacy path.

Add track_change and quality_change replan operations. They carry no failure
classification and route through the existing replan transaction, so they
inherit its idempotency, capacity reservation, and staged-successor commit
for free. Because nothing failed, the previous route stays eligible: neither
the attempted-key history nor the failed-plan exclusion applies to them.

Publish the server ladder on the plan as available_qualities so the quality
menu is server-owned; the rungs come from the same resolutionLabelV3 and
ladderBitrateKbpsV3 helpers the planner itself uses, not a parallel table.

Plan audio-only sources through their own reduced route family: original_http
when the client decodes the codec, otherwise a progressive AAC conversion.
The plan advertises audio/mp4 for that remux and the transport now serves the
same value, because a declared-tier client probes the advertised MIME with
isTypeSupported before attaching a source buffer, and "video/mp4" on a stream
with no video track is exactly the mismatch that makes the probe lie.

Name the protocol's string vocabulary (dynamic ranges, transformations,
executors, validated claims, terminal reasons) as constants while touching
these lines, so the wire values have one definition.

Part of #135

* docs(playback): publish the v3 protocol contract and fix subtitle ordinals

Protocol v3 exists only as Go code today, so the Android and Apple ports have
no authority to implement against other than reading this repository. Publish
the contract as a normative document, machine-checkable schemas, and generated
golden fixtures, and fix the one place where the server's own wire output
disagreed with the ordinal space it publishes.

- docs/architecture/playback-protocol-v3.md is self-contained enough for a
  third-party client: endpoints and status codes, evidence tiers and their
  bound-matching rules, delivery classes, the timeline model, replan
  semantics, registries, track identity, plan identity, quality, and
  transformations.
- docs/design/schemas/playback-v3/ carries JSON Schemas for the five wire
  shapes plus valid and invalid fixtures, following the client-diagnostics
  layout. internal/playback/contract validates every fixture against its
  schema, so a schema that drifts from the Go types fails the Go suite.
- cmd/playbackfixtures generates internal/playback/testdata/protocol_v3 from
  the production planner. `make playback-fixtures` writes them and
  `make verify-playback-fixtures` (wired into CI) fails when they are stale.
  These files are what the client ports consume, so drift would otherwise
  surface as a playback bug on three platforms at once.

The subtitle fix: combined ordinals are one dense space over externals, then
embedded tracks, then downloaded ones, but the legacy URL builder skipped
burn-in-only tracks while assigning indices, so every track after a DVD/DVB
track was numbered one too low and resolved to its neighbour. Ordinal
assignment now lives in playback.BuildSubtitleInventoryV3 and both the plan
inventory and the legacy `subtitle_urls` shape project from it; the legacy
shape still filters burn-in-only entries but keeps each track's real index.

Part of #135

* feat(web): migrate the players to the neutral playback v3 contract

The web player was the last client still speaking the legacy start
protocol: it picked its own file version from a codec probe, posted an
ffmpeg recipe to start a transcode, PATCHed an endpoint to change audio
tracks, and derived its own quality ladder. None of that survives a
server-owned plan, and none of it produced telemetry the apps could be
compared against.

Video player: starts with a v3 request that advertises `declared`
evidence from `isTypeSupported` probes and the three delivery classes,
then consumes the returned plan for its URL, timeline, tracks and
warnings. Quality and track changes become replans (`quality_change`,
`track_change`), the quality menu renders `available_qualities` instead
of computing rungs, and playback failures emit `route-events` so web
failures land in the same diagnostics as Android and Apple. The
duration comes from `source.duration_seconds` rather than the playback
engine, and the "how was this delivered" overlay reads the plan's
delivery and server transformations instead of comparing codec strings.

Audiobook player: starts against the audio-only planner path with a
single `original` rung, and takes its seek anchor from
`timeline.player_start_seconds` so the progressive-remux route (which
anchors the stream and restarts the player clock at zero) does not seek
twice.

Server side, `disable_progress_persistence` left the wire, so the rule
it encoded is now derived. Resume state is keyed on the item, but every
part of a multipart presentation shares that key while carrying its own
file-local clock — persisting part 4's position would store "12 minutes
in" as the book's resume point. `PresentationPartTotal > 1` expresses
that directly and generalizes to multipart movies and split episodes,
and a client can no longer forget to ask or lie about it.

`useTranscodeQuality` and the legacy response types are deleted, and
`WEBTEST_KNOWN_FAILURES` loses the audiobook entry along with its fix.

Part of #135

* feat(playback)!: make v3 the only playback protocol

Protocol v3 shipped behind a flag, alongside the legacy start path it was
designed to replace. Running both meant every planner change had to be made
twice, in two shapes that disagree about who decides the route: the legacy
body carried a decision the client had already made, while v3 asks the server
to make it. This deletes the legacy half.

Removed:

- `handleStartPlaybackLegacy` and its request/response bodies. The
  `POST /playback/start` route stays, but the protocol-version dispatch
  envelope is now a strict v3 decode — a body that does not declare
  `protocol_version: 3` gets `426 client_upgrade_required` so an outdated app
  can render a clear "update required" state instead of misreading a plan.
  Deliberately not a `400`: the request may be well-formed for the protocol it
  was written against.
- `POST /playback/transcode/start`, superseded by the `quality_change` replan
  operation, and `PATCH /playback/{session_id}/audio`, superseded by
  `track_change`. Both mutated a session without re-planning.
- The shadow planner and both rollout settings rows. With v3 the only
  protocol, `playback.protocol_v3_enabled` would mean "no playback at all";
  `playback.protocol_v3_shadow_enabled` gated a comparison against a path that
  no longer exists. `409 protocol_disabled` on route-events goes with them, and
  capability `enabled` is now constant `true` (the field stays — clients
  feature-detect against it).
- Version-selection helpers in `internal/playback/resolver.go` that only legacy
  start reached. `Resolve`/`ClientCapabilities`/`PlayDecision` stay: downloads
  consumes them. `internal/jellycompat` has its own resolution surface and is
  untouched.

Behaviour the legacy handlers owned and v3 now owns explicitly: series version
and audio-track preferences are persisted on start and on a `track_change`
replan (not on failure recovery, whose forced route is not a user choice); an
omitted `start_position` resolves to the profile's saved resume point; and an
omitted audio track resolves through the series preference, the profile audio
language, then the library override. Both are settled before planning, because
the plan's timeline is cut at the start position. Spec §2.2 documents this as
"omission is a request, not a default".

The encode-target clamp that lived in the deleted transcode handler is already
enforced in the planner, twice — `availableQualitiesV3` omits rungs at or above
the source height, and the encode path clamps `targetHeight` to it.

Unchanged: progress, stop, HLS manifest and segment delivery, the realtime
control socket, stream tokens and restart reconstruction, watch together,
downloads, jellycompat.

Every removal is recorded in the pre-lock removals table in
docs/architecture/v1-scope.md.

Part of #135

* fix(scanner): stop recording embedded cover art as a video track

ffprobe reports embedded cover art as a video stream carrying
disposition.attached_pic. convertProbeData appended every "video" stream
to VideoTracks without consulting isMainVideoStream, the predicate that
already existed for duration decisions, so the picture was persisted as a
playable track. That misreports the file twice:

  - An audio file with a cover picks up a video track, so it no longer
    satisfies MediaFile.IsAudioOnly and the v3 planner routes an
    audiobook through the video path instead of planAudioOnlyV3.
  - When the picture is ordered ahead of the real stream, the flat
    codec_video/resolution/hdr columns describe the poster: a 954x720
    h264 episode was stored as mjpeg 480x480.

Filter attached_pic streams out of the track loop. The guard is the
disposition flag, not the codec name, so a genuine MJPEG video is still
probed as video — the library has one.

Already-probed rows self-heal on the next playback: NeedsCriticalProbeRepair
already reprobes tracks missing color_range, which covers 21 of the 23
affected rows, and applyProbeData overwrites VideoTracks wholesale. The
remaining two need a rescan; nothing persisted records attached_pic, and
keying repair off still-image codec names would reprobe the genuine MJPEG
file on every playback forever.

Part of the playback v3 neutral-contract work: it is what lets Android
drop AUDIOBOOK_COVER_ART_CODECS, which fabricated decode support the
client cannot honestly claim under video_evidence: "exact".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(playback): publish subtitle URLs even when playback starts with subtitles off

The v3 plan's subtitle inventory is the authoritative track list a client builds
its subtitle menu from, but the handler only rewrote it with session-scoped URLs
when a track was actually selected. A start or replan that resolved to
`subtitle.mode: "off"` therefore returned the planner's URL-less inventory, so a
client whose picker reads the inventory had a menu it could not fetch anything
from. The Cast path hits this every time: it starts with subtitles off and needs
the receiver's text tracks up front.

attachSubtitleArtifactV3 now scopes and publishes the inventory unconditionally
and gates only the artifact stamping on the selection. Spec §8 records that the
`url` on a sidecar entry does not depend on the current selection.

Part of the v3 neutral-contract finalization.

* chore(playback): reconcile neutral v3 with main

* fix(playback): preserve subtitle intent across replans

* fix(playback): retain subtitle inventory on adapted routes

* fix(playback): software-decode High10 AVC for QSV

* fix(playback): scale High10 frames before QSV upload

* fix(playback): preserve empty subtitle inventories

* fix(playback): freeze terminal attempt contract

* chore(playback): name fixture contract tokens

* fix(playback): close v3 conformance review gaps

* chore(playback): name conformance category

* fix(playback): complete v3 conformance contract

* fix(playback): keep schema fixtures generated

* fix(playback): emit schema-valid conformance arrays

* fix(playback): omit empty replan failures

* fix(web): omit empty replan failures

* fix(playback): close neutral v3 contract gaps

* fix(playback): harden v3 replan, transcode, and quality-ladder edge cases

Review remediation for the neutral v3 cutover, server side:

- A failed replan no longer overwrites the durable StartResponse with a
  terminal or advances the replan request ID; an idempotent start replay
  of a still-healthy session returns the original plan.
- SoftwareVideoDecode is now derived inside the transcode layer from
  source facts (codec/profile/bit depth) carried on TranscodeOpts, so
  jellycompat, downloads, recipe-card reconstruction, and transcode
  nodes get the High10 software-decode fix, not just the v3 handler.
  video_to_h264 recipe version bumps to 2 so mixed-version node pools
  that would silently drop the flag fail validation instead.
- Local transport startup shares the 30s ManifestStartupTimeout; a
  timeout with the process still running stays retryable and is no
  longer persisted as a durable terminal against the attempt.
- Sparse replan bodies (failure_recovery et al) no longer reset a
  user-selected quality preference to auto; the empty-value guard now
  covers every operation.
- availableQualitiesV3 publishes no fixed rungs when the source height
  is unknown, keeping the no-upscaling ladder contract.
- The proxy remux path serves audio-only fMP4 as audio/mp4 via a new
  additive AudioOnly token claim, matching the integrated path.
- Plain text subtitle sidecars accept any requested extension again
  (served as VTT), restoring the permissive v1 behavior; ASS and bitmap
  handling is unchanged.
- The 4K-disallowed terminal message discloses when a lower-resolution
  alternate exists but was pinned away by quality "original".

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(web): keep playback alive through failed replans and honest audio claims

Review remediation for the neutral v3 cutover, web player:

- A failed or refused replan no longer unmounts the player: the fatal
  error screen is reserved for loads with no adopted plan, and replan
  failures surface through the existing non-fatal replanError path.
- changeQuality rolls its optimistic preference back when the replan is
  refused or errors, so a failed switch is not silently applied by the
  next unrelated replan and the menu shows the real active rung.
- The capability probe now tests mp3/vorbis codecs and mp3/flac/ogg
  containers (MediaSource with a canPlayType fallback), restoring
  direct play for mp3 audiobooks instead of per-part AAC re-encodes.
- Reanchor seeks issued while a replan is in flight coalesce and run
  when it settles instead of being silently dropped with the scrubber
  pinned to a phantom position.
- Subtitle refresh/translation replans use the resume anchor while the
  media element has no metadata, so a subtitle_ready broadcast during
  startup no longer restarts a resumed stream at 0:00.
- An exhausted failure-recovery chain sets a visible error instead of
  returning silently.

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): accept video-only and VP9 probe metadata

Treat audio and video probe completeness independently so legitimate video-only assets converge without repeated ffprobe repair. Allow unknown codec profile/level metadata to fall through to server adaptation while preserving exact direct-decode constraints.

Fixes #574

* fix(playback): address protocol v3 review findings

* fix(playback): harden lease and probe repair decisions

* fix(playback): close remaining v3 review gaps

* fix(playback): recover failed transcode starts

* fix(playback): address remaining review-bot findings on v3 replan and audio planning

Server:
- The deferred replan lease release is bounded by a 3s timeout so a
  saturated pool or DB outage cannot wedge a handler goroutine that
  holds the per-session store lock on an uncancellable context.
- planAudioOnlyV3 honors the request bandwidth cap: an over-cap source
  skips the original_http direct route and converts to AAC with the
  same bandwidth_cap_applied warning and decision reason the video
  ladder uses. Unknown source bitrate never triggers the cap.
- A copy-audio progressive plan rejected only by a per-delivery
  audio_decode_codecs subset retries as an AAC conversion instead of
  returning adaptation_unavailable, and the AAC recipe respects the
  delivery's max_channels.

Web:
- failure_recovery replans issued while another replan is in flight
  queue (superseding a pending seek reanchor) instead of being
  silently dropped with the fatal overlay already suppressed.
- A terminal response to a fresh non-preserving start clears the
  previous plan and stops its session, so episode navigation cannot
  keep rendering the prior item under the new title.
- A refused recovery replan for a transport-dead plan surfaces the
  error and re-arms the plan failure key, so transient recovery
  failures no longer strand an endless spinner; the audiobook player
  gets the same guard reset.
- A track-less subtitle_translation_completed hands off to the
  refreshed persisted track once the inventory settles, clearing the
  live overlay, instead of pinning the synthetic live track forever.

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): reuse HLS transport for sidecar replans

* fix(playback): stabilize copy HLS remount timeline

* fix(playback): address v3 review findings

* fix(playback): satisfy player contract types

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 18:14:49 -04:00

664 lines
24 KiB
Go

package models
import (
"strings"
"time"
)
const (
mediaBaseTypeAudiobook = "audiobook"
mediaBaseTypePodcast = "podcast"
mediaCodecMJPEG = "mjpeg"
)
// MediaFolder represents a row in the media_folders table.
type MediaFolder struct {
ID int
Paths []string // from media_folder_paths child table
Type string // movies, series, mixed
Name string
Enabled bool
MetadataLanguage string // ISO 639-1 code (e.g. "en", "ja")
AutoTranslateMetadata bool // AI-translate descriptions when providers lack this language
ChapterThumbnailsEnabled bool
IntroDetectionEnabled bool
// TrailerKinds is the allow-list of remote video kinds (ExtraKind values)
// fetched during metadata refresh for this library. Empty disables remote
// videos entirely.
TrailerKinds []string
PosterPath string // S3 key for library poster image
LastScannedAt *time.Time // nullable
ScanWarningCode *string
ScanWarningMessage *string
ScanWarningAt *time.Time
AllowEmptyCleanupOnce bool
SortOrder int
}
// MediaFile represents a row in the media_files table.
type MediaFile struct {
ID int
ContentID string // Sonyflake ID (nullable until matched)
EpisodeID string // FK to episodes.content_id (nullable)
ExtraID string // FK to media_extras.content_id (nullable); set only for local extras files, which keep ContentID/EpisodeID empty
SeasonNumber int // parsed from filename (nullable)
EpisodeNumber int // parsed from filename (nullable)
MediaFolderID int
CanonicalRootPath string
ObservedRootPath string
ContentGroupKey string
GroupKeyVersion int
BaseTitle string
BaseYear int
BaseType string
IdentityConfidence string
IdentityJSON []byte
FilePath string
FileSize int64
FileModifiedAt *time.Time
FileHash string // OSHash (16-char hex)
CodecVideo string // h264, hevc, av1
CodecAudio string // aac, opus, flac
Resolution string // 1080p, 2160p
AudioChannels int
HDR bool
Container string // mkv, mp4
Duration int // seconds
Bitrate int // kbps
VideoTracks []VideoTrack // JSONB
AudioTracks []AudioTrack // JSONB
SubtitleTracks []SubtitleTrack // JSONB
ExternalSubtitles []ExternalSubtitle // JSONB
Chapters []MediaChapter // JSONB; nil means not yet probed for chapters
ChapterThumbnailRetryAfter *time.Time
ChapterThumbnailFailureCount int
ChapterThumbnailLastError string
IntroStart *float64
IntroEnd *float64
CreditsStart *float64
CreditsEnd *float64
RecapStart *float64
RecapEnd *float64
PreviewStart *float64
PreviewEnd *float64
MarkersSource *string
MarkersConfidence *float64
IntroMarkersSource *string
IntroMarkersProvider *string
IntroMarkersConfidence *float64
IntroMarkersAlgorithm *string
IntroMarkersDetectedAt *time.Time
CreditsMarkersSource *string
CreditsMarkersProvider *string
CreditsMarkersConfidence *float64
CreditsMarkersAlgorithm *string
CreditsMarkersDetectedAt *time.Time
RecapMarkersSource *string
RecapMarkersProvider *string
RecapMarkersConfidence *float64
RecapMarkersAlgorithm *string
RecapMarkersDetectedAt *time.Time
PreviewMarkersSource *string
PreviewMarkersProvider *string
PreviewMarkersConfidence *float64
PreviewMarkersAlgorithm *string
PreviewMarkersDetectedAt *time.Time
EditionRaw string
EditionKey string
EditionConfidence *float64
EditionSource string
PresentationKind string
PresentationGroupKey string
PresentationPartIndex int
PresentationPartTotal int
MultiEpisodeStart int
MultiEpisodeEnd int
ProbeSource string // arrs, local
ProbeUpdatedAt *time.Time
MatchAttemptedAt *time.Time
MissingSince *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
// MediaChapter represents a single media chapter derived from embedded file metadata.
type MediaChapter struct {
Index int `json:"index"`
Title string `json:"title"`
StartSeconds float64 `json:"start_seconds"`
EndSeconds float64 `json:"end_seconds"`
Source string `json:"source"`
ThumbnailPath string `json:"thumbnail_path,omitempty"`
ThumbnailThumbhash string `json:"thumbnail_thumbhash,omitempty"`
ThumbnailRetryAfter *time.Time `json:"thumbnail_retry_after,omitempty"`
ThumbnailFailedAt *time.Time `json:"thumbnail_failed_at,omitempty"`
ThumbnailLastError string `json:"thumbnail_last_error,omitempty"`
}
// OverlaySummary is the compact media badge payload shared across API surfaces.
type OverlaySummary struct {
Resolution string `json:"resolution,omitempty"`
HDR string `json:"hdr,omitempty"`
Audio string `json:"audio,omitempty"`
AudioChannels string `json:"audio_channels,omitempty"` // "Stereo", "5.1", "7.1"
VideoCodec string `json:"video_codec,omitempty"` // "H.264", "H.265", "AV1"
Container string `json:"container,omitempty"` // "MKV", "MP4"
AspectRatio string `json:"aspect_ratio,omitempty"` // "16:9", "2.39:1"
ReleaseType string `json:"release_type,omitempty"`
Edition string `json:"edition,omitempty"`
MultiAudio bool `json:"multi_audio,omitempty"` // ≥2 distinct audio languages
MultiSub bool `json:"multi_sub,omitempty"` // ≥1 subtitle track (embedded or external)
}
// PrimaryDVProfile returns the Dolby Vision profile of the first video
// track (0 when none/unprobed).
func (f *MediaFile) PrimaryDVProfile() int {
if f == nil || len(f.VideoTracks) == 0 {
return 0
}
return f.VideoTracks[0].DVProfile
}
// AudioOnlyProbeFacts is the compact probe shape needed to distinguish known
// audio media from incomplete video probes and legacy attached cover art.
type AudioOnlyProbeFacts struct {
BaseType string
CodecVideo string
CodecAudio string
HasVideoTracks bool
HasAudioTracks bool
HasNonImageVideoTracks bool
}
func isImageVideoCodec(codec string) bool {
switch strings.ToLower(strings.TrimSpace(codec)) {
case mediaCodecMJPEG, "jpeg", "png", "webp", "gif", "bmp":
return true
default:
return false
}
}
// HasLegacyAttachedPictureVideo reports the stale shape in which every video
// track on known audio media is embedded cover art.
func (f AudioOnlyProbeFacts) HasLegacyAttachedPictureVideo() bool {
knownAudioType := f.BaseType == mediaBaseTypeAudiobook || f.BaseType == mediaBaseTypePodcast
hasAudio := f.HasAudioTracks || strings.TrimSpace(f.CodecAudio) != ""
if !knownAudioType || !hasAudio {
return false
}
if !f.HasVideoTracks {
return isImageVideoCodec(f.CodecVideo)
}
if f.HasNonImageVideoTracks {
return false
}
return strings.TrimSpace(f.CodecVideo) == "" || isImageVideoCodec(f.CodecVideo)
}
// IsAudioOnly reports whether these facts contain positive evidence for known
// audio media with no playable video stream.
func (f AudioOnlyProbeFacts) IsAudioOnly() bool {
knownAudioType := f.BaseType == mediaBaseTypeAudiobook || f.BaseType == mediaBaseTypePodcast
hasAudio := f.HasAudioTracks || strings.TrimSpace(f.CodecAudio) != ""
return (knownAudioType && hasAudio && !f.HasVideoTracks && strings.TrimSpace(f.CodecVideo) == "") || f.HasLegacyAttachedPictureVideo()
}
// AudioOnlyProbeFacts returns the compact stream evidence for this media file.
func (f *MediaFile) AudioOnlyProbeFacts() AudioOnlyProbeFacts {
if f == nil {
return AudioOnlyProbeFacts{}
}
facts := AudioOnlyProbeFacts{
BaseType: f.BaseType,
CodecVideo: f.CodecVideo,
CodecAudio: f.CodecAudio,
HasVideoTracks: len(f.VideoTracks) > 0,
HasAudioTracks: len(f.AudioTracks) > 0,
}
for _, track := range f.VideoTracks {
if !isImageVideoCodec(track.Codec) {
facts.HasNonImageVideoTracks = true
break
}
}
return facts
}
// HasLegacyAttachedPictureVideo reports the stale catalog shape produced when
// older probes recorded embedded cover art as a video track. BaseType and
// audio evidence keep genuine MJPEG video from being normalized away.
func (f *MediaFile) HasLegacyAttachedPictureVideo() bool {
return f.AudioOnlyProbeFacts().HasLegacyAttachedPictureVideo()
}
// IsAudioOnly reports whether a probed file carries no playable video stream —
// audiobooks and podcasts, as opposed to a video file whose probe is incomplete.
// It also normalizes legacy audiobook/podcast cover-art rows until a repair
// probe removes their stale image-only video metadata.
func (f *MediaFile) IsAudioOnly() bool {
return f.AudioOnlyProbeFacts().IsAudioOnly()
}
// NormalizeVideoBitDepth returns an explicit probe value when available and
// otherwise derives the bit depth from stable ffprobe fields. FFprobe commonly
// omits bits_per_raw_sample for HEVC even though the pixel format and profile
// are conclusive (for example yuv420p10le / Main 10).
func NormalizeVideoBitDepth(explicit int, pixelFormat, profile string) int {
if explicit > 0 {
return explicit
}
pixelFormat = strings.ToLower(strings.TrimSpace(pixelFormat))
for _, candidate := range []struct {
markers []string
depth int
}{
{markers: []string{"p016", "p16", "gray16", "16le", "16be"}, depth: 16},
{markers: []string{"p014", "p14", "gray14", "14le", "14be"}, depth: 14},
{markers: []string{"p012", "p12", "gray12", "12le", "12be"}, depth: 12},
{markers: []string{"p010", "p10", "gray10", "10le", "10be"}, depth: 10},
{markers: []string{"p009", "p9", "gray9", "9le", "9be"}, depth: 9},
} {
for _, marker := range candidate.markers {
if strings.Contains(pixelFormat, marker) {
return candidate.depth
}
}
}
profile = strings.ToLower(strings.TrimSpace(profile))
for _, marker := range []string{"main 12", "main12", "12-bit", "12 bit"} {
if strings.Contains(profile, marker) {
return 12
}
}
for _, marker := range []string{"main 10", "main10", "10-bit", "10 bit"} {
if strings.Contains(profile, marker) {
return 10
}
}
switch pixelFormat {
case "yuv420p", "yuv422p", "yuv444p", "yuvj420p", "yuvj422p", "yuvj444p", "nv12", "nv21", "rgb24", "bgr24":
return 8
}
return 0
}
// VideoTrack represents a probed video stream stored as JSONB.
type VideoTrack struct {
Title string `json:"title,omitempty"`
Codec string `json:"codec,omitempty"`
DolbyVision string `json:"dolby_vision,omitempty"`
DVProfile int `json:"dv_profile,omitempty"`
DVBLCompatID int `json:"dv_bl_compat_id,omitempty"`
DVELPresent bool `json:"dv_el_present,omitempty"`
DVEnhancementLayer string `json:"dv_enhancement_layer,omitempty"` // none, mel, fel, unknown
HDR10Plus bool `json:"hdr10_plus,omitempty"`
Profile string `json:"profile,omitempty"`
Level int `json:"level,omitempty"`
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
AspectRatio string `json:"aspect_ratio,omitempty"`
Interlaced bool `json:"interlaced"`
FrameRate string `json:"frame_rate,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
VideoRange string `json:"video_range,omitempty"`
VideoRangeType string `json:"video_range_type,omitempty"`
ColorRange string `json:"color_range,omitempty"`
ColorPrimaries string `json:"color_primaries,omitempty"`
ColorSpace string `json:"color_space,omitempty"`
ColorTransfer string `json:"color_transfer,omitempty"`
BitDepth int `json:"bit_depth,omitempty"`
PixelFormat string `json:"pixel_format,omitempty"`
ReferenceFrames int `json:"reference_frames,omitempty"`
// MultiplePPS records whether an H.264 stream redefines the same
// pic_parameter_set_id in-band with more than one distinct content. Such
// streams cannot be safely stream-copied into an avc1/fMP4 HLS segment:
// the avcC declares a single parameter set, so strict decoders
// (VideoToolbox on Safari/Chrome-macOS) desync on the mid-GOP switches.
//
// This is a runtime-only field: it is computed at playback start by a
// bitstream scan and held in memory, never serialized to the database
// (`json:"-"`). nil means "not analyzed in this process yet".
MultiplePPS *bool `json:"-"`
// VideoCopyUnsafe is set when conflicting PPS data is detected or when the
// safety scan cannot establish that video stream-copy is safe. It is
// runtime-only so transient scan failures are retried on a later request.
VideoCopyUnsafe bool `json:"-"`
}
// AudioTrack represents a probed audio stream stored as JSONB.
type AudioTrack struct {
Title string `json:"title,omitempty"`
EmbeddedTitle string `json:"embedded_title,omitempty"`
Language string `json:"language,omitempty"`
Codec string `json:"codec,omitempty"`
Profile string `json:"profile,omitempty"`
Layout string `json:"layout,omitempty"`
Channels int `json:"channels,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
SampleRate int `json:"sample_rate,omitempty"`
BitDepth int `json:"bit_depth,omitempty"`
Default bool `json:"default"`
}
// SubtitleTrack represents an embedded subtitle track stored as JSONB.
type SubtitleTrack struct {
Index int `json:"index"`
Language string `json:"language"`
Codec string `json:"codec"`
Title string `json:"title,omitempty"`
EmbeddedTitle string `json:"embedded_title,omitempty"`
Resolution string `json:"resolution,omitempty"`
Forced bool `json:"forced"`
Default bool `json:"default"`
HearingImpaired bool `json:"hearing_impaired"`
External bool `json:"external"`
FileName string `json:"file_name,omitempty"`
}
// ExternalSubtitle represents a sidecar subtitle file stored as JSONB.
type ExternalSubtitle struct {
Path string `json:"path"`
Language string `json:"language"`
Format string `json:"format"` // srt, vtt, ass, ssa, sub
Title string `json:"title,omitempty"`
EmbeddedTitle string `json:"embedded_title,omitempty"`
Resolution string `json:"resolution,omitempty"`
Forced bool `json:"forced"`
Default bool `json:"default"`
HearingImpaired bool `json:"hearing_impaired"`
}
// PersonKind identifies the role type a person has on a media item.
type PersonKind int
const (
PersonKindActor PersonKind = 1
PersonKindDirector PersonKind = 2
PersonKindWriter PersonKind = 3
PersonKindProducer PersonKind = 4
PersonKindGuestStar PersonKind = 5
PersonKindComposer PersonKind = 6
PersonKindAuthor PersonKind = 7
PersonKindNarrator PersonKind = 8
)
// String returns the Jellyfin-compatible type string for this PersonKind.
func (k PersonKind) String() string {
switch k {
case PersonKindActor:
return "Actor"
case PersonKindDirector:
return "Director"
case PersonKindWriter:
return "Writer"
case PersonKindProducer:
return "Producer"
case PersonKindGuestStar:
return "GuestStar"
case PersonKindComposer:
return "Composer"
case PersonKindAuthor:
return "Author"
case PersonKindNarrator:
return "Narrator"
default:
return "Unknown"
}
}
// PersonKindFromJob maps a crew job title to a PersonKind.
func PersonKindFromJob(job string) PersonKind {
switch strings.ToLower(strings.TrimSpace(job)) {
case "director":
return PersonKindDirector
case "writer", "screenplay", "story", "novel":
return PersonKindWriter
case "composer", "original music composer", "music":
return PersonKindComposer
default:
return PersonKindProducer
}
}
// Person represents a deduplicated person entity.
type Person struct {
ID int64
Name string
SortName string
Bio string
BirthDate *time.Time
DeathDate *time.Time
Birthplace string
Homepage string
PhotoPath string
PhotoSourcePath string
PhotoThumbhash string
TmdbID string
ImdbID string
TvdbID string
PlexGUID string
CreatedAt time.Time
UpdatedAt time.Time
}
// ItemPerson represents a person's credit on a specific media item.
type ItemPerson struct {
Person
Kind PersonKind
Character string
SortOrder int
}
// AudiobookSeriesMembership captures a book's membership in an audiobook
// series and its optional sequence number within that series.
type AudiobookSeriesMembership struct {
Name string
Index *float64
}
// MediaItem represents a row in the media_items table.
type MediaItem struct {
ContentID string // Sonyflake ID (PK)
Type string // movie, series
Title string
SortTitle string
DefaultMetadataLanguage string
OriginalTitle string
Year int
Genres []string
ContentRating string // PG-13, TV-MA
Runtime int // minutes
Overview string
Tagline string
RatingIMDB *float64
RatingTMDB *float64
RatingRTCritic *int
RatingRTAudience *int
ImdbID string
TmdbID string
TvdbID string
PosterPath string // S3 path
PosterSourcePath string // provider-origin path kept when caching rewrites PosterPath; feeds outbound embeds
PosterThumbhash string
BackdropPath string
BackdropSourcePath string
BackdropThumbhash string
LogoPath string
LogoSourcePath string
MetadataS3Path string
MetadataEtag string
SeasonCount *int // series only
MangaChapterCount *int // manga series only: loose manga_chapters rows without a volume token
MangaVolumeCount *int // manga series only: distinct non-empty volume tokens in manga_chapters
Studios []string
Networks []string
Countries []string
Keywords []string
OriginalLanguage string
ReleaseDate *string // ISO date, nullable
FirstAirDate *string // ISO date (series only), nullable
LastAirDate *string // ISO date (series only), nullable
AirTime *string // Series broadcast time (e.g. "20:00"), nullable
AirTimezone *string // Series broadcast timezone (IANA name, e.g. "America/New_York"), nullable
ShowStatus string // Series lifecycle: "returning", "ended", "cancelled", "in_production", "upcoming", or "" if unknown (series; manga uses its own domain, e.g. "Ongoing")
People []ItemPerson
AudiobookSeries []AudiobookSeriesMembership
MatchedAt *time.Time
EpisodeMetadataIncomplete bool
EpisodeMetadataLastCheckedAt *time.Time
LastRefreshed *time.Time `json:"last_refreshed,omitempty"`
RefreshFailures int `json:"refresh_failures"`
LockedFields []int `json:"locked_fields"`
Status string `json:"status"` // pending, matched, unmatched
CreatedAt time.Time
UpdatedAt time.Time
AddedAt *time.Time // populated by browse queries (MIN(mil.first_seen_at))
}
// MediaItemAlias is a provider-confirmed searchable title for a media item.
type MediaItemAlias struct {
ContentID string
Title string
Language string
Kind string
Provider string
}
// Season represents a row in the seasons table.
type Season struct {
ContentID string
SeriesID string
SeasonNumber int
Title string
DefaultMetadataLanguage string
Overview string
AirDate *time.Time
PosterPath string
PosterSourcePath string
PosterThumbhash string
MetadataS3Path string
MetadataEtag string
MetadataSource string
CreatedAt time.Time
UpdatedAt time.Time
}
// Episode represents a row in the episodes table.
type Episode struct {
ContentID string // episode Sonyflake ID (PK)
SeriesID string
SeasonID string // FK to seasons.content_id
SeasonNumber int
EpisodeNumber int
Title string
DefaultMetadataLanguage string
Overview string
AirDate *time.Time
Runtime int // minutes
RatingIMDB *float64
RatingTMDB *float64
ImdbID string
TmdbID string
TvdbID string
StillPath string
StillSourcePath string
StillThumbhash string
MetadataS3Path string
MetadataEtag string
MetadataSource string
CreatedAt time.Time
UpdatedAt time.Time
}
// MediaItemRoot represents a row in the media_item_roots table.
// It records which content_id owns a given canonical root path within a library folder.
type MediaItemRoot struct {
MediaFolderID int
CanonicalRootPath string
ContentID string
FirstSeenAt time.Time
LastSeenAt time.Time
}
// MediaItemGroup represents a row in the media_item_groups table.
type MediaItemGroup struct {
MediaFolderID int
GroupKeyVersion int
ContentGroupKey string
ContentID string
FirstSeenAt time.Time
LastSeenAt time.Time
}
// MediaItemLibrary represents a row in the media_item_libraries junction table.
type MediaItemLibrary struct {
ContentID string
MediaFolderID int
FirstSeenAt time.Time
}
// EpisodeLibrary represents a row in the episode_libraries junction table.
type EpisodeLibrary struct {
EpisodeID string
MediaFolderID int
FirstSeenAt time.Time
}
// Localization field provenance values. Precedence when writing:
// manual beats provider beats ai — a provider refresh may overwrite an AI
// translation but never a manual edit, and AI never overwrites either
// (except provider, when the admin explicitly forces a re-translation).
const (
LocalizationSourceProvider = "provider"
LocalizationSourceAI = "ai"
LocalizationSourceManual = "manual"
)
type MediaItemLocalization struct {
ContentID string
Language string
Title string
SortTitle string
Overview string
Tagline string
PosterPath string
PosterSourcePath string
PosterThumbhash string
BackdropPath string
BackdropSourcePath string
BackdropThumbhash string
LogoPath string
LogoSourcePath string
OverviewSource string // provider | ai | manual
TaglineSource string // provider | ai | manual
CreatedAt time.Time
UpdatedAt time.Time
}
type SeasonLocalization struct {
SeasonContentID string
Language string
Title string
Overview string
PosterPath string
PosterSourcePath string
PosterThumbhash string
OverviewSource string // provider | ai | manual
CreatedAt time.Time
UpdatedAt time.Time
}
type EpisodeLocalization struct {
EpisodeContentID string
Language string
Title string
Overview string
OverviewSource string // provider | ai | manual
CreatedAt time.Time
UpdatedAt time.Time
}