Files
silo-server/internal/scanner/types.go
T
854d07cf8f feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation

* docs(playback): incorporate protocol v3 review

* feat(playback): implement protocol v3 server

* fix(playback): persist empty route diagnostics

* feat(playback): harden protocol v3 HDR routing

* feat(playback): complete protocol v3 client contract

* fix(playback): harden protocol v3 recovery

* fix(playback): restore dovi_rpu strip filter for DV remuxes

The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl
bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg;
the probe failed closed on every deployment, disabling the new validated
DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1
remux path from main. Restore dovi_rpu across the probe, remux and HLS
copy arguments, and the recipe-card constant.

Also from review: validate the remux DV mode for every profile (garbage
modes on non-P7 sources silently no-opped), reject preserve mode for P7
outright (a base-layer-only remux cannot preserve dual-layer DV), tag
dvhe sample entries only for the explicit v3 preserve recipe so legacy
web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the
token-frozen DV mode in the proxy remux path instead of legacy-auto.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): correct v3 planner policy and contract validation

Review fixes to the v3 planner and wire contracts:

- Bar Profile 7 sources from the non-strip progressive remux route: a
  base-layer-only remux can never deliver native dual-layer DV, so the
  planner no longer emits plans claiming validated Dolby Vision while
  the executed remux drops the enhancement layer.
- Accept the device-quirks feature flag from either capability location,
  matching every other dual-location feature check.
- Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with
  a degradation warning instead of leaving them unplayable under v3.
- Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and
  wire the previously dead Metered signal into conservative auto rungs.
- Degrade to the validated source-quality route instead of a terminal
  when only an implicit quality reduction demanded an unsupported
  transcode; explicit user-selected rungs keep terminal behavior.
- Bound inner capability lists and strings; compare attempt keys exactly
  instead of case-folded; make ParseTrackIDV3 strict about canonical
  numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising
  burn-in for unknown subtitle codecs; probe every h264 encoder rather
  than requiring libx264; normalize the file-level bitrate fallback.
- Evaluate subtitle renderability against the engine each candidate
  route executes on, not always media3_direct.
- Pin the with-quirks attempt-key preimage arity in the cross-language
  fixture so the Kotlin client stays in lockstep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 control-plane reliability

Review fixes to the v3 session, store, and handler layer:

- Bound concurrent replans with a slot semaphore: each replan pins a
  pooled connection for its advisory lock while issuing further store
  queries from the same pool, so an unbounded recovery storm could turn
  every connection into a lock holder and deadlock the server.
- Make CompleteReplan a real compare-and-swap (base-revision predicate,
  ErrReplanSupersededV3) and map BeginReplan insert races to a replay
  instead of a raw unique violation.
- Fingerprint start requests (request_digest column): an attempt ID
  reused with different input is now a 409-style conflict rather than a
  silent replay, and both replay paths check session liveness so dead
  sessions surface as retryable terminals.
- Pre-delete expired attempt rows on SaveAttempt so a retry during the
  cleanup window cannot wedge on an unreachable conflict.
- Align the in-memory store's semantics with Postgres and add DB-backed
  planstore tests (SILO_TEST_DATABASE_URL), including a regression test
  inserting every route-event name against the real CHECK constraint.
- Session manager: v3 route-set updates own RemuxDVMode outright so a
  replan onto an SDR source clears a stale strip mode; replacement
  reservations survive unrelated legacy stream updates; replacement
  admission excludes the replaced session explicitly instead of
  decrementing totals it may no longer be part of; the admission CAS
  loop is bounded and decider errors are logged.
- Map transient store failures to 500s instead of terminal 404/403s;
  authorize route events via identity-only projections after the rate
  limiter; keep sanitized diagnostics deterministic.
- Merge the server-computed durable plan key into replan exclusions so
  unreproducible client history cannot re-select the failed route.
- Remap tracks only when the effective edition changes (a same-file
  replan no longer switches audio to a lookalike track) and remap
  ID-only subtitle selections on edition fallback.
- Cache the v3/shadow feature flags for five seconds instead of one
  settings SELECT per playback request; stop remote transports
  best-effort when the start call times out; carry dvm/tid claims and
  the transport-scoped job identity through the legacy audio-change
  re-mint; index playback_route_events(received_at) for the retention
  delete; run store maintenance for DB-less deployments too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): reap idle node jobs and gate WebVTT conversion

- Add an idle reaper to the transcode node: a job untouched by manifest
  or segment requests for ten minutes is closed and unregistered. After
  a v3 replan retires a transport ID, a stale in-flight stream token
  could resurrect the old job via reconstruct and encode to end-of-file
  for nobody; jobs waiting on readiness count registration as access
  and are never reaped mid-wait, and reaping keeps the recipe so a
  still-valid token reconstructs on the next hit.
- Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with
  415 before headers are written instead of spawning an ffmpeg command
  that always fails mid-response, and make the extract-format override
  fall back to source-driven mapping for bitmap codecs.
- Drain error bodies on non-202 node responses so the HTTP transport
  can reuse connections.
- Pin the transcode-dir cleanup separator-boundary semantics with a
  regression test (a session ID sharing another's prefix must not
  retain foreign directories).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): close v3 planner policy gaps from review

- Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has
  no rung below 480p/1500kbps, so lower caps were silently exceeded even
  though the cap is documented as a hard delivery ceiling.
- Treat video-only media as audio-compatible instead of forcing an AAC
  conversion (or an audio_conversion_unsupported terminal) onto a file
  with no audio stream. Tracks whose codec failed to probe keep the gate.
- Only promise a bitmap subtitle sidecar for embedded PGS with an engine
  that renders embedded bitmap: external/downloaded bitmap and embedded
  DVD/DVB published artifact URLs that always failed at fetch. They now
  fall through to burn-in or its terminal.
- Accept client_video_transformations_v1 from either client_features or
  the nested context when validating client-executor transformations,
  matching the planner's dual-source reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): probe and execute DV remuxes with one ffmpeg binary

The v3 transformation registry probed the configured playback.ffmpeg_path
while progressive remux execution resolved the process-global discovery
path, so a deployment where only one binary carries dovi_rpu could plan a
server_dv7_to_hdr10 route and then fail it at stream time. Resolution now
goes through a shared ResolveFFmpegPath (configured path first, discovery
fallback — the same rule the transcode pipeline already used), the
dovi_rpu probe is cached per binary path, and the stream handler and proxy
worker pass their configured path into ServeRemuxWithDVMode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 replan identity and control-plane limits

- Seed failure-replan track selections from the durable current plan
  before overlaying the request: after an alternate-version fallback the
  normalized request still carries requested-edition track IDs, so a
  replan omitting unchanged tracks was rejected as a track/file mismatch.
- Remap ID-only audio selections across edition changes (parse the ID to
  an index like the subtitle remap already does) instead of leaving a
  stale file-bound ID to fail validation.
- Release the node planner reservation when a prepared remote transport
  rolls back after the node accepted the job; repeated failed starts
  could otherwise pin max-job/bandwidth budgets for the full reservation
  age.
- Size the replan semaphore below the PostgreSQL pool via a store
  capacity advisor: with max_connections at or below the fixed bound,
  advisory-lock holders could starve the inner store queries they need
  to finish.
- Contain shadow-planner panics with a recover boundary; it runs on a
  bare goroutine where an escaped panic kills the process for what is
  telemetry-only work. Document why the memory store's session lock is
  deliberately a no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): serialize node job teardown against reconstructs

- Look up and touch manifest/segment sessions in one critical section so
  the idle reaper cannot unregister a job between the lookup and its
  liveness refresh.
- Re-validate each reap candidate under the per-session lifecycle lock
  before closing it: Close removes the output directory, and without the
  lock it could race a token reconstruct and wipe the segments the fresh
  ffmpeg is writing.
- Take the lifecycle lock in handleStop so a stop racing a RequireReady
  start's readiness wait blocks until registration and tears the job
  down, instead of 404ing and orphaning the ffmpeg until the reaper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:51:27 -04:00

165 lines
4.5 KiB
Go

package scanner
// ScanResult contains the outcome of scanning a media folder.
type ScanResult struct {
New int
Updated int
Unchanged int
Missing int
FilesDeleted int
MembershipsRemoved int
ItemsDeleted int
Errors int
EmptyRootGuarded bool
RootObservations []RootObservation
}
// FileHints contains the OSHash gathered during scanning.
type FileHints struct {
FileHash string // OSHash from xattr or computed
}
// ProbeData contains media file technical information from ffprobe.
type ProbeData struct {
CodecVideo string
CodecAudio string
Resolution string // 1080p, 2160p, etc.
AudioChannels int
HDR bool
Container string
Duration int // seconds
Bitrate int // kbps
VideoTracks []VideoTrackInfo
AudioTracks []AudioTrackInfo
SubtitleTracks []SubtitleTrackInfo
Chapters []ChapterInfo
FormatTags map[string]string // format-level tags from ffprobe (title, artist, album, date, etc.)
}
// VideoTrackInfo describes a probed video track.
type VideoTrackInfo struct {
Title string
Codec string
DolbyVision string
DVProfile int
DVBLCompatID int
DVELPresent bool
DVEnhancementLayer string
HDR10Plus bool
Profile string
Level int
Width int
Height int
AspectRatio string
Interlaced bool
FrameRate string
Bitrate int
VideoRange string
VideoRangeType string
ColorPrimaries string
ColorSpace string
ColorTransfer string
BitDepth int
PixelFormat string
ReferenceFrames int
}
// AudioTrackInfo describes a probed audio track.
type AudioTrackInfo struct {
Title string
EmbeddedTitle string
Language string
Codec string
Profile string
Layout string
Channels int
Bitrate int
SampleRate int
BitDepth int
Default bool
}
// SubtitleTrackInfo describes an embedded subtitle track from probing.
type SubtitleTrackInfo struct {
Index int
Language string
Codec string
Title string
EmbeddedTitle string
Resolution string
Forced bool
Default bool
HearingImpaired bool
}
// ExternalSubtitleInfo describes a discovered sidecar subtitle file.
type ExternalSubtitleInfo struct {
Path string
Language string
Format string // srt, vtt, ass, ssa, sub
Title string
Forced bool
}
// ChapterInfo describes an embedded chapter extracted from ffprobe.
type ChapterInfo struct {
Index int
Title string
StartSeconds float64
EndSeconds float64
Source string
}
// IntroCreditsMarkers contains intro/credits timing from S3 markers.
type IntroCreditsMarkers struct {
IntroStart *float64
IntroEnd *float64
CreditsStart *float64
CreditsEnd *float64
}
// MarkerUpdate is the narrow marker-only update payload shared by scanner and analyzers.
type MarkerUpdate struct {
IntroStart *float64
IntroEnd *float64
CreditsStart *float64
CreditsEnd *float64
RecapStart *float64
RecapEnd *float64
PreviewStart *float64
PreviewEnd *float64
MarkersSource string
MarkersProvider *string
MarkersConfidence *float64
MarkersAlgorithm string
// Optional per-segment provenance overrides. When set for a segment,
// UpsertMarkers writes these source/provider/confidence/algorithm values
// for that segment instead of the shared Markers* fields above. Used by
// merged multi-provider results where each segment may come from a
// different provider/source class; left nil for single-source writes
// (scanner/s3/local).
IntroProvenance *SegmentProvenance
CreditsProvenance *SegmentProvenance
RecapProvenance *SegmentProvenance
PreviewProvenance *SegmentProvenance
}
// SegmentProvenance is a per-segment attribution override for MarkerUpdate.
// When Source is empty, the shared MarkerUpdate.MarkersSource applies.
type SegmentProvenance struct {
Source string
Provider *string
Confidence *float64
Algorithm string
}
// HasAnySegment reports whether the update would write at least one segment.
// An update with no segment bounds set is a no-op and skipped by UpsertMarkers.
func (u MarkerUpdate) HasAnySegment() bool {
return u.IntroStart != nil || u.IntroEnd != nil ||
u.CreditsStart != nil || u.CreditsEnd != nil ||
u.RecapStart != nil || u.RecapEnd != nil ||
u.PreviewStart != nil || u.PreviewEnd != nil
}