Files
silo-server/internal/playback/remux.go
T
854d07cf8f feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation

* docs(playback): incorporate protocol v3 review

* feat(playback): implement protocol v3 server

* fix(playback): persist empty route diagnostics

* feat(playback): harden protocol v3 HDR routing

* feat(playback): complete protocol v3 client contract

* fix(playback): harden protocol v3 recovery

* fix(playback): restore dovi_rpu strip filter for DV remuxes

The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl
bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg;
the probe failed closed on every deployment, disabling the new validated
DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1
remux path from main. Restore dovi_rpu across the probe, remux and HLS
copy arguments, and the recipe-card constant.

Also from review: validate the remux DV mode for every profile (garbage
modes on non-P7 sources silently no-opped), reject preserve mode for P7
outright (a base-layer-only remux cannot preserve dual-layer DV), tag
dvhe sample entries only for the explicit v3 preserve recipe so legacy
web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the
token-frozen DV mode in the proxy remux path instead of legacy-auto.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): correct v3 planner policy and contract validation

Review fixes to the v3 planner and wire contracts:

- Bar Profile 7 sources from the non-strip progressive remux route: a
  base-layer-only remux can never deliver native dual-layer DV, so the
  planner no longer emits plans claiming validated Dolby Vision while
  the executed remux drops the enhancement layer.
- Accept the device-quirks feature flag from either capability location,
  matching every other dual-location feature check.
- Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with
  a degradation warning instead of leaving them unplayable under v3.
- Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and
  wire the previously dead Metered signal into conservative auto rungs.
- Degrade to the validated source-quality route instead of a terminal
  when only an implicit quality reduction demanded an unsupported
  transcode; explicit user-selected rungs keep terminal behavior.
- Bound inner capability lists and strings; compare attempt keys exactly
  instead of case-folded; make ParseTrackIDV3 strict about canonical
  numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising
  burn-in for unknown subtitle codecs; probe every h264 encoder rather
  than requiring libx264; normalize the file-level bitrate fallback.
- Evaluate subtitle renderability against the engine each candidate
  route executes on, not always media3_direct.
- Pin the with-quirks attempt-key preimage arity in the cross-language
  fixture so the Kotlin client stays in lockstep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 control-plane reliability

Review fixes to the v3 session, store, and handler layer:

- Bound concurrent replans with a slot semaphore: each replan pins a
  pooled connection for its advisory lock while issuing further store
  queries from the same pool, so an unbounded recovery storm could turn
  every connection into a lock holder and deadlock the server.
- Make CompleteReplan a real compare-and-swap (base-revision predicate,
  ErrReplanSupersededV3) and map BeginReplan insert races to a replay
  instead of a raw unique violation.
- Fingerprint start requests (request_digest column): an attempt ID
  reused with different input is now a 409-style conflict rather than a
  silent replay, and both replay paths check session liveness so dead
  sessions surface as retryable terminals.
- Pre-delete expired attempt rows on SaveAttempt so a retry during the
  cleanup window cannot wedge on an unreachable conflict.
- Align the in-memory store's semantics with Postgres and add DB-backed
  planstore tests (SILO_TEST_DATABASE_URL), including a regression test
  inserting every route-event name against the real CHECK constraint.
- Session manager: v3 route-set updates own RemuxDVMode outright so a
  replan onto an SDR source clears a stale strip mode; replacement
  reservations survive unrelated legacy stream updates; replacement
  admission excludes the replaced session explicitly instead of
  decrementing totals it may no longer be part of; the admission CAS
  loop is bounded and decider errors are logged.
- Map transient store failures to 500s instead of terminal 404/403s;
  authorize route events via identity-only projections after the rate
  limiter; keep sanitized diagnostics deterministic.
- Merge the server-computed durable plan key into replan exclusions so
  unreproducible client history cannot re-select the failed route.
- Remap tracks only when the effective edition changes (a same-file
  replan no longer switches audio to a lookalike track) and remap
  ID-only subtitle selections on edition fallback.
- Cache the v3/shadow feature flags for five seconds instead of one
  settings SELECT per playback request; stop remote transports
  best-effort when the start call times out; carry dvm/tid claims and
  the transport-scoped job identity through the legacy audio-change
  re-mint; index playback_route_events(received_at) for the retention
  delete; run store maintenance for DB-less deployments too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): reap idle node jobs and gate WebVTT conversion

- Add an idle reaper to the transcode node: a job untouched by manifest
  or segment requests for ten minutes is closed and unregistered. After
  a v3 replan retires a transport ID, a stale in-flight stream token
  could resurrect the old job via reconstruct and encode to end-of-file
  for nobody; jobs waiting on readiness count registration as access
  and are never reaped mid-wait, and reaping keeps the recipe so a
  still-valid token reconstructs on the next hit.
- Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with
  415 before headers are written instead of spawning an ffmpeg command
  that always fails mid-response, and make the extract-format override
  fall back to source-driven mapping for bitmap codecs.
- Drain error bodies on non-202 node responses so the HTTP transport
  can reuse connections.
- Pin the transcode-dir cleanup separator-boundary semantics with a
  regression test (a session ID sharing another's prefix must not
  retain foreign directories).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): close v3 planner policy gaps from review

- Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has
  no rung below 480p/1500kbps, so lower caps were silently exceeded even
  though the cap is documented as a hard delivery ceiling.
- Treat video-only media as audio-compatible instead of forcing an AAC
  conversion (or an audio_conversion_unsupported terminal) onto a file
  with no audio stream. Tracks whose codec failed to probe keep the gate.
- Only promise a bitmap subtitle sidecar for embedded PGS with an engine
  that renders embedded bitmap: external/downloaded bitmap and embedded
  DVD/DVB published artifact URLs that always failed at fetch. They now
  fall through to burn-in or its terminal.
- Accept client_video_transformations_v1 from either client_features or
  the nested context when validating client-executor transformations,
  matching the planner's dual-source reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): probe and execute DV remuxes with one ffmpeg binary

The v3 transformation registry probed the configured playback.ffmpeg_path
while progressive remux execution resolved the process-global discovery
path, so a deployment where only one binary carries dovi_rpu could plan a
server_dv7_to_hdr10 route and then fail it at stream time. Resolution now
goes through a shared ResolveFFmpegPath (configured path first, discovery
fallback — the same rule the transcode pipeline already used), the
dovi_rpu probe is cached per binary path, and the stream handler and proxy
worker pass their configured path into ServeRemuxWithDVMode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 replan identity and control-plane limits

- Seed failure-replan track selections from the durable current plan
  before overlaying the request: after an alternate-version fallback the
  normalized request still carries requested-edition track IDs, so a
  replan omitting unchanged tracks was rejected as a track/file mismatch.
- Remap ID-only audio selections across edition changes (parse the ID to
  an index like the subtitle remap already does) instead of leaving a
  stale file-bound ID to fail validation.
- Release the node planner reservation when a prepared remote transport
  rolls back after the node accepted the job; repeated failed starts
  could otherwise pin max-job/bandwidth budgets for the full reservation
  age.
- Size the replan semaphore below the PostgreSQL pool via a store
  capacity advisor: with max_connections at or below the fixed bound,
  advisory-lock holders could starve the inner store queries they need
  to finish.
- Contain shadow-planner panics with a recover boundary; it runs on a
  bare goroutine where an escaped panic kills the process for what is
  telemetry-only work. Document why the memory store's session lock is
  deliberately a no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): serialize node job teardown against reconstructs

- Look up and touch manifest/segment sessions in one critical section so
  the idle reaper cannot unregister a job between the lookup and its
  liveness refresh.
- Re-validate each reap candidate under the per-session lifecycle lock
  before closing it: Close removes the output directory, and without the
  lock it could race a token reconstruct and wipe the segments the fresh
  ffmpeg is writing.
- Take the lifecycle lock in handleStop so a stop racing a RequireReady
  start's readiness wait blocks until registration and tears the job
  down, instead of 404ing and orphaning the ffmpeg until the reaper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:51:27 -04:00

363 lines
13 KiB
Go

package playback
import (
"bytes"
"context"
"fmt"
"io"
"log/slog"
"net/http"
"os"
"os/exec"
"strconv"
"strings"
"sync"
"github.com/Silo-Server/silo-server/internal/httpstream"
)
var (
resolvedFFmpegPath string
ffmpegOnce sync.Once
doviRPUMu sync.Mutex
doviRPUCache map[string]bool
)
// ffmpegBinary returns the path to the ffmpeg binary.
// Resolved once at first call, then cached for the process lifetime.
func ffmpegBinary() string {
ffmpegOnce.Do(func() {
const jellyfinPath = "/usr/lib/jellyfin-ffmpeg/ffmpeg"
if _, err := exec.LookPath(jellyfinPath); err == nil {
resolvedFFmpegPath = jellyfinPath
return
}
resolvedFFmpegPath = "ffmpeg"
})
return resolvedFFmpegPath
}
// ResolveFFmpegPath returns the ffmpeg binary the playback pipeline executes
// for the given configured path: the configured path when set, otherwise the
// process-global discovery (jellyfin-ffmpeg install, then PATH). Capability
// probes must resolve through this same function so a feature advertised at
// planning time is guaranteed present in the binary that later runs.
func ResolveFFmpegPath(configured string) string {
if strings.TrimSpace(configured) != "" {
return configured
}
return ffmpegBinary()
}
// supportsDoviRPUFilter reports whether the given FFmpeg binary can strip
// Dolby Vision RPU metadata via the dovi_rpu bitstream filter (FFmpeg 7.1+).
// The enhancement layer itself is dropped by stream mapping, so stripping the
// RPUs yields a clean HDR10 base layer. Probed once per binary path.
func supportsDoviRPUFilter(bin string) bool {
doviRPUMu.Lock()
defer doviRPUMu.Unlock()
if available, ok := doviRPUCache[bin]; ok {
return available
}
out, err := exec.Command(bin, "-hide_banner", "-bsfs").Output()
available := err == nil && bytes.Contains(out, []byte("dovi_rpu"))
if !available {
slog.Warn("ffmpeg lacks the dovi_rpu bitstream filter (needs FFmpeg 7.1+); validated Profile 7 HDR10 remux is disabled", "ffmpeg", bin)
}
if doviRPUCache == nil {
doviRPUCache = make(map[string]bool)
}
doviRPUCache[bin] = available
return available
}
// remuxDVProfile neutralizes a Dolby Vision profile the local ffmpeg cannot
// handle. Profile 7 is the only profile that triggers an RPU strip in
// buildRemuxArgs; when the dovi_rpu filter is unavailable the remux must
// still start (an unknown bitstream filter aborts ffmpeg immediately), so
// fall back to the pre-strip behavior instead of failing playback.
func remuxDVProfile(dvProfile int, canStripRPU bool) int {
if dvProfile == 7 && !canStripRPU {
return 0
}
return dvProfile
}
// RemuxSession represents a running ffmpeg remux process that copies
// codecs to a new container format without re-encoding.
type RemuxSession struct {
cmd *exec.Cmd
cancel context.CancelFunc
outputPipe io.ReadCloser
}
// RemuxDVMode makes Profile 7 handling an explicit byte-level recipe choice.
// The empty/legacy mode exists only for pre-v3 callers and old stream tokens.
type RemuxDVMode string
const (
RemuxDVLegacyAutoV3 RemuxDVMode = "legacy_auto"
RemuxDVPreserveV3 RemuxDVMode = "preserve"
RemuxDVStripToHDR10V3 RemuxDVMode = "strip_to_hdr10"
RemuxDVRejectP7V3 RemuxDVMode = "reject_profile_7"
)
// buildRemuxArgs constructs the ffmpeg argument list for a remux operation.
// The args perform codec copy (-c copy) into the target container format,
// using fragmented output for streaming (frag_keyframe+delay_moov+default_base_moof) and
// pipe:1 for stdout output.
// When transcodeAudio is true, video is copied but audio is transcoded to
// stereo AAC (handles cases like DTS/TrueHD that browsers cannot decode).
// dvProfile is the file's Dolby Vision profile (0 = none). Profile 7 remuxes
// strip DV RPUs: the enhancement layer is dropped by the video map below, so
// the RPUs would dangle — stripping yields a clean HDR10 base layer (the
// Apple-parity fallback for devices without a P7 decoder). Profile 8 RPUs
// stay: the base layer is self-contained and DV clients can render it.
func buildRemuxArgs(filePath, outputFormat string, seekSeconds float64, transcodeAudio bool, audioTrackIndex int, dvProfile int, tagDVSampleEntry bool) []string {
args := []string{
"-nostdin",
"-hide_banner",
"-loglevel", "error",
// Cap the input probe to speed up startup on large MKV containers
// (defaults scan up to 5 seconds / several MB). Mirrors the transcode
// pipeline, which has run these caps in production without issue.
"-fflags", "+genpts+fastseek",
"-analyzeduration", "3000000",
"-probesize", "5000000",
}
// Add seek if requested (before input for fast seek).
if seekSeconds > 0 {
args = append(args, "-ss", strconv.FormatFloat(seekSeconds, 'f', 3, 64))
if transcodeAudio {
// In copy mode (-c:v copy) video must start at a keyframe, but
// accurate_seek (the default) trims re-encoded audio to the exact
// seek point. This mismatch causes A/V desync equal to the gap
// between the keyframe and the seek point. Disabling accurate seek
// keeps both streams aligned at the keyframe boundary.
args = append(args, "-noaccurate_seek")
}
}
args = append(args, "-i", filePath)
// Strip metadata/chapters and skip subtitle/data streams — the remux
// output is fed to an HTML <video> tag, none of it is consumed.
args = append(args,
"-map_metadata", "-1",
"-map_chapters", "-1",
)
// Select specific video and audio streams.
args = append(args, "-map", "0:v:0")
if audioTrackIndex >= 0 {
args = append(args, "-map", fmt.Sprintf("0:a:%d?", audioTrackIndex))
} else {
args = append(args, "-map", "0:a:0?")
}
args = append(args, "-sn", "-dn")
if dvProfile == 7 {
args = append(args, "-bsf:v", "dovi_rpu=strip=1")
} else if (dvProfile == 5 || dvProfile == 8) && tagDVSampleEntry {
// FFmpeg carries the DOVI configuration record into MP4 but otherwise
// labels copied HEVC as hev1. Media3 keys decoder selection from the
// sample entry, so retain an explicit Dolby Vision tag as well.
// dvhe keeps FFmpeg's dvvC box; forcing dvh1 makes FFmpeg 7.1 omit it.
// Only the explicit v3 preserve recipe opts in: legacy web/jellycompat
// consumers keep the pre-v3 hev1 labeling their demuxers accept.
args = append(args, "-tag:v", "dvhe")
}
if transcodeAudio {
// Video copy + stereo AAC encode is effectively single-threaded work.
// ffmpeg's default auto-threading spawns one filter thread per CPU
// core for the implicit downmix/resampler, all idle. Pin to one.
args = append(args,
"-threads", "1",
"-filter_threads", "1",
"-filter_complex_threads", "1",
"-c:v", "copy",
"-c:a", "aac",
"-ac", "2",
"-b:a", "192k",
)
} else {
args = append(args, "-c", "copy")
}
args = append(args,
"-avoid_negative_ts", "make_zero",
"-f", outputFormat,
// delay_moov lets the MP4 muxer inspect the first audio packet before
// writing codec configuration. empty_moov fails immediately for copied
// E-AC-3/Atmos tracks because their frame size is not known at header time.
"-movflags", "frag_keyframe+delay_moov+default_base_moof",
"pipe:1",
)
return args
}
// StartRemux starts an ffmpeg process that copies codecs to a new container.
// When transcodeAudio is false the command is:
//
// ffmpeg -i {input} -c copy -f {format} -movflags frag_keyframe+delay_moov+default_base_moof pipe:1
//
// When transcodeAudio is true video is copied but audio is transcoded to AAC.
// The caller must call Close() when done to clean up resources.
func StartRemux(ctx context.Context, filePath, outputFormat string, seekSeconds float64, transcodeAudio bool, audioTrackIndex int, dvProfile int) (*RemuxSession, error) {
return StartRemuxWithDVMode(ctx, filePath, outputFormat, seekSeconds, transcodeAudio, audioTrackIndex, dvProfile, RemuxDVLegacyAutoV3, "")
}
// StartRemuxWithDVMode starts a remux with explicit Dolby Vision behavior.
// ffmpegPath selects the binary to execute (empty = process-global discovery);
// v3 callers must pass the configured playback path so the strip capability
// promised by the planner's probe holds for the binary that actually runs.
func StartRemuxWithDVMode(ctx context.Context, filePath, outputFormat string, seekSeconds float64, transcodeAudio bool, audioTrackIndex int, dvProfile int, mode RemuxDVMode, ffmpegPath string) (*RemuxSession, error) {
ctx, cancel := context.WithCancel(ctx)
bin := ResolveFFmpegPath(ffmpegPath)
effectiveProfile := dvProfile
tagDVSampleEntry := false
switch mode {
case "", RemuxDVLegacyAutoV3:
effectiveProfile = remuxDVProfile(dvProfile, supportsDoviRPUFilter(bin))
case RemuxDVStripToHDR10V3:
if dvProfile != 7 && dvProfile != 8 {
cancel()
return nil, fmt.Errorf("Dolby Vision HDR10 strip requires profile 7 or 8")
}
if !supportsDoviRPUFilter(bin) {
cancel()
return nil, fmt.Errorf("Dolby Vision HDR10 remux requires the dovi_rpu bitstream filter")
}
// buildRemuxArgs uses profile 7 as the explicit strip sentinel; the
// filter is equally required for a compatible profile 8 base layer.
effectiveProfile = 7
case RemuxDVPreserveV3:
if dvProfile == 7 {
// The remux maps only the base-layer stream, so dual-layer P7
// cannot be preserved: the EL is dropped and its RPUs would
// dangle. Callers must strip to HDR10 or transcode instead.
cancel()
return nil, fmt.Errorf("Dolby Vision profile 7 cannot be preserved in a progressive remux")
}
tagDVSampleEntry = true
case RemuxDVRejectP7V3:
if dvProfile == 7 {
cancel()
return nil, fmt.Errorf("profile 7 remux is not eligible")
}
default:
cancel()
return nil, fmt.Errorf("unknown remux Dolby Vision mode %q", mode)
}
args := buildRemuxArgs(filePath, outputFormat, seekSeconds, transcodeAudio, audioTrackIndex, effectiveProfile, tagDVSampleEntry)
cmd := exec.CommandContext(ctx, bin, args...)
stdout, err := cmd.StdoutPipe()
if err != nil {
cancel()
return nil, fmt.Errorf("stdout pipe: %w", err)
}
if err := cmd.Start(); err != nil {
cancel()
return nil, fmt.Errorf("start ffmpeg: %w", err)
}
return &RemuxSession{
cmd: cmd,
cancel: cancel,
outputPipe: stdout,
}, nil
}
// Read implements io.Reader, piping ffmpeg stdout to the caller.
func (s *RemuxSession) Read(p []byte) (int, error) {
return s.outputPipe.Read(p)
}
// Close stops the ffmpeg process and cleans up all resources.
// It is safe to call Close multiple times.
func (s *RemuxSession) Close() error {
s.cancel()
// Drain the pipe so cmd.Wait does not block.
_, _ = io.Copy(io.Discard, s.outputPipe)
return s.cmd.Wait()
}
// containerMIME maps output format names to MIME types for HTTP responses.
func containerMIME(format string) string {
switch format {
case "mp4":
return "video/mp4"
case "webm":
return "video/webm"
case "matroska":
return "video/x-matroska"
case "mpegts":
return "video/mp2t"
default:
return "application/octet-stream"
}
}
// ServeRemux streams a remuxed file to the HTTP response.
// It starts an ffmpeg remux session and copies the output directly to the
// response writer. The response is streamed (chunked transfer) since the
// total size is not known in advance.
// When transcodeAudio is true, audio is transcoded to AAC while video is copied.
func ServeRemux(w http.ResponseWriter, r *http.Request, filePath, outputFormat string, seekSeconds float64, transcodeAudio bool, audioTrackIndex int, dvProfile int) error {
return ServeRemuxWithDVMode(w, r, filePath, outputFormat, seekSeconds, transcodeAudio, audioTrackIndex, dvProfile, RemuxDVLegacyAutoV3, "")
}
// ServeRemuxWithDVMode streams an explicitly declared Dolby Vision recipe.
// ffmpegPath selects the binary to execute (empty = process-global discovery).
func ServeRemuxWithDVMode(w http.ResponseWriter, r *http.Request, filePath, outputFormat string, seekSeconds float64, transcodeAudio bool, audioTrackIndex int, dvProfile int, mode RemuxDVMode, ffmpegPath string) error {
// Remux output streams for the length of the title; roll the write
// deadline with progress instead of the server's absolute WriteTimeout.
w = httpstream.NewRollingDeadlineWriter(w)
// Check file exists before starting ffmpeg to return a proper 404.
// Headers must be written before streaming begins, so we can't detect
// ffmpeg errors after WriteHeader(200) has been sent.
if _, err := os.Stat(filePath); err != nil {
if os.IsNotExist(err) {
http.Error(w, "file not found", http.StatusNotFound)
return err
}
http.Error(w, "failed to access file", http.StatusInternalServerError)
return err
}
session, err := StartRemuxWithDVMode(r.Context(), filePath, outputFormat, seekSeconds, transcodeAudio, audioTrackIndex, dvProfile, mode, ffmpegPath)
if err != nil {
http.Error(w, "failed to start remux", http.StatusInternalServerError)
return err
}
defer session.Close()
w.Header().Set("Content-Type", containerMIME(outputFormat))
w.Header().Set("Transfer-Encoding", "chunked")
w.WriteHeader(http.StatusOK)
// Stream ffmpeg output to the HTTP response.
buf := make([]byte, 32*1024) // 32 KB buffer
for {
n, readErr := session.Read(buf)
if n > 0 {
if _, writeErr := w.Write(buf[:n]); writeErr != nil {
return nil // Client disconnected.
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
}
if readErr != nil {
return nil // EOF or error — done streaming.
}
}
}