Files
silo-server/internal/playback/subtitles.go
T
854d07cf8f feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation

* docs(playback): incorporate protocol v3 review

* feat(playback): implement protocol v3 server

* fix(playback): persist empty route diagnostics

* feat(playback): harden protocol v3 HDR routing

* feat(playback): complete protocol v3 client contract

* fix(playback): harden protocol v3 recovery

* fix(playback): restore dovi_rpu strip filter for DV remuxes

The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl
bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg;
the probe failed closed on every deployment, disabling the new validated
DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1
remux path from main. Restore dovi_rpu across the probe, remux and HLS
copy arguments, and the recipe-card constant.

Also from review: validate the remux DV mode for every profile (garbage
modes on non-P7 sources silently no-opped), reject preserve mode for P7
outright (a base-layer-only remux cannot preserve dual-layer DV), tag
dvhe sample entries only for the explicit v3 preserve recipe so legacy
web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the
token-frozen DV mode in the proxy remux path instead of legacy-auto.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): correct v3 planner policy and contract validation

Review fixes to the v3 planner and wire contracts:

- Bar Profile 7 sources from the non-strip progressive remux route: a
  base-layer-only remux can never deliver native dual-layer DV, so the
  planner no longer emits plans claiming validated Dolby Vision while
  the executed remux drops the enhancement layer.
- Accept the device-quirks feature flag from either capability location,
  matching every other dual-location feature check.
- Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with
  a degradation warning instead of leaving them unplayable under v3.
- Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and
  wire the previously dead Metered signal into conservative auto rungs.
- Degrade to the validated source-quality route instead of a terminal
  when only an implicit quality reduction demanded an unsupported
  transcode; explicit user-selected rungs keep terminal behavior.
- Bound inner capability lists and strings; compare attempt keys exactly
  instead of case-folded; make ParseTrackIDV3 strict about canonical
  numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising
  burn-in for unknown subtitle codecs; probe every h264 encoder rather
  than requiring libx264; normalize the file-level bitrate fallback.
- Evaluate subtitle renderability against the engine each candidate
  route executes on, not always media3_direct.
- Pin the with-quirks attempt-key preimage arity in the cross-language
  fixture so the Kotlin client stays in lockstep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 control-plane reliability

Review fixes to the v3 session, store, and handler layer:

- Bound concurrent replans with a slot semaphore: each replan pins a
  pooled connection for its advisory lock while issuing further store
  queries from the same pool, so an unbounded recovery storm could turn
  every connection into a lock holder and deadlock the server.
- Make CompleteReplan a real compare-and-swap (base-revision predicate,
  ErrReplanSupersededV3) and map BeginReplan insert races to a replay
  instead of a raw unique violation.
- Fingerprint start requests (request_digest column): an attempt ID
  reused with different input is now a 409-style conflict rather than a
  silent replay, and both replay paths check session liveness so dead
  sessions surface as retryable terminals.
- Pre-delete expired attempt rows on SaveAttempt so a retry during the
  cleanup window cannot wedge on an unreachable conflict.
- Align the in-memory store's semantics with Postgres and add DB-backed
  planstore tests (SILO_TEST_DATABASE_URL), including a regression test
  inserting every route-event name against the real CHECK constraint.
- Session manager: v3 route-set updates own RemuxDVMode outright so a
  replan onto an SDR source clears a stale strip mode; replacement
  reservations survive unrelated legacy stream updates; replacement
  admission excludes the replaced session explicitly instead of
  decrementing totals it may no longer be part of; the admission CAS
  loop is bounded and decider errors are logged.
- Map transient store failures to 500s instead of terminal 404/403s;
  authorize route events via identity-only projections after the rate
  limiter; keep sanitized diagnostics deterministic.
- Merge the server-computed durable plan key into replan exclusions so
  unreproducible client history cannot re-select the failed route.
- Remap tracks only when the effective edition changes (a same-file
  replan no longer switches audio to a lookalike track) and remap
  ID-only subtitle selections on edition fallback.
- Cache the v3/shadow feature flags for five seconds instead of one
  settings SELECT per playback request; stop remote transports
  best-effort when the start call times out; carry dvm/tid claims and
  the transport-scoped job identity through the legacy audio-change
  re-mint; index playback_route_events(received_at) for the retention
  delete; run store maintenance for DB-less deployments too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): reap idle node jobs and gate WebVTT conversion

- Add an idle reaper to the transcode node: a job untouched by manifest
  or segment requests for ten minutes is closed and unregistered. After
  a v3 replan retires a transport ID, a stale in-flight stream token
  could resurrect the old job via reconstruct and encode to end-of-file
  for nobody; jobs waiting on readiness count registration as access
  and are never reaped mid-wait, and reaping keeps the recipe so a
  still-valid token reconstructs on the next hit.
- Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with
  415 before headers are written instead of spawning an ffmpeg command
  that always fails mid-response, and make the extract-format override
  fall back to source-driven mapping for bitmap codecs.
- Drain error bodies on non-202 node responses so the HTTP transport
  can reuse connections.
- Pin the transcode-dir cleanup separator-boundary semantics with a
  regression test (a session ID sharing another's prefix must not
  retain foreign directories).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): close v3 planner policy gaps from review

- Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has
  no rung below 480p/1500kbps, so lower caps were silently exceeded even
  though the cap is documented as a hard delivery ceiling.
- Treat video-only media as audio-compatible instead of forcing an AAC
  conversion (or an audio_conversion_unsupported terminal) onto a file
  with no audio stream. Tracks whose codec failed to probe keep the gate.
- Only promise a bitmap subtitle sidecar for embedded PGS with an engine
  that renders embedded bitmap: external/downloaded bitmap and embedded
  DVD/DVB published artifact URLs that always failed at fetch. They now
  fall through to burn-in or its terminal.
- Accept client_video_transformations_v1 from either client_features or
  the nested context when validating client-executor transformations,
  matching the planner's dual-source reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): probe and execute DV remuxes with one ffmpeg binary

The v3 transformation registry probed the configured playback.ffmpeg_path
while progressive remux execution resolved the process-global discovery
path, so a deployment where only one binary carries dovi_rpu could plan a
server_dv7_to_hdr10 route and then fail it at stream time. Resolution now
goes through a shared ResolveFFmpegPath (configured path first, discovery
fallback — the same rule the transcode pipeline already used), the
dovi_rpu probe is cached per binary path, and the stream handler and proxy
worker pass their configured path into ServeRemuxWithDVMode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): harden v3 replan identity and control-plane limits

- Seed failure-replan track selections from the durable current plan
  before overlaying the request: after an alternate-version fallback the
  normalized request still carries requested-edition track IDs, so a
  replan omitting unchanged tracks was rejected as a track/file mismatch.
- Remap ID-only audio selections across edition changes (parse the ID to
  an index like the subtitle remap already does) instead of leaving a
  stale file-bound ID to fail validation.
- Release the node planner reservation when a prepared remote transport
  rolls back after the node accepted the job; repeated failed starts
  could otherwise pin max-job/bandwidth budgets for the full reservation
  age.
- Size the replan semaphore below the PostgreSQL pool via a store
  capacity advisor: with max_connections at or below the fixed bound,
  advisory-lock holders could starve the inner store queries they need
  to finish.
- Contain shadow-planner panics with a recover boundary; it runs on a
  bare goroutine where an escaped panic kills the process for what is
  telemetry-only work. Document why the memory store's session lock is
  deliberately a no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcode): serialize node job teardown against reconstructs

- Look up and touch manifest/segment sessions in one critical section so
  the idle reaper cannot unregister a job between the lookup and its
  liveness refresh.
- Re-validate each reap candidate under the per-session lifecycle lock
  before closing it: Close removes the output directory, and without the
  lock it could race a token reconstruct and wipe the segments the fresh
  ffmpeg is writing.
- Take the lifecycle lock in handleStop so a stop racing a RequireReady
  start's readiness wait blocks until registration and tears the job
  down, instead of 404ing and orphaning the ffmpeg until the reaper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:51:27 -04:00

324 lines
9.7 KiB
Go

package playback
import (
"bytes"
"context"
"errors"
"fmt"
"net/http"
"os"
"os/exec"
"strconv"
"strings"
)
// bitmapSubtitleCodecs lists subtitle codecs that cannot be extracted as text
// and must be burned into the video stream.
var bitmapSubtitleCodecs = map[string]bool{
"pgs": true,
"hdmv_pgs_subtitle": true,
"dvd_subtitle": true,
"dvb_subtitle": true,
}
// NeedsBurnIn reports whether the given subtitle codec is bitmap-based and
// requires burning into the video stream (cannot be extracted as text).
func NeedsBurnIn(subtitleCodec string) bool {
return bitmapSubtitleCodecs[strings.ToLower(subtitleCodec)]
}
// pgsSubtitleCodecs lists PGS (Blu-ray bitmap) subtitle codec names. Unlike
// other bitmap codecs, PGS can be extracted losslessly to a .sup elementary
// stream for capable native clients, so burn-in is not the only delivery
// option. The web player burns in all bitmap codecs; DVD/DVB bitmap subs also
// require burn-in for native clients that cannot render them directly.
var pgsSubtitleCodecs = map[string]bool{
"pgs": true,
"hdmv_pgs_subtitle": true,
}
// IsPGS reports whether the given subtitle codec is PGS format.
func IsPGS(codec string) bool {
return pgsSubtitleCodecs[strings.ToLower(codec)]
}
// assSubtitleCodecs lists subtitle codecs that are ASS/SSA format and support
// rich styling (fonts, colors, positioning, karaoke, typesetting).
var assSubtitleCodecs = map[string]bool{
"ass": true,
"ssa": true,
}
// IsASS reports whether the given subtitle codec is ASS/SSA format.
func IsASS(codec string) bool {
return assSubtitleCodecs[strings.ToLower(codec)]
}
// ExtractSubtitle extracts a subtitle track from a media file using ffmpeg.
// It returns the raw subtitle data, the detected format (e.g., "srt", "ass"),
// and any error encountered.
func ExtractSubtitle(ctx context.Context, filePath string, trackIndex int, ffmpegPath ...string) ([]byte, string, error) {
// Determine the output format based on what ffmpeg can extract.
// We default to SRT as a safe text-based format.
outputFormat := "srt"
args := []string{
"-i", filePath,
"-map", fmt.Sprintf("0:s:%d", trackIndex),
"-f", outputFormat,
"pipe:1",
}
ffmpeg := "ffmpeg"
if len(ffmpegPath) > 0 && ffmpegPath[0] != "" {
ffmpeg = ffmpegPath[0]
}
cmd := exec.CommandContext(ctx, ffmpeg, args...)
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, "", fmt.Errorf("ffmpeg subtitle extraction failed: %w (stderr: %s)",
err, truncateStderr(stderr.String()))
}
data := stdout.Bytes()
if len(data) == 0 {
return nil, "", fmt.Errorf("ffmpeg produced empty subtitle output for track %d", trackIndex)
}
return data, outputFormat, nil
}
// ExtractSubtitleWithFormat extracts a subtitle track from a media file in the
// specified output format. Only "srt" and "ass" are allowed as output formats.
func ExtractSubtitleWithFormat(ctx context.Context, filePath string, trackIndex int, outputFormat string, ffmpegPath ...string) ([]byte, error) {
switch outputFormat {
case "srt", "ass":
default:
return nil, fmt.Errorf("unsupported subtitle extraction format: %q (must be \"srt\" or \"ass\")", outputFormat)
}
args := []string{
"-i", filePath,
"-map", fmt.Sprintf("0:s:%d", trackIndex),
"-f", outputFormat,
"pipe:1",
}
ffmpeg := "ffmpeg"
if len(ffmpegPath) > 0 && ffmpegPath[0] != "" {
ffmpeg = ffmpegPath[0]
}
cmd := exec.CommandContext(ctx, ffmpeg, args...)
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("ffmpeg subtitle extraction failed: %w (stderr: %s)",
err, truncateStderr(stderr.String()))
}
data := stdout.Bytes()
if len(data) == 0 {
return nil, fmt.Errorf("ffmpeg produced empty subtitle output for track %d", trackIndex)
}
return data, nil
}
// LoadExternalSubtitleRaw reads an external subtitle file and returns its raw
// contents without any conversion. Used for serving ASS/SSA files directly.
func LoadExternalSubtitleRaw(subtitlePath string) ([]byte, error) {
data, err := os.ReadFile(subtitlePath)
if err != nil {
return nil, fmt.Errorf("read external subtitle: %w", err)
}
return data, nil
}
// ParseSubtitleTrackParam parses a subtitle track URL parameter that may
// contain an optional format extension (e.g. "5" or "5.ass" or "5.vtt").
// Returns the track index and the requested format (empty string if no
// extension was provided).
func ParseSubtitleTrackParam(param string) (int, string, error) {
format := ""
indexStr := param
if dotIdx := strings.LastIndex(param, "."); dotIdx >= 0 {
format = strings.ToLower(param[dotIdx+1:])
indexStr = param[:dotIdx]
}
index, err := strconv.Atoi(indexStr)
if err != nil || index < 0 {
return 0, "", fmt.Errorf("invalid subtitle track parameter: %q", param)
}
return index, format, nil
}
// ServeSubtitle writes subtitle data to the HTTP response with the appropriate
// content type for the given format.
func ServeSubtitle(w http.ResponseWriter, data []byte, format string) {
switch strings.ToLower(format) {
case "ass", "ssa":
w.Header().Set("Content-Type", "text/x-ssa; charset=utf-8")
case "srt", "subrip":
w.Header().Set("Content-Type", "application/x-subrip; charset=utf-8")
default:
w.Header().Set("Content-Type", "text/vtt; charset=utf-8")
}
w.Header().Set("Content-Length", strconv.Itoa(len(data)))
w.WriteHeader(http.StatusOK)
w.Write(data) //nolint:errcheck
}
// LoadExternalSubtitleAsVTT reads a sidecar subtitle file and converts it to
// WebVTT for player consumption.
func LoadExternalSubtitleAsVTT(ctx context.Context, subtitlePath, format string, ffmpegPath ...string) ([]byte, error) {
switch strings.ToLower(format) {
case "srt", "vtt", "webvtt":
data, err := os.ReadFile(subtitlePath)
if err != nil {
return nil, fmt.Errorf("read external subtitle: %w", err)
}
return ConvertToVTT(data, format)
default:
args := []string{
"-i", subtitlePath,
"-f", "webvtt",
"pipe:1",
}
ffmpeg := "ffmpeg"
if len(ffmpegPath) > 0 && ffmpegPath[0] != "" {
ffmpeg = ffmpegPath[0]
}
cmd := exec.CommandContext(ctx, ffmpeg, args...)
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("ffmpeg external subtitle conversion failed: %w (stderr: %s)",
err, truncateStderr(stderr.String()))
}
data := stdout.Bytes()
if len(data) == 0 {
return nil, fmt.Errorf("ffmpeg produced empty subtitle output for %s", subtitlePath)
}
return data, nil
}
}
// ConvertToVTT converts subtitle data from a given format to WebVTT.
// Currently supports conversion from SRT format.
func ConvertToVTT(input []byte, fromFormat string) ([]byte, error) {
switch strings.ToLower(fromFormat) {
case "srt":
return srtToVTT(input), nil
case "vtt", "webvtt":
// Already VTT, return as-is.
return input, nil
default:
return nil, fmt.Errorf("unsupported subtitle format for VTT conversion: %s", fromFormat)
}
}
// ConvertToVTTWithFFmpeg converts subtitle formats that require a real parser
// (notably ASS/SSA) from in-memory data. It keeps downloaded subtitle
// conversion on the same executable path as external sidecar conversion.
func ConvertToVTTWithFFmpeg(ctx context.Context, input []byte, fromFormat, ffmpegPath string) ([]byte, error) {
if converted, err := ConvertToVTT(input, fromFormat); err == nil {
return converted, nil
}
inputFormat := strings.ToLower(strings.TrimSpace(fromFormat))
if inputFormat == "ssa" {
inputFormat = "ass"
}
if inputFormat == "" {
return nil, errors.New("subtitle format is required")
}
if strings.TrimSpace(ffmpegPath) == "" {
ffmpegPath = "ffmpeg"
}
cmd := exec.CommandContext(ctx, ffmpegPath,
"-hide_banner", "-loglevel", "error",
"-f", inputFormat, "-i", "pipe:0",
"-f", "webvtt", "pipe:1",
)
cmd.Stdin = bytes.NewReader(input)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("ffmpeg subtitle conversion failed: %w (stderr: %s)", err, truncateStderr(stderr.String()))
}
if stdout.Len() == 0 {
return nil, errors.New("ffmpeg produced empty subtitle output")
}
return stdout.Bytes(), nil
}
// srtToVTT converts SRT subtitle content to WebVTT format.
// SRT uses commas for millisecond separators; VTT uses periods.
// VTT requires a "WEBVTT" header.
func srtToVTT(input []byte) []byte {
var buf bytes.Buffer
buf.WriteString("WEBVTT\n\n")
lines := strings.SplitSeq(string(input), "\n")
for line := range lines {
// SRT timestamps use comma: 00:01:23,456 --> 00:01:25,789
// VTT timestamps use period: 00:01:23.456 --> 00:01:25.789
if isSRTTimeLine(line) {
line = strings.ReplaceAll(line, ",", ".")
}
buf.WriteString(line)
buf.WriteByte('\n')
}
return buf.Bytes()
}
// isSRTTimeLine checks if a line looks like an SRT timestamp line.
// Format: 00:01:23,456 --> 00:01:25,789
func isSRTTimeLine(line string) bool {
trimmed := strings.TrimSpace(line)
if !strings.Contains(trimmed, "-->") {
return false
}
parts := strings.Split(trimmed, "-->")
if len(parts) != 2 {
return false
}
// Check that the left side looks like a timestamp with a comma.
left := strings.TrimSpace(parts[0])
return strings.Contains(left, ",") && strings.Contains(left, ":")
}
// truncateStderr limits stderr output to a reasonable length for error messages.
func truncateStderr(s string) string {
const maxLen = 500
if len(s) > maxLen {
return s[:maxLen] + "..."
}
return s
}
// FormatSubtitleTrackArg formats a subtitle track index for ffmpeg mapping.
func FormatSubtitleTrackArg(trackIndex int) string {
return "0:s:" + strconv.Itoa(trackIndex)
}