* docs(playback): plan protocol v3 server implementation * docs(playback): incorporate protocol v3 review * feat(playback): implement protocol v3 server * fix(playback): persist empty route diagnostics * feat(playback): harden protocol v3 HDR routing * feat(playback): complete protocol v3 client contract * fix(playback): harden protocol v3 recovery * fix(playback): restore dovi_rpu strip filter for DV remuxes The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg; the probe failed closed on every deployment, disabling the new validated DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1 remux path from main. Restore dovi_rpu across the probe, remux and HLS copy arguments, and the recipe-card constant. Also from review: validate the remux DV mode for every profile (garbage modes on non-P7 sources silently no-opped), reject preserve mode for P7 outright (a base-layer-only remux cannot preserve dual-layer DV), tag dvhe sample entries only for the explicit v3 preserve recipe so legacy web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the token-frozen DV mode in the proxy remux path instead of legacy-auto. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): correct v3 planner policy and contract validation Review fixes to the v3 planner and wire contracts: - Bar Profile 7 sources from the non-strip progressive remux route: a base-layer-only remux can never deliver native dual-layer DV, so the planner no longer emits plans claiming validated Dolby Vision while the executed remux drops the enhancement layer. - Accept the device-quirks feature flag from either capability location, matching every other dual-location feature check. - Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with a degradation warning instead of leaving them unplayable under v3. - Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and wire the previously dead Metered signal into conservative auto rungs. - Degrade to the validated source-quality route instead of a terminal when only an implicit quality reduction demanded an unsupported transcode; explicit user-selected rungs keep terminal behavior. - Bound inner capability lists and strings; compare attempt keys exactly instead of case-folded; make ParseTrackIDV3 strict about canonical numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising burn-in for unknown subtitle codecs; probe every h264 encoder rather than requiring libx264; normalize the file-level bitrate fallback. - Evaluate subtitle renderability against the engine each candidate route executes on, not always media3_direct. - Pin the with-quirks attempt-key preimage arity in the cross-language fixture so the Kotlin client stays in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 control-plane reliability Review fixes to the v3 session, store, and handler layer: - Bound concurrent replans with a slot semaphore: each replan pins a pooled connection for its advisory lock while issuing further store queries from the same pool, so an unbounded recovery storm could turn every connection into a lock holder and deadlock the server. - Make CompleteReplan a real compare-and-swap (base-revision predicate, ErrReplanSupersededV3) and map BeginReplan insert races to a replay instead of a raw unique violation. - Fingerprint start requests (request_digest column): an attempt ID reused with different input is now a 409-style conflict rather than a silent replay, and both replay paths check session liveness so dead sessions surface as retryable terminals. - Pre-delete expired attempt rows on SaveAttempt so a retry during the cleanup window cannot wedge on an unreachable conflict. - Align the in-memory store's semantics with Postgres and add DB-backed planstore tests (SILO_TEST_DATABASE_URL), including a regression test inserting every route-event name against the real CHECK constraint. - Session manager: v3 route-set updates own RemuxDVMode outright so a replan onto an SDR source clears a stale strip mode; replacement reservations survive unrelated legacy stream updates; replacement admission excludes the replaced session explicitly instead of decrementing totals it may no longer be part of; the admission CAS loop is bounded and decider errors are logged. - Map transient store failures to 500s instead of terminal 404/403s; authorize route events via identity-only projections after the rate limiter; keep sanitized diagnostics deterministic. - Merge the server-computed durable plan key into replan exclusions so unreproducible client history cannot re-select the failed route. - Remap tracks only when the effective edition changes (a same-file replan no longer switches audio to a lookalike track) and remap ID-only subtitle selections on edition fallback. - Cache the v3/shadow feature flags for five seconds instead of one settings SELECT per playback request; stop remote transports best-effort when the start call times out; carry dvm/tid claims and the transport-scoped job identity through the legacy audio-change re-mint; index playback_route_events(received_at) for the retention delete; run store maintenance for DB-less deployments too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): reap idle node jobs and gate WebVTT conversion - Add an idle reaper to the transcode node: a job untouched by manifest or segment requests for ten minutes is closed and unregistered. After a v3 replan retires a transport ID, a stale in-flight stream token could resurrect the old job via reconstruct and encode to end-of-file for nobody; jobs waiting on readiness count registration as access and are never reaped mid-wait, and reaping keeps the recipe so a still-valid token reconstructs on the next hit. - Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with 415 before headers are written instead of spawning an ffmpeg command that always fails mid-response, and make the extract-format override fall back to source-driven mapping for bitmap codecs. - Drain error bodies on non-202 node responses so the HTTP transport can reuse connections. - Pin the transcode-dir cleanup separator-boundary semantics with a regression test (a session ID sharing another's prefix must not retain foreign directories). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): close v3 planner policy gaps from review - Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has no rung below 480p/1500kbps, so lower caps were silently exceeded even though the cap is documented as a hard delivery ceiling. - Treat video-only media as audio-compatible instead of forcing an AAC conversion (or an audio_conversion_unsupported terminal) onto a file with no audio stream. Tracks whose codec failed to probe keep the gate. - Only promise a bitmap subtitle sidecar for embedded PGS with an engine that renders embedded bitmap: external/downloaded bitmap and embedded DVD/DVB published artifact URLs that always failed at fetch. They now fall through to burn-in or its terminal. - Accept client_video_transformations_v1 from either client_features or the nested context when validating client-executor transformations, matching the planner's dual-source reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): probe and execute DV remuxes with one ffmpeg binary The v3 transformation registry probed the configured playback.ffmpeg_path while progressive remux execution resolved the process-global discovery path, so a deployment where only one binary carries dovi_rpu could plan a server_dv7_to_hdr10 route and then fail it at stream time. Resolution now goes through a shared ResolveFFmpegPath (configured path first, discovery fallback — the same rule the transcode pipeline already used), the dovi_rpu probe is cached per binary path, and the stream handler and proxy worker pass their configured path into ServeRemuxWithDVMode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 replan identity and control-plane limits - Seed failure-replan track selections from the durable current plan before overlaying the request: after an alternate-version fallback the normalized request still carries requested-edition track IDs, so a replan omitting unchanged tracks was rejected as a track/file mismatch. - Remap ID-only audio selections across edition changes (parse the ID to an index like the subtitle remap already does) instead of leaving a stale file-bound ID to fail validation. - Release the node planner reservation when a prepared remote transport rolls back after the node accepted the job; repeated failed starts could otherwise pin max-job/bandwidth budgets for the full reservation age. - Size the replan semaphore below the PostgreSQL pool via a store capacity advisor: with max_connections at or below the fixed bound, advisory-lock holders could starve the inner store queries they need to finish. - Contain shadow-planner panics with a recover boundary; it runs on a bare goroutine where an escaped panic kills the process for what is telemetry-only work. Document why the memory store's session lock is deliberately a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): serialize node job teardown against reconstructs - Look up and touch manifest/segment sessions in one critical section so the idle reaper cannot unregister a job between the lookup and its liveness refresh. - Re-validate each reap candidate under the per-session lifecycle lock before closing it: Close removes the output directory, and without the lock it could race a token reconstruct and wipe the segments the fresh ffmpeg is writing. - Take the lifecycle lock in handleStop so a stop racing a RequireReady start's readiness wait blocks until registration and tears the job down, instead of 404ing and orphaning the ffmpeg until the reaper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
270 lines
8.8 KiB
Go
270 lines
8.8 KiB
Go
package playback
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"strings"
|
|
"sync"
|
|
"time"
|
|
)
|
|
|
|
var ErrIdempotencyKeyReusedV3 = errors.New("idempotency key reused")
|
|
var ErrPlaybackAttemptExistsV3 = errors.New("playback attempt already exists")
|
|
var ErrStaleReplanLeaseV3 = errors.New("stale replan lease")
|
|
|
|
// ErrReplanSupersededV3 means a CompleteReplan lost the revision compare: a
|
|
// newer replan already moved the attempt past the caller's base revision.
|
|
var ErrReplanSupersededV3 = errors.New("replan superseded")
|
|
|
|
type AttemptRecordV3 struct {
|
|
PlaybackAttemptID string
|
|
SessionID string
|
|
UserID int
|
|
ProfileID string
|
|
RequestedMediaFileID int
|
|
EffectiveMediaFileID int
|
|
CurrentPlanID string
|
|
CurrentReplanRequestID string
|
|
CurrentPlan PlanV3
|
|
NormalizedRequest StartRequestV3
|
|
// RequestDigest fingerprints the normalized start request so an attempt-ID
|
|
// reused with different input is a detectable idempotency violation rather
|
|
// than a silent replay of the old plan.
|
|
RequestDigest string
|
|
ExpiresAt time.Time
|
|
}
|
|
|
|
// AttemptIdentityV3 carries only the ownership columns of an attempt so
|
|
// per-event authorization checks avoid decoding the plan and request JSONB.
|
|
type AttemptIdentityV3 struct {
|
|
PlaybackAttemptID string
|
|
SessionID string
|
|
UserID int
|
|
ProfileID string
|
|
}
|
|
|
|
type RouteEventRecordV3 struct {
|
|
RouteEventV3
|
|
UserID int
|
|
ProfileID string
|
|
ClientName string
|
|
ClientVersion string
|
|
ClientModel string
|
|
}
|
|
|
|
type ReplanLeaseStateV3 string
|
|
|
|
const (
|
|
ReplanLeaseOwnedV3 ReplanLeaseStateV3 = "owned"
|
|
ReplanLeaseInFlightV3 ReplanLeaseStateV3 = "in_flight"
|
|
ReplanLeaseCompletedV3 ReplanLeaseStateV3 = "completed"
|
|
)
|
|
|
|
type ReplanLeaseV3 struct {
|
|
State ReplanLeaseStateV3
|
|
Response json.RawMessage
|
|
}
|
|
|
|
type PlanStoreV3 interface {
|
|
AcquireSessionLock(context.Context, string) (func(), error)
|
|
SaveAttempt(context.Context, AttemptRecordV3) error
|
|
GetAttempt(context.Context, string) (*AttemptRecordV3, error)
|
|
GetAttemptByPlaybackAttemptID(context.Context, string) (*AttemptRecordV3, error)
|
|
GetAttemptIdentity(context.Context, string) (*AttemptIdentityV3, error)
|
|
GetAttemptIdentityByPlaybackAttemptID(context.Context, string) (*AttemptIdentityV3, error)
|
|
BeginReplan(context.Context, string, string, string, string, time.Time) (ReplanLeaseV3, error)
|
|
// CompleteReplan commits a replan atomically; the attempt row is only
|
|
// updated while its current_replan_request_id still equals the caller's
|
|
// base revision, otherwise ErrReplanSupersededV3 is returned.
|
|
CompleteReplan(ctx context.Context, sessionID, requestID, baseReplanRequestID string, response json.RawMessage, record AttemptRecordV3) error
|
|
RecordRouteEvent(context.Context, RouteEventRecordV3) error
|
|
CleanupExpired(context.Context, time.Time) (int64, error)
|
|
}
|
|
|
|
type memoryReplanV3 struct {
|
|
digest string
|
|
base string
|
|
lease time.Time
|
|
completed bool
|
|
response json.RawMessage
|
|
}
|
|
|
|
type MemoryPlanStoreV3 struct {
|
|
mu sync.Mutex
|
|
attempts map[string]AttemptRecordV3
|
|
replans map[string]memoryReplanV3
|
|
events []RouteEventRecordV3
|
|
}
|
|
|
|
func NewMemoryPlanStoreV3() *MemoryPlanStoreV3 {
|
|
return &MemoryPlanStoreV3{attempts: make(map[string]AttemptRecordV3), replans: make(map[string]memoryReplanV3)}
|
|
}
|
|
|
|
// AcquireSessionLock is deliberately a no-op. The store lock exists to
|
|
// serialize replans across processes sharing one PostgreSQL database; the
|
|
// memory store only ever backs a single-process, DB-less deployment, where
|
|
// the handler's own per-session replan mutex already provides the same
|
|
// serialization before this lock is taken.
|
|
func (s *MemoryPlanStoreV3) AcquireSessionLock(context.Context, string) (func(), error) {
|
|
return func() {}, nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) SaveAttempt(_ context.Context, record AttemptRecordV3) error {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
now := time.Now()
|
|
// Expired rows are replaceable, mirroring the Postgres pre-delete: they
|
|
// linger until the hourly cleanup and must not wedge a legitimate retry.
|
|
for sessionID, existing := range s.attempts {
|
|
if existing.ExpiresAt.After(now) {
|
|
continue
|
|
}
|
|
if existing.PlaybackAttemptID == record.PlaybackAttemptID || sessionID == record.SessionID {
|
|
s.deleteAttemptLocked(sessionID)
|
|
}
|
|
}
|
|
for sessionID, existing := range s.attempts {
|
|
if existing.PlaybackAttemptID != record.PlaybackAttemptID && sessionID != record.SessionID {
|
|
continue
|
|
}
|
|
if existing.PlaybackAttemptID == record.PlaybackAttemptID &&
|
|
existing.RequestDigest != "" && record.RequestDigest != "" && existing.RequestDigest != record.RequestDigest {
|
|
return ErrIdempotencyKeyReusedV3
|
|
}
|
|
return ErrPlaybackAttemptExistsV3
|
|
}
|
|
s.attempts[record.SessionID] = record
|
|
return nil
|
|
}
|
|
|
|
// ReplaceAttempt overwrites a session's attempt record unconditionally. It is
|
|
// not part of PlanStoreV3: durable stores treat attempts as insert-once and
|
|
// replan-updated, so only in-memory test setups may rewrite one in place.
|
|
func (s *MemoryPlanStoreV3) ReplaceAttempt(_ context.Context, record AttemptRecordV3) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
s.attempts[record.SessionID] = record
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) deleteAttemptLocked(sessionID string) {
|
|
delete(s.attempts, sessionID)
|
|
for key := range s.replans {
|
|
if strings.HasPrefix(key, sessionID+":") {
|
|
delete(s.replans, key)
|
|
}
|
|
}
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) GetAttemptByPlaybackAttemptID(_ context.Context, attemptID string) (*AttemptRecordV3, error) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
for _, record := range s.attempts {
|
|
if record.PlaybackAttemptID == attemptID && record.ExpiresAt.After(time.Now()) {
|
|
copy := record
|
|
return ©, nil
|
|
}
|
|
}
|
|
return nil, ErrSessionNotFound
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) GetAttempt(_ context.Context, sessionID string) (*AttemptRecordV3, error) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
record, ok := s.attempts[sessionID]
|
|
if !ok || !record.ExpiresAt.After(time.Now()) {
|
|
return nil, ErrSessionNotFound
|
|
}
|
|
copy := record
|
|
return ©, nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) BeginReplan(_ context.Context, sessionID, requestID, digest, baseReplanRequestID string, leaseUntil time.Time) (ReplanLeaseV3, error) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
key := sessionID + ":" + requestID
|
|
existing, ok := s.replans[key]
|
|
if !ok {
|
|
s.replans[key] = memoryReplanV3{digest: digest, base: baseReplanRequestID, lease: leaseUntil}
|
|
return ReplanLeaseV3{State: ReplanLeaseOwnedV3}, nil
|
|
}
|
|
if existing.digest != digest {
|
|
return ReplanLeaseV3{}, ErrIdempotencyKeyReusedV3
|
|
}
|
|
if existing.completed {
|
|
return ReplanLeaseV3{State: ReplanLeaseCompletedV3, Response: append(json.RawMessage(nil), existing.response...)}, nil
|
|
}
|
|
if time.Now().Before(existing.lease) {
|
|
return ReplanLeaseV3{State: ReplanLeaseInFlightV3}, nil
|
|
}
|
|
if existing.base != baseReplanRequestID {
|
|
return ReplanLeaseV3{}, ErrStaleReplanLeaseV3
|
|
}
|
|
existing.lease = leaseUntil
|
|
s.replans[key] = existing
|
|
return ReplanLeaseV3{State: ReplanLeaseOwnedV3}, nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) CompleteReplan(_ context.Context, sessionID, requestID, baseReplanRequestID string, response json.RawMessage, record AttemptRecordV3) error {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
existing, ok := s.attempts[sessionID]
|
|
if !ok {
|
|
return ErrSessionNotFound
|
|
}
|
|
if existing.CurrentReplanRequestID != baseReplanRequestID {
|
|
return ErrReplanSupersededV3
|
|
}
|
|
key := sessionID + ":" + requestID
|
|
entry, ok := s.replans[key]
|
|
if !ok {
|
|
return ErrSessionNotFound
|
|
}
|
|
entry.completed = true
|
|
entry.response = append(json.RawMessage(nil), response...)
|
|
s.replans[key] = entry
|
|
s.attempts[sessionID] = record
|
|
return nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) GetAttemptIdentity(ctx context.Context, sessionID string) (*AttemptIdentityV3, error) {
|
|
record, err := s.GetAttempt(ctx, sessionID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &AttemptIdentityV3{PlaybackAttemptID: record.PlaybackAttemptID, SessionID: record.SessionID, UserID: record.UserID, ProfileID: record.ProfileID}, nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) GetAttemptIdentityByPlaybackAttemptID(ctx context.Context, attemptID string) (*AttemptIdentityV3, error) {
|
|
record, err := s.GetAttemptByPlaybackAttemptID(ctx, attemptID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &AttemptIdentityV3{PlaybackAttemptID: record.PlaybackAttemptID, SessionID: record.SessionID, UserID: record.UserID, ProfileID: record.ProfileID}, nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) RecordRouteEvent(_ context.Context, record RouteEventRecordV3) error {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
s.events = append(s.events, record)
|
|
return nil
|
|
}
|
|
|
|
func (s *MemoryPlanStoreV3) CleanupExpired(_ context.Context, now time.Time) (int64, error) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
var count int64
|
|
for sessionID, record := range s.attempts {
|
|
if !record.ExpiresAt.After(now) {
|
|
delete(s.attempts, sessionID)
|
|
for key := range s.replans {
|
|
if strings.HasPrefix(key, sessionID+":") {
|
|
delete(s.replans, key)
|
|
}
|
|
}
|
|
count++
|
|
}
|
|
}
|
|
return count, nil
|
|
}
|