* fix(player): keep text subtitles in sync across copy-mode restarts and sparse cue windows - Rebase already-loaded cues in place when streamOriginSeconds changes (copy-mode session restart) instead of leaving them offset by the delta. - Stop inferring end-of-input from where a window's cues stop; only the known media duration marks EOF, so a dialogue gap no longer silently ends prefetching for the rest of playback. - Anchor the first window fetch to the intended start position (resume target or pending seek) while the element still reports currentTime=0, and reset coverage on forward seeks past the fetched window. * fix(playback): align encoded transcode start with the declared segment boundary A mid-segment start (resume, seek restart, audio switch) spawned ffmpeg at the raw seek position while labeling its first segment with the grid number, whose synthetic-manifest start is up to one segment earlier. hls.js aligns the first fragment's content to that declared position, shifting the whole session's timeline late by seek mod segment_duration (0-2s): subtitles trail dialogue by a constant per-session offset and progress/resume positions drift by the same amount. Snap the ffmpeg start position down to the segment boundary for encoded sessions so declared and produced timelines match exactly; the player still seeks to the precise requested position. Copy-mode sessions serve ffmpeg's real manifest and keep the raw seek. * fix(api): forward http.Flusher through response-writer middleware wrappers Streamed subtitle extracts (and any progressive response) flush per chunk via an http.Flusher assertion, but none of the status-capturing middleware wrappers implemented Flush, so the assertion failed and cues sat in Go's response buffer until ffmpeg finished. On large remuxes where a 600s window takes 20s+ to demux, captions appeared only when the whole window completed instead of within the first seconds. Give every wrapper a Flush() (satisfies plain assertions, including chi's Compress) and Unwrap() (satisfies http.ResponseController). The jellycompat image-proxy tag rewriter flushes only in passthrough mode since it buffers JSON bodies for rewriting. Regression test asserts the API chain forwards Flush end to end. * feat(player): PGS subtitles honor size, position, and background settings Port the tvOS/iOS bitmap-cue styling to the web player. libpgs now decodes in worker mode but draws to a hidden source canvas on the main thread; a compositor detects cue regions from the frame's alpha channel and re-places them on a visible overlay canvas per the shared subtitle appearance settings: size scale (with the 0.85 authored-size compensation), vertical position preset (dialogue-band cues only — floating signs keep authored placement, matching the Apple implementation), and the background box. Font family, text color, and outline are baked into the source pixels and remain inapplicable. * fix(player): anchor PGS position presets to the text overlay's reference frame The initial port used silo-apple's 30/1080 bottom margin and video-relative lower-third/top anchors; the web text overlay anchors to a 16:9 reference frame with 7%/18% offsets that extends into the letterbox for wide content. Use the same anchors so PGS dialogue lands exactly where SRT text does. * feat(player): size PGS cues to match the text subtitle line height Replace the authored-size ladder (0.85 × font-size ratio) with per-cue text-line matching: the region detector reports the tallest text line inside each cue, and the compositor scales the cue so one line of bitmap text renders at the same pixel height as the SRT overlay's font at the current preset. Authored size differences between discs no longer leak through; upscaling is capped at 2.5× to keep small bitmaps from going blurry. * feat(subtitles): opt-in windowed PGS extraction to cut mid-file load latency PGS extracts always demuxed the source from byte 0, so starting a large remux mid-file meant minutes before the first bitmap cue. The web player now opts in to windowed extraction (?windowed=1&position=&duration=) and re-points libpgs at a fresh window on seeks and near coverage end; ffmpeg input-side -ss with -copyts keeps absolute source timestamps. Without the explicit opt-in the endpoint behaves byte-identically, so Apple/Android and other single-fetch consumers are unaffected. ASS remains unconditionally non-windowed (its header only exists at offset 0). * feat(playback): cache extracted PGS subtitle tracks Every selection of an embedded PGS track re-ran a full ffmpeg extract that demuxes the entire source file from byte 0 — minutes for a large remux — and responses were Cache-Control: no-store, so repeat selections, re-watches, and multiple viewers all paid full price. Add a disk cache for full-track .sup extracts under <transcode_dir>/subtitle-cache, created lazily: - Keyed by source path hash + subtitle stream ordinal + source mtime+size (encoded in the filename), so a replaced source file implicitly invalidates its entries; the source is stat'ed on every lookup. - Cache miss: ffmpeg stdout is teed to the response (first viewer still streams progressively, first-byte latency unchanged) and into a temp file that is fsynced and atomically renamed into the cache on clean ffmpeg exit. Any error — ffmpeg failure, client disconnect, tee write failure, or the source changing mid-extract — discards the temp file, so a partial entry is never served. - Cache hit: served via http.ServeContent (Range support, Content-Length, Last-Modified from the source mtime) with a revalidatable Cache-Control instead of no-store. - Concurrent requests for the same in-flight track run their own uncached extract (mutex + in-flight key set) rather than blocking on another client's connection. - Scan-on-commit LRU eviction under a 2 GiB cap (recency tracked by bumping entry mtime on hit; atime is unreliable under relatime), plus sweep of crash-orphaned .part temp files. - Windowed PGS requests (?windowed=) bypass the cache in both directions: their output covers only a slice of the track. Both the integrated API handler and the standalone proxy subtitle path share the same playback.SubtitleCache.ServeSUPExtract helper. VTT (already windowed and fast) and ASS (small) stay uncached. No API surface change. AI-use disclosure: implemented with Claude Code. * fix(playback): check Close error returns in subtitle cache paths Silence errcheck on the cache-hit defer and the test's simulated disk-full Close. AI-use disclosure: implemented with Claude Code. * feat(playback): warm PGS cache in background and window from cached track Windowed PGS requests bypassed the cache entirely, so every window fetch re-demuxed the multi-GB original file. Now a windowed miss kicks off a detached background warm (full-track extract into the cache, at most 2 concurrent server-wide, coalesced with client-driven fills), and once the entry exists windowed extracts read the 15-80MB cached .sup instead — seeks and re-enables become near-instant after the first load. Verified empirically that ffmpeg preserves absolute PTS when windowing a sup input. * feat(player): hold playback while PGS subtitle cues load When a PGS track is enabled (or a seek lands outside the fetched window), extraction takes seconds and dialogue could play unsubtitled. The player now pauses until the renderer's parsed data covers the playhead — tracked via libpgs' parsed-timestamp watermark, the exact predicate it renders by — showing a "Loading subtitles…" indicator after 500ms. User play/pause always wins over the hold, a 20s safety timeout prevents stranding playback, and background prefetch never pauses. If future libpgs versions reshape the observed internals the hook degrades to the old play-through behavior. * perf(player): shrink uncached PGS window to 600s Draining a windowed extract from the source reads the full interleaved container across the window (~1GB per 100s of remux on measured hardware); a 3600s window cost ~12GB of reads per fetch while cold. Once the server cache is warm a window costs milliseconds regardless of size, so smaller windows only add trivially cheap re-fetches. * feat(subtitles): burn in PGS/bitmap subtitles for the web player The web player rendered PGS client-side via libpgs, which required extracting the .sup track — a cold ffmpeg demux that took seconds even windowed, since c:s copy still reads the whole interleaved container up to the playhead. Every other server (Plex, Jellyfin default, Emby) burns image subtitles into the video instead, and that is the only path with no per-seek extraction cost. Selecting a bitmap subtitle (PGS/DVD/DVB) now restarts the transcode with subtitle_burn_in at the current aligned position, reusing the same restart machinery as an audio/quality switch so the segment-boundary timeline alignment holds. The server composites the decoded subtitle onto the video with an overlay filter_complex graph (libass's subtitles= filter is text-only); overlay runs at native resolution before any target scaling, and hardware pipelines round-trip through CPU like the text path. Burn-in forces a video encode, so copy-video recipes are upgraded to h264 both client- and server-side. Text subtitles keep the instant, styled, client-side path. The .sup streaming endpoints, cache, and windowing are retained for the Apple client, which renders PGS natively. The now-dead web PGS stack (usePGSSubtitles, pgsPlacement, libpgs dep) is removed. Tradeoff: bitmap subtitles no longer honor web appearance settings (baked into the video) and toggling one restarts the transcode (~1-2s buffering), matching Plex behavior. * fix(player): rebuild text subtitle track when turning off PGS burn-in Selecting an SRT track that turned off bitmap burn-in restarted the transcode, and the client TextTrack built in that same moment was orphaned when the <video> element reloaded, so the subtitles never rendered (and a seek could not recover the dead track). Rebuild the text track once the new stream settles, gated on the burn-in-off transition so quality/audio switches and copy-mode seek restarts keep their subtitles without a needless re-extract. * fix(player): render web subtitles behind the control HUD The text subtitle overlay sat at z-20, above the controls layer (z-10), so cues painted over the bottom HUD and cluttered the control bar. Drop it to z-[5] — above the video, below the controls — so the HUD paints over the cues while it is visible. When controls are hidden the whole controls layer is opacity-0, so cues remain fully visible. * feat(player): lift web subtitles above the control bar while it's visible Rather than hiding bottom-anchored cues behind the HUD, raise them just above the control bar (measured height + a small gap) whenever the bar is visible in the foreground player, then settle them back when it hides. The bar is a roughly fixed pixel height while the cue offset scales with the player, so the bar is measured via ResizeObserver rather than hardcoded. Top-anchored cues never collide with the bottom HUD, so they stay put. z-[5] is retained as a safety so any residual overlap tucks behind the bar. * fix(player): coalesce same-tick transcode restarts into one dispatch Starting playback with a persisted bitmap subtitle fired transcode/start twice within milliseconds: the auto-start effect dispatched before subtitle auto-selection restored the burn-in, whose effect then forced a second start. The first request was already on the wire (no abort signal was passed to fetch), so the server spawned an ffmpeg only to kill it for the second start — visible in production as an ffmpeg exit error ~1ms after every such session start, and slowing time to first frame. Defer the network dispatch by one macrotask so back-to-back restart calls in a tick collapse into a single request carrying the final parameters; state updates stay synchronous. Pass the abort signal into playerFetch so a superseded in-flight request is actually cancelled, and drop any deferred dispatch on unmount so a stray transcode/start cannot land after the session's exit DELETE. * fix(catalog): resolve effective subtitle defaults for movie item details Movie pre-play subtitle selectors were missing the effective defaults (including per-item overrides saved from a previous play) that episodes and watch payloads already resolve. Extract applyToItemDetail/ applyToWatchDetail helpers and apply defaults for movies in buildMediaItemDetail. The SubtitlesPopover now also eagerly loads downloaded subtitles when the saved preference points at one so the closed trigger's Auto summary reflects the override. * feat(player): scale subtitle font size with the rendered video Replace fixed rem font sizes with px values defined at a 720px 16:9 reference height, scaled proportionally with the actually-rendered video (object-fit: contain) so subtitles keep the same relative size as the window grows or shrinks, with a 12px legibility floor. Rename useSubtitlePositionStyle to useSubtitleLayout, returning both the position style and the font scale, and add unit tests for the appearance helpers. * fix(player): satisfy strict index checks in transcode quality test * feat(player): let the pre-play Auto option clear the saved subtitle override A manual in-player subtitle selection persists as an 'always' override for that movie/series, but nothing in the UI could undo it — auto selection stayed pinned to the chosen track forever. Choosing 'Auto' in the pre-play subtitles popover now also deletes the stored preference (movie content ID / episode series ID) and invalidates item details so profile-level auto selection applies again. * feat(player): persist pre-play subtitle selections as the item override Choosing a track (or Off) in the pre-play subtitles popover only lived in component state: it applied to that playback session but vanished on returning to the detail page. Persist it through PUT /subtitle-prefs — the same 'always'/'off' override a manual in-player selection saves — keyed by movie content ID or episode series ID, and invalidate item details so the effective defaults reflect it immediately. * feat(ui): show the saved subtitle override and richer pre-play pill summaries A stored per-item override displayed as 'Auto: <language>', hiding both that an override exists and which track it is. The pre-play subtitle pill now shows the resolved track directly (name with (SDH)/(Forced) markers plus format, skipping markers the name already carries), the matching list row gets the checkmark instead of Auto, and the Auto row reads 'Reset to profile defaults'. Subtitle, audio, edition, and version pill summaries also truncate much later (max-w-44/sm:max-w-64). * fix(player): recover text subtitles from stream reloads and failed window fetches Three failure modes could silently freeze or stop web text subtitles: - A stream restart (seek-triggered transcode restart, quality/audio switch) reloads the <video> element and can orphan the programmatic TextTrack — cuechange stops firing and the last cue freezes on screen. Only the PGS-burn-in-off transition rebuilt the track. Now every settled stream URL change bumps the generation, and the rebuild carries loaded cues (converted back to source time) and window coverage over so it costs no refetch. - The sliding-window fetcher committed windowEnd before the fetch ran, so a failed or hung window counted as covered and was never retried — subtitles silently stopped for up to 10 minutes. Coverage now commits only after the window streams in fully; failures leave the range uncovered and retry after a 5s backoff. - A hung extraction (one fetch in flight at a time, no deadline) blocked every future window for the session. Reads now arm a 30s stall timer that aborts a response which stops delivering chunks; slow-but- progressing streams keep resetting it. Diagnosed from a session where ffmpeg took 69s to stream one subtitle window and a transcode restart landed mid-fetch, freezing the active cue. * fix(playback): keep subtitle selections stable across file changes * fix(web): clarify subtitle labels and positioning * fix(player): hide HUD when pointer leaves * fix(web): tidy subtitle track badges * fix(playback): remap audio tracks across file versions * fix(web): tidy audio track labels * docs(playback): clarify bitmap subtitle appearance * fix(playback): preserve selection state on restart * fix(http): preserve response state across flushes * fix(web): preserve pending quality for burn-in * fix(playback): preserve subtitle inventory identity --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
604 lines
21 KiB
Go
604 lines
21 KiB
Go
package handlers
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"log/slog"
|
|
"net/http"
|
|
"os"
|
|
"strconv"
|
|
"strings"
|
|
|
|
"github.com/go-chi/chi/v5"
|
|
|
|
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
|
|
"github.com/Silo-Server/silo-server/internal/config"
|
|
evt "github.com/Silo-Server/silo-server/internal/events"
|
|
"github.com/Silo-Server/silo-server/internal/models"
|
|
"github.com/Silo-Server/silo-server/internal/playback"
|
|
"github.com/Silo-Server/silo-server/internal/subtitles"
|
|
)
|
|
|
|
// FilePathResolver looks up a media file by its ID.
|
|
type FilePathResolver interface {
|
|
GetByID(ctx context.Context, id int) (*models.MediaFile, error)
|
|
}
|
|
|
|
// StreamHandler handles HTTP endpoints for streaming media content.
|
|
type StreamHandler struct {
|
|
sessionMgr SessionManagerInterface
|
|
fileResolver FilePathResolver
|
|
MissingMarker MissingFileMarker
|
|
EventsHub *evt.Hub
|
|
AdminStore PlaybackAdminStore
|
|
SessionSyncer PlaybackSessionSyncer
|
|
// TM is the shared transcode/reconstruct manager (same instance as the
|
|
// PlaybackHandler's). It lets a direct/remux stream rebuild its playback
|
|
// Session from the recipe card after a server restart instead of 404-ing.
|
|
// May be nil (tests / minimal setups) — reconstruct is then simply off.
|
|
TM *playback.TranscodeManager
|
|
// JWTSecret verifies the stream token carried on the serve URL (?st=), which
|
|
// is the reconstruction descriptor for direct/remux after a restart. Empty
|
|
// disables token-based reconstruct (tests / minimal setups).
|
|
JWTSecret string
|
|
// PlaybackConfig returns the current playback config; read it through
|
|
// ffmpegPath(). May be nil (tests).
|
|
PlaybackConfig func() config.PlaybackConfig
|
|
// SubtitleCache stores full-track PGS (.sup) extracts under the transcode
|
|
// dir so repeat selections skip the whole-file ffmpeg demux. May be nil
|
|
// (tests / minimal setups) — extraction then always streams uncached.
|
|
SubtitleCache *playback.SubtitleCache
|
|
SubtitleRepo subtitles.Repository // optional; enables S3-sourced subtitles
|
|
S3Client subtitles.S3Client // optional; needed for fetching S3 subtitles
|
|
S3Bucket string // bucket for subtitle storage
|
|
}
|
|
|
|
// ffmpegPath returns the currently configured ffmpeg binary path.
|
|
func (h *StreamHandler) ffmpegPath() string {
|
|
if h.PlaybackConfig != nil {
|
|
return h.PlaybackConfig().FFmpegPath
|
|
}
|
|
return ""
|
|
}
|
|
|
|
// NewStreamHandler creates a new StreamHandler backed by the given session
|
|
// manager and file resolver.
|
|
func NewStreamHandler(sessionMgr SessionManagerInterface, fileResolver FilePathResolver) *StreamHandler {
|
|
return &StreamHandler{
|
|
sessionMgr: sessionMgr,
|
|
fileResolver: fileResolver,
|
|
// A bare manager (no recipe store) behaves as "no reconstruct" — plain
|
|
// GetSession + ownership — so HandleStream has a single code path. The
|
|
// router overwrites this with the shared manager to enable reconstruct.
|
|
TM: playback.NewTranscodeManager(),
|
|
}
|
|
}
|
|
|
|
// HandleStream serves the video stream for a playback session.
|
|
// For direct play: serves the file with HTTP byte-range support.
|
|
// For remux: starts an ffmpeg remux and streams the output.
|
|
// For transcode: returns 400 (transcode uses manifest/segment endpoints).
|
|
func (h *StreamHandler) HandleStream(w http.ResponseWriter, r *http.Request) {
|
|
userID := apimw.GetUserID(r.Context())
|
|
if userID == 0 {
|
|
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
|
|
return
|
|
}
|
|
|
|
sessionID := chi.URLParam(r, "session_id")
|
|
if sessionID == "" {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Session ID is required")
|
|
return
|
|
}
|
|
setPlaybackSessionLogContext(r, sessionID)
|
|
|
|
// Look up the session, reconstructing it from the recipe card on a not-found
|
|
// miss (e.g. after a server restart) so a direct/remux stream resumes instead
|
|
// of 404-ing. The client re-supplies its position (HTTP Range for direct, the
|
|
// ?seek= query for remux), so no runtime beyond the Session needs rebuilding.
|
|
// Without a token (or signing secret) reconstruct is off, collapsing to a
|
|
// plain GetSession + ownership check.
|
|
card := streamCardFromToken(r.URL.Query().Get(streamTokenParam), sessionID, h.JWTSecret)
|
|
session, status := h.TM.LoadOrReconstructSession(r.Context(), h.sessionMgr.GetSession, sessionID, userID, card)
|
|
switch status {
|
|
case playback.SessionMissing:
|
|
writePlaybackSessionNotFound(w)
|
|
return
|
|
case playback.SessionLoadFailed:
|
|
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load playback session")
|
|
return
|
|
case playback.SessionForbidden:
|
|
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
|
|
return
|
|
}
|
|
|
|
file, err := h.fileResolver.GetByID(r.Context(), session.MediaFileID)
|
|
if err != nil {
|
|
if isPlaybackFileLookupMissing(err) {
|
|
h.abortPlaybackSession(r.Context(), session)
|
|
writeError(w, http.StatusNotFound, "not_found", "Media file not found")
|
|
return
|
|
}
|
|
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load media file")
|
|
return
|
|
}
|
|
if file == nil {
|
|
h.abortPlaybackSession(r.Context(), session)
|
|
writeError(w, http.StatusNotFound, "not_found", "Media file not found")
|
|
return
|
|
}
|
|
if err := preflightPlaybackFile(r.Context(), file, h.MissingMarker, h.EventsHub); err != nil {
|
|
if isPlaybackFileMissing(err) {
|
|
h.abortPlaybackSession(r.Context(), session)
|
|
}
|
|
writePlaybackFilePreflightError(w, err)
|
|
return
|
|
}
|
|
|
|
switch session.PlayMethod {
|
|
case playback.PlayDirect:
|
|
if err := h.sessionMgr.BeginTransport(sessionID); err == nil {
|
|
defer func() {
|
|
_ = h.sessionMgr.EndTransport(sessionID)
|
|
}()
|
|
}
|
|
if err := playback.ServeDirectPlay(w, r, file.FilePath); err != nil {
|
|
h.handleTransportStartFailure(r.Context(), session, file, err)
|
|
}
|
|
|
|
case playback.PlayRemux:
|
|
if err := h.sessionMgr.BeginTransport(sessionID); err == nil {
|
|
defer func() {
|
|
_ = h.sessionMgr.EndTransport(sessionID)
|
|
}()
|
|
}
|
|
seekSeconds := 0.0
|
|
if seekStr := r.URL.Query().Get("seek"); seekStr != "" {
|
|
if s, err := strconv.ParseFloat(seekStr, 64); err == nil && s >= 0 {
|
|
seekSeconds = s
|
|
}
|
|
}
|
|
if err := playback.ServeRemux(w, r, file.FilePath, "mp4", seekSeconds, session.TranscodeAudio, session.AudioTrackIndex, file.PrimaryDVProfile()); err != nil {
|
|
h.handleTransportStartFailure(r.Context(), session, file, err)
|
|
}
|
|
|
|
case playback.PlayTranscode:
|
|
writeError(w, http.StatusBadRequest, "bad_request",
|
|
"Transcode streams use manifest/segment endpoints")
|
|
|
|
default:
|
|
writeError(w, http.StatusInternalServerError, "internal_error",
|
|
"Unknown play method")
|
|
}
|
|
}
|
|
|
|
// HandleSubtitle extracts a subtitle track from the media file associated with
|
|
// a playback session and serves it as WebVTT or raw ASS depending on the
|
|
// URL extension (e.g. /subtitles/2.ass or /subtitles/2.vtt).
|
|
func (h *StreamHandler) HandleSubtitle(w http.ResponseWriter, r *http.Request) {
|
|
userID := apimw.GetUserID(r.Context())
|
|
if userID == 0 {
|
|
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
|
|
return
|
|
}
|
|
|
|
sessionID := chi.URLParam(r, "session_id")
|
|
if sessionID == "" {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Session ID is required")
|
|
return
|
|
}
|
|
setPlaybackSessionLogContext(r, sessionID)
|
|
|
|
trackParam := chi.URLParam(r, "track")
|
|
trackIndex, _, err := playback.ParseSubtitleTrackParam(trackParam)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Invalid subtitle track index")
|
|
return
|
|
}
|
|
|
|
session, err := h.sessionMgr.GetSession(sessionID)
|
|
if err != nil {
|
|
writePlaybackSessionNotFound(w)
|
|
return
|
|
}
|
|
|
|
if session.UserID != userID {
|
|
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
|
|
return
|
|
}
|
|
|
|
fileID, err := subtitleSourceFileID(r, session)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
|
|
return
|
|
}
|
|
file, err := h.fileResolver.GetByID(r.Context(), fileID)
|
|
if err != nil || file == nil {
|
|
writeError(w, http.StatusNotFound, "not_found", "Media file not found")
|
|
return
|
|
}
|
|
|
|
externalCount := len(file.ExternalSubtitles)
|
|
if trackIndex < externalCount {
|
|
sub := file.ExternalSubtitles[trackIndex]
|
|
|
|
// Serve ASS/SSA external subtitles as raw data for client-side rendering.
|
|
if playback.IsASS(sub.Format) {
|
|
data, err := playback.LoadExternalSubtitleRaw(sub.Path)
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "internal_error",
|
|
"Failed to load external subtitle")
|
|
return
|
|
}
|
|
playback.ServeSubtitle(w, data, "ass")
|
|
return
|
|
}
|
|
|
|
vttData, err := playback.LoadExternalSubtitleAsVTT(r.Context(), sub.Path, sub.Format, h.ffmpegPath())
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "internal_error",
|
|
"Failed to load external subtitle")
|
|
return
|
|
}
|
|
playback.ServeSubtitle(w, vttData, "vtt")
|
|
return
|
|
}
|
|
|
|
embeddedIndex := trackIndex - externalCount
|
|
|
|
// Check embedded tracks.
|
|
if embeddedIndex < len(file.SubtitleTracks) {
|
|
track := file.SubtitleTracks[embeddedIndex]
|
|
// PGS is the one bitmap codec we can deliver without burn-in: the
|
|
// track is copied losslessly into a .sup stream and rendered
|
|
// client-side. DVD/DVB bitmap subs still require burn-in.
|
|
if playback.NeedsBurnIn(track.Codec) && !playback.IsPGS(track.Codec) {
|
|
writeError(w, http.StatusBadRequest, "bad_request",
|
|
"Bitmap subtitle tracks cannot be extracted as text")
|
|
return
|
|
}
|
|
|
|
// Dedicated streaming extract — ffmpeg seeks to the current
|
|
// playback position and pipes cues to the response as they're
|
|
// demuxed, so the first byte lands within ~1s even on network
|
|
// storage. Works identically for direct-play, remux, and
|
|
// transcode because it doesn't depend on any other ffmpeg.
|
|
h.streamEmbeddedSubtitle(w, r, file, embeddedIndex, session)
|
|
return
|
|
}
|
|
|
|
// Check downloaded subtitles (from S3).
|
|
if h.SubtitleRepo != nil && h.S3Client != nil {
|
|
downloaded, err := h.SubtitleRepo.ListDownloadedSubtitles(r.Context(), file.ID)
|
|
if err != nil {
|
|
// A DB failure here must not masquerade as "track not found":
|
|
// surface it as an internal error (with a server-side signal)
|
|
// so the real failure is diagnosable instead of looking like an
|
|
// intermittent 404 to the client.
|
|
slog.ErrorContext(r.Context(), "list downloaded subtitles failed", "component", "api",
|
|
"file_id", file.ID,
|
|
"track", trackIndex,
|
|
"error", err,
|
|
)
|
|
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list downloaded subtitles")
|
|
return
|
|
}
|
|
|
|
downloadedIndex := embeddedIndex - len(file.SubtitleTracks)
|
|
if downloadedIndex >= 0 && downloadedIndex < len(downloaded) {
|
|
dl := downloaded[downloadedIndex]
|
|
data, err := h.S3Client.GetObject(r.Context(), h.S3Bucket, dl.S3Key)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadGateway, "s3_error", "Failed to load subtitle from storage")
|
|
return
|
|
}
|
|
|
|
// Serve ASS/SSA downloaded subtitles as raw data.
|
|
if playback.IsASS(string(dl.Format)) {
|
|
playback.ServeSubtitle(w, data, "ass")
|
|
return
|
|
}
|
|
|
|
// If the subtitle is already VTT, serve directly.
|
|
if dl.Format == subtitles.FormatVTT {
|
|
playback.ServeSubtitle(w, data, "vtt")
|
|
return
|
|
}
|
|
|
|
// Convert to VTT using the playback conversion pipeline.
|
|
vttData, err := playback.ConvertToVTT(data, string(dl.Format))
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "convert_error", "Failed to convert subtitle")
|
|
return
|
|
}
|
|
playback.ServeSubtitle(w, vttData, "vtt")
|
|
return
|
|
}
|
|
}
|
|
|
|
writeError(w, http.StatusNotFound, "not_found", "Subtitle track not found")
|
|
}
|
|
|
|
// subtitleSourceFileID pins a subtitle URL to the file whose track list was
|
|
// used to create it. A quality/seek restart may change session.MediaFileID to
|
|
// an alternate version; interpreting the old combined track index against the
|
|
// alternate file can silently serve a different language. Only the session's
|
|
// requested or current effective file may be named by the authenticated URL.
|
|
func subtitleSourceFileID(r *http.Request, session *playback.Session) (int, error) {
|
|
if session == nil {
|
|
return 0, errors.New("playback session is required")
|
|
}
|
|
raw := strings.TrimSpace(r.URL.Query().Get("file_id"))
|
|
if raw == "" {
|
|
return session.MediaFileID, nil
|
|
}
|
|
fileID, err := strconv.Atoi(raw)
|
|
if err != nil || fileID <= 0 {
|
|
return 0, errors.New("invalid subtitle source file")
|
|
}
|
|
if fileID != session.MediaFileID && fileID != session.RequestedMediaFileID {
|
|
return 0, errors.New("subtitle source file does not belong to playback session")
|
|
}
|
|
return fileID, nil
|
|
}
|
|
|
|
// HandleSubtitleFonts extracts embedded container font attachments for ASS/SSA
|
|
// playback. The web player loads these bytes into JASSUB before creating the
|
|
// renderer so libass can resolve script font names deterministically.
|
|
func (h *StreamHandler) HandleSubtitleFonts(w http.ResponseWriter, r *http.Request) {
|
|
userID := apimw.GetUserID(r.Context())
|
|
if userID == 0 {
|
|
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
|
|
return
|
|
}
|
|
|
|
sessionID := chi.URLParam(r, "session_id")
|
|
if sessionID == "" {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Session ID is required")
|
|
return
|
|
}
|
|
setPlaybackSessionLogContext(r, sessionID)
|
|
|
|
session, err := h.sessionMgr.GetSession(sessionID)
|
|
if err != nil {
|
|
writePlaybackSessionNotFound(w)
|
|
return
|
|
}
|
|
if session.UserID != userID {
|
|
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
|
|
return
|
|
}
|
|
|
|
fileID, err := subtitleSourceFileID(r, session)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
|
|
return
|
|
}
|
|
file, err := h.fileResolver.GetByID(r.Context(), fileID)
|
|
if err != nil {
|
|
writeError(w, http.StatusNotFound, "not_found", "Media file not found")
|
|
return
|
|
}
|
|
if file == nil {
|
|
writeError(w, http.StatusNotFound, "not_found", "Media file not found")
|
|
return
|
|
}
|
|
if err := preflightPlaybackFile(r.Context(), file, h.MissingMarker, h.EventsHub); err != nil {
|
|
if isPlaybackFileMissing(err) {
|
|
h.abortPlaybackSession(r.Context(), session)
|
|
}
|
|
writePlaybackFilePreflightError(w, err)
|
|
return
|
|
}
|
|
|
|
trackParam := chi.URLParam(r, "track")
|
|
trackIndex, _, err := playback.ParseSubtitleTrackParam(trackParam)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Invalid subtitle track index")
|
|
return
|
|
}
|
|
|
|
embeddedIndex := trackIndex - len(file.ExternalSubtitles)
|
|
if embeddedIndex < 0 || embeddedIndex >= len(file.SubtitleTracks) {
|
|
writeError(w, http.StatusNotFound, "not_found", "Embedded subtitle track not found")
|
|
return
|
|
}
|
|
if !playback.IsASS(file.SubtitleTracks[embeddedIndex].Codec) {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "Subtitle font bundles are only available for ASS/SSA tracks")
|
|
return
|
|
}
|
|
|
|
fonts, err := playback.ExtractAttachedSubtitleFonts(r.Context(), file.FilePath, h.ffmpegPath())
|
|
if err != nil {
|
|
slog.WarnContext(r.Context(), "subtitle font extraction failed", "component", "api",
|
|
"file_id", file.ID,
|
|
"track", trackIndex,
|
|
"error", err,
|
|
)
|
|
writeError(w, http.StatusInternalServerError, "font_extract_failed", "Failed to extract subtitle fonts")
|
|
return
|
|
}
|
|
|
|
w.Header().Set("Content-Type", "application/json")
|
|
w.Header().Set("Access-Control-Allow-Origin", "*")
|
|
w.Header().Set("Cache-Control", "no-store")
|
|
if err := json.NewEncoder(w).Encode(playback.EncodeSubtitleFontBundle(fonts)); err != nil {
|
|
slog.WarnContext(r.Context(), "subtitle font response encode failed", "component", "api", "error", err)
|
|
}
|
|
}
|
|
|
|
func (h *StreamHandler) syncSessionsNow(ctx context.Context, reason string) {
|
|
if h == nil || h.SessionSyncer == nil {
|
|
return
|
|
}
|
|
if err := h.SessionSyncer.SyncNow(ctx); err != nil {
|
|
slog.ErrorContext(ctx, "failed to sync sessions", "component", "api", "reason", reason, "error", err)
|
|
}
|
|
}
|
|
|
|
func (h *StreamHandler) finalizeSessionAbort(ctx context.Context, session *playback.Session, syncNow bool, syncReason string) {
|
|
if h == nil || session == nil || session.ID == "" {
|
|
return
|
|
}
|
|
if ctx == nil {
|
|
ctx = context.Background()
|
|
}
|
|
|
|
if h.AdminStore != nil {
|
|
if err := h.AdminStore.DeleteSession(ctx, session.ID); err != nil {
|
|
slog.ErrorContext(ctx, "failed to delete synced session", "component", "api", "session", session.ID, "error", err)
|
|
}
|
|
}
|
|
if syncNow {
|
|
h.syncSessionsNow(ctx, syncReason)
|
|
}
|
|
}
|
|
|
|
func (h *StreamHandler) abortPlaybackSession(ctx context.Context, session *playback.Session) {
|
|
if h == nil || session == nil || session.ID == "" {
|
|
return
|
|
}
|
|
if err := h.sessionMgr.StopSession(session.ID); err != nil {
|
|
return
|
|
}
|
|
h.finalizeSessionAbort(ctx, session, true, "stream_abort")
|
|
}
|
|
|
|
func (h *StreamHandler) handleTransportStartFailure(ctx context.Context, session *playback.Session, file *models.MediaFile, err error) {
|
|
if ctx == nil || session == nil || err == nil {
|
|
return
|
|
}
|
|
if preflightErr := preflightPlaybackFile(ctx, file, h.MissingMarker, h.EventsHub); preflightErr != nil {
|
|
err = preflightErr
|
|
}
|
|
if isPlaybackFileMissing(err) || errors.Is(err, os.ErrNotExist) {
|
|
h.abortPlaybackSession(ctx, session)
|
|
return
|
|
}
|
|
slog.WarnContext(ctx, "stream transport startup failed", "component", "api",
|
|
"session", session.ID,
|
|
"file_id", session.MediaFileID,
|
|
"error", err,
|
|
"playback_session_id", session.ID,
|
|
)
|
|
}
|
|
|
|
// streamEmbeddedSubtitle runs a dedicated ffmpeg for a single embedded
|
|
// track, seeked to the best-known playback position, and pipes its
|
|
// stdout directly to w. Because this ffmpeg is independent of the video
|
|
// pipeline, it works the same for direct play, remux, and transcode.
|
|
func (h *StreamHandler) streamEmbeddedSubtitle(w http.ResponseWriter, r *http.Request, file *models.MediaFile, embeddedIndex int, session *playback.Session) {
|
|
track := file.SubtitleTracks[embeddedIndex]
|
|
outFormat := "vtt"
|
|
switch {
|
|
case playback.IsASS(track.Codec):
|
|
outFormat = "ass"
|
|
case playback.IsPGS(track.Codec):
|
|
outFormat = "sup"
|
|
}
|
|
|
|
// ASS is fetched exactly once and consumed whole by its client-side
|
|
// renderer (JASSUB), so it must never be windowed. PGS defaults to
|
|
// the same whole-track behavior, but a client that manages its own
|
|
// sliding window (the web player's libpgs hook) opts in explicitly
|
|
// with ?windowed=1 + ?position=/?duration=; there is deliberately no
|
|
// session-position fallback for sup — an implicit window would
|
|
// silently drop cues for clients that fetch once. Note
|
|
// subtitleSeekPosition falls back to the session's last reported
|
|
// position even without a ?position= query — relying on
|
|
// StreamExtractSubtitle's codec guard alone would still log a
|
|
// misleading nonzero seek here.
|
|
var seek, duration float64
|
|
var allowWindow bool
|
|
switch outFormat {
|
|
case "vtt":
|
|
seek = subtitleSeekPosition(r, session)
|
|
duration = subtitleWindowDuration(r)
|
|
case "sup":
|
|
allowWindow, seek, duration = playback.PGSWindowRequest(r.URL.Query())
|
|
}
|
|
slog.InfoContext(r.Context(), "subtitle stream requested", "component", "api",
|
|
"file_id", file.ID,
|
|
"embedded_index", embeddedIndex,
|
|
"track_language", track.Language,
|
|
"track_codec", track.Codec,
|
|
"track_probed_index", track.Index,
|
|
"seek_seconds", seek,
|
|
"duration_seconds", duration,
|
|
)
|
|
|
|
opts := playback.StreamExtractOpts{
|
|
InputPath: file.FilePath,
|
|
TrackIndex: embeddedIndex,
|
|
SourceCodec: track.Codec,
|
|
SeekSeconds: seek,
|
|
DurationSeconds: duration,
|
|
AllowWindow: allowWindow,
|
|
FFmpegPath: h.ffmpegPath(),
|
|
}
|
|
|
|
w.Header().Set("Access-Control-Allow-Origin", "*")
|
|
|
|
// Full-track PGS extracts are expensive (whole-file demux) and byte-
|
|
// identical across requests, so they are served from / teed into the
|
|
// subtitle cache; windowed PGS requests extract their slice from the
|
|
// cached full track when present (warming it in the background when
|
|
// not). All other formats stream uncached: VTT is already windowed
|
|
// and fast, ASS is small.
|
|
if outFormat == "sup" {
|
|
err := h.SubtitleCache.ServeSUPExtract(w, r, opts, playback.StreamExtractSubtitle)
|
|
playback.LogSubtitleStreamError(r.Context(), err, file.ID, embeddedIndex)
|
|
return
|
|
}
|
|
|
|
switch outFormat {
|
|
case "ass":
|
|
w.Header().Set("Content-Type", "text/x-ssa; charset=utf-8")
|
|
default:
|
|
w.Header().Set("Content-Type", "text/vtt; charset=utf-8")
|
|
}
|
|
w.Header().Set("Cache-Control", "no-store")
|
|
w.WriteHeader(http.StatusOK)
|
|
|
|
opts.Writer = w
|
|
if err := playback.StreamExtractSubtitle(r.Context(), opts); err != nil {
|
|
// Headers already committed — best we can do is log and let
|
|
// the client see a truncated response.
|
|
playback.LogSubtitleStreamError(r.Context(), err, file.ID, embeddedIndex)
|
|
}
|
|
}
|
|
|
|
// subtitleSeekPosition picks the best-known starting position for a
|
|
// subtitle extract. A caller-supplied ?position= query wins (the player
|
|
// has the most accurate clock), falling back to the session's last
|
|
// reported position, then to 0.
|
|
func subtitleSeekPosition(r *http.Request, session *playback.Session) float64 {
|
|
if raw := r.URL.Query().Get("position"); raw != "" {
|
|
if v, err := strconv.ParseFloat(raw, 64); err == nil && v >= 0 {
|
|
return v
|
|
}
|
|
}
|
|
if session != nil && session.Position > 0 {
|
|
return session.Position
|
|
}
|
|
return 0
|
|
}
|
|
|
|
// subtitleWindowDuration picks the bounded extract length. The client
|
|
// overrides via ?duration=; absent that we use a 10-minute window,
|
|
// which is long enough that a single fetch covers many minutes of
|
|
// uninterrupted playback but short enough that the ffmpeg process
|
|
// finishes (and frees its input handle) well before the next window
|
|
// is requested.
|
|
func subtitleWindowDuration(r *http.Request) float64 {
|
|
const defaultDuration = 600.0
|
|
const maxDuration = 3600.0
|
|
if raw := r.URL.Query().Get("duration"); raw != "" {
|
|
if v, err := strconv.ParseFloat(raw, 64); err == nil && v > 0 && v <= maxDuration {
|
|
return v
|
|
}
|
|
}
|
|
return defaultDuration
|
|
}
|