Files
silo-server/internal/playback/prepare_file_test.go
567cdd1a15 feat(playback): balance transcode sessions across multiple GPUs (#425)
* feat(playback): balance transcode sessions across multiple GPUs

playback.hw_device now accepts a comma-separated render-device list (e.g.
"/dev/dri/renderD128,/dev/dri/renderD129"). Each transcode session resolves
the list to one concrete device at spawn — the present device with the
fewest active GPU sessions, ties keeping list order — and holds that device
for its whole lifetime (seek/audio restarts reuse it); the reservation
releases on session shutdown, idempotently, including early spawn-failure
paths. Software-accel sessions never reserve, so they cannot skew the
balance.

A single configured value keeps the historical pass-through contract and an
empty value still auto-detects, so existing deployments are unaffected.
PickRenderDevice is list-aware too, picking least-loaded without reserving,
which lets the non-session consumers (chapter thumbnails, download
artifacts, transcode nodes) spread load best-effort when given a list.

Motivation: hosts with two identical media GPUs (e.g. dual Arc A310)
previously pinned every session to one device while the second sat idle.

* feat(admin): GPU device picker for playback hw_device

The hw-accel detection endpoint now reports render_device_details — each
render device with a human label derived from its sysfs PCI vendor/device
ids ("Intel GPU (0x56a6)") — and the Playback settings page renders them as
per-device toggles instead of requiring a hand-typed device path. No
selection means auto (first available device); one selection pins every
session; multiple selections balance least-loaded. The stored
playback.hw_device value stays the comma-separated list, written in stable
detection order regardless of click order, and a configured-but-undetected
device stays visible so a temporarily missing GPU is not silently dropped
on save.

* fix(playback): make GPU selection and reservation atomic

Review follow-up: resolveSessionHWDevice previously selected the
least-loaded device and incremented its count in two separate critical
sections, so concurrent session starts could all pick the same device
before any reservation landed. Device presence checks now happen outside
the lock and selection + reservation share one critical section; a
concurrency test asserts an exact split across two devices for eight
simultaneous starts, which the two-step version cannot guarantee.

* refactor(playback): one typed GPU acquisition boundary, release on process exit

Replace the CSV-handling spread across resolveSessionHWDevice and
PickRenderDevice with HWDeviceSet + AcquireHWDevice in hwdevice.go: every
GPU workload resolves exactly one device immediately before spawn.
Balancing is explicitly QSV/VAAPI-only — NVENC addresses GPUs by CUDA
index/UUID, so a multi-entry list warns and uses the first entry instead
of collapsing through the path-presence filter. Sessions now release
their reservation only after ffmpeg has been reaped (shutdown waits on
done first), closing the window where a new start could pick a device
the old process still occupied. Render-device sysfs descriptions move to
gpudetect.go so the allocator file owns only selection/reservation.

* fix(downloads): prepared downloads acquire a GPU through the shared pool

PrepareFile resolves the configured hw_device list to one concrete
device via AcquireHWDevice and holds the reservation until ffmpeg exits
(Run is synchronous, so the deferred release is the process-exit
boundary). Download encodes now participate in the same active-load
accounting as streaming sessions instead of best-effort spreading.

* fix(chapterthumbs): resolve hw_device list per extraction via the shared pool

ExtractFrame acquires one concrete device from AcquireHWDevice for the
hardware attempt (released when the attempt finishes) instead of passing
the raw comma-separated value to ffmpeg as a single device path. The
service stops pre-resolving and caching a device at first use — the raw
configured value flows through and each extraction resolves it.

* fix(transcodenode): fresh starts use this node's configured hw_device

/transcode/start constructed TranscodeOpts with an empty HWDevice, so
fresh sessions auto-detected the first GPU and bypassed the configured
list while reconstructed sessions honored it. Both paths now feed the
node-local config value into StartTranscode's shared resolution.

* feat(admin): node-aware GPU inventory on /admin/system/hw-accel

playback.hw_device is one cluster-wide value consumed by every transcode
node, but the endpoint probed only whichever healthy node had the fewest
jobs — an admin could configure devices that don't exist on the other
nodes. The endpoint now probes every healthy node concurrently and
returns a nodes array (URL, name, resolved accel, devices, or probe
error) alongside the backward-compatible flat fields, and the config doc
states the homogeneous-path contract explicitly.

* feat(admin): GPU picker survives empty detection, warns on node divergence

The picker rows are now the union of detected devices and configured
entries, so configured-but-missing devices stay visible (and
deselectable) when detection returns nothing or an older node omits
render_device_details (plain render_devices paths fall back to a generic
label). Per-node inventories from the hw-accel endpoint drive two
warnings: a banner when responding nodes report different device sets,
and a per-row note listing nodes missing that device. The multi-select
is hidden for NVENC — balancing is QSV/VA-API only — with a notice when
a multi-device value is already stored.

* style: gofmt touched files

* fix(playback): release GPU reservations on process exit

---------

Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-08-04 11:33:15 -04:00

130 lines
4.5 KiB
Go

package playback
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestBuildPrepareFileArgsEmitsFaststartMP4(t *testing.T) {
cases := []struct {
name string
video string
audio string
}{
{"remux", "copy", "copy"},
{"transcode", "h264", "aac"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
args := buildPrepareFileArgs(TranscodeOpts{
InputPath: "/media/in.mkv",
SourceVideoCodec: "h264",
TargetCodecVideo: tc.video,
TargetCodecAudio: tc.audio,
HWAccel: "none",
AudioTrackIndex: -1,
}, "/artifacts/out.mp4")
joined := strings.Join(args, " ")
if !strings.Contains(joined, "-movflags +faststart") {
t.Fatalf("%s args missing -movflags +faststart: %s", tc.name, joined)
}
if !strings.Contains(joined, "-f mp4") {
t.Fatalf("%s args missing -f mp4: %s", tc.name, joined)
}
if strings.Contains(joined, "-f hls") || strings.Contains(joined, "hls_segment") {
t.Fatalf("%s args must not emit HLS: %s", tc.name, joined)
}
if args[len(args)-1] != "/artifacts/out.mp4" {
t.Fatalf("%s output path must be last arg: %s", tc.name, joined)
}
})
}
// Remux copies the video stream rather than re-encoding.
remux := strings.Join(buildPrepareFileArgs(TranscodeOpts{
InputPath: "/m.mkv", TargetCodecVideo: "copy", TargetCodecAudio: "copy", HWAccel: "none", AudioTrackIndex: -1,
}, "/o.mp4"), " ")
if !strings.Contains(remux, "-c:v copy") {
t.Fatalf("remux must copy video: %s", remux)
}
}
func TestResolvePrepareTarget(t *testing.T) {
settings := AdminSettings{TranscodeEnabled: true, Allow4KTranscode: true}
file := &models.MediaFile{CodecVideo: "h264", CodecAudio: "dts", Container: "mkv", Resolution: "1080p"}
// remux with an undecodable audio codec → copy video, transcode audio to AAC.
caps := ClientCapabilities{CodecsVideo: []string{"h264"}, CodecsAudio: []string{"aac"}, Containers: []string{"mp4"}, MaxResolution: "2160p"}
rt := ResolvePrepareTarget(file, "remux", caps, settings)
if rt.Container != "mp4" || rt.CodecVideo != "copy" || rt.CodecAudio != "aac" {
t.Fatalf("remux target = %+v, want copy video / aac audio / mp4", rt)
}
// remux with a decodable audio codec → copy both streams.
capsAudioOK := ClientCapabilities{CodecsVideo: []string{"h264"}, CodecsAudio: []string{"aac", "dts"}, Containers: []string{"mp4"}, MaxResolution: "2160p"}
rt = ResolvePrepareTarget(file, "remux", capsAudioOK, settings)
if rt.CodecAudio != "copy" {
t.Fatalf("remux audio = %q, want copy", rt.CodecAudio)
}
// transcode → H.264/AAC, downscaled to the client max when the source exceeds it.
rt = ResolvePrepareTarget(file, "transcode", ClientCapabilities{MaxResolution: "720p"}, settings)
if rt.CodecVideo != "h264" || rt.CodecAudio != "aac" || rt.Resolution != "720p" {
t.Fatalf("transcode target = %+v, want h264/aac/720p", rt)
}
// transcode where the source already fits → keep source resolution (no scale).
rt = ResolvePrepareTarget(file, "transcode", ClientCapabilities{MaxResolution: "1080p"}, settings)
if rt.Resolution != "" {
t.Fatalf("transcode resolution = %q, want empty (source)", rt.Resolution)
}
}
func TestPrepareFileResolvesOneDeviceAndReleasesAfterExit(t *testing.T) {
resetDeviceLoad(t)
devA, devB := "/dev/dri/renderD888", "/dev/dri/renderD889"
fakeDeviceStat(t, devA, devB)
// Fake ffmpeg: record argv, create the output (last arg) so finalize works.
dir := t.TempDir()
argsFile := filepath.Join(dir, "args.txt")
script := filepath.Join(dir, "ffmpeg")
if err := os.WriteFile(script, []byte("#!/bin/sh\nprintf '%s\\n' \"$@\" > "+argsFile+"\neval \"touch \\${$#}\"\n"), 0o755); err != nil {
t.Fatal(err)
}
outputPath := filepath.Join(dir, "artifact.mp4")
err := PrepareFile(context.Background(), TranscodeOpts{
InputPath: "/nonexistent/input.mkv",
TargetCodecVideo: "h264",
TargetCodecAudio: "aac",
FFmpegPath: script,
HWAccel: "vaapi",
HWDevice: devA + "," + devB,
}, outputPath)
if err != nil {
t.Fatalf("PrepareFile: %v", err)
}
argv, err := os.ReadFile(argsFile)
if err != nil {
t.Fatal(err)
}
got := string(argv)
if !strings.Contains(got, devA) {
t.Fatalf("ffmpeg args missing resolved device %s:\n%s", devA, got)
}
if strings.Contains(got, devA+","+devB) {
t.Fatalf("ffmpeg args contain the raw device list:\n%s", got)
}
if count := hwDeviceActiveCount(devA); count != 0 {
t.Fatalf("active count after PrepareFile returned = %d, want 0", count)
}
}