Files
silo-server/internal/nodepool/transcode_pool.go
T
7958f0bbf0 feat(nodepool): node groups, per-node caps, and local transcode fallback control (#126)
* feat(nodepool): node groups, per-node caps, and local transcode fallback control

Group co-located transcode and proxy nodes so transcoded streams are
served by a proxy on the same host/LAN instead of bouncing across the
internal network (fixes #93):

- New nodepool.Planner is the single selection entry point: it picks the
  transcode node and its group's proxy together (round-robin within the
  group), replacing the independent ProxyPool.Pick/TranscodePool.Acquire
  calls scattered across the native and jellycompat handlers, and absorbs
  the duplicated soft-affinity pick logic.
- A group is only eligible while all of its enabled members are healthy;
  ungrouped nodes keep the historical behavior.
- New per-node max_jobs cap (transcodes for transcode nodes, streams for
  proxies; NULL = unlimited), enforced via health-reported job counts
  plus short-lived reservations that expire once fresher health data
  arrives. Proxy health now reports real stream counts, including HLS
  sessions via idle-expiry tracking.
- New playback.local_transcode_fallback setting (default on) lets admins
  refuse API-server transcoding when no eligible node exists.
- Health checks now publish updated node copies under the pool lock
  instead of mutating shared structs in place, fixing a data race.
- Admin UI: group + cap fields on the node form, group/cap columns, and
  the new fallback toggle in playback settings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(nodepool): proxy bandwidth measurement and egress caps

Proxy nodes now measure their stream egress (rolling 60s average over
everything under /stream) and report it via the health endpoint. A new
per-proxy max_bandwidth_kbps cap lets the planner route new streams away
from saturated proxies:

- Admission combines the measured egress with the estimated bitrate of
  the new stream (transcode target bitrate, or source bitrate for direct
  play/remux) so a stream is only admitted where it fits.
- Recently admitted streams are bridged as bandwidth reservations for the
  meter window, since the rolling average only converges on a new
  stream's rate gradually.
- A group whose proxies lack bandwidth headroom is treated as full: its
  transcode nodes are skipped, same as the job cap.
- Admin UI: per-proxy "Max Egress Bandwidth (Mbps)" field and a live
  egress column; manual health checks return the measured rate.

Active streams are never interrupted - the cap only gates new admissions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* perf(playback): trim node-mode time-to-stream-start

Distributed playback paid several avoidable costs before the first frame
that integrated mode doesn't have. This trims the safe ones:

- Web player preconnects to the stream origin (the proxy node) as soon as
  /playback/start returns, overlapping DNS/TCP/TLS handshakes with the
  transcode dispatch instead of paying them at the first manifest fetch.
- The transcode node no longer blocks its 202 on monitoring work: the
  Redis session-track write moves off the request path, and a replaced
  session's segment directory is renamed aside and deleted in the
  background instead of synchronously (RemoveAll of a long session can
  take seconds on slow disks during quality switches).
- The proxy's node-facing HTTP client gets a tuned transport: a larger
  idle-connection pool (Go's default of 2 per host causes connection
  churn and TLS re-handshakes when many viewers stream through one
  proxy->node pair) and a response-header timeout so a hung transcode
  node can no longer hang client requests indefinitely.
- jellycompat's remote transcode dispatch gains the same 10s timeout the
  native path has had; an unreachable node previously hung the compat
  manifest request until the OS gave up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 17:18:18 -04:00

90 lines
2.1 KiB
Go

package nodepool
import (
"sync"
"time"
)
// TranscodePool manages transcode nodes with least-connections selection.
// Thread-safe for concurrent use.
type TranscodePool struct {
nodes []*Node
mu sync.RWMutex
}
// NewTranscodePool creates an empty transcode pool.
func NewTranscodePool() *TranscodePool {
return &TranscodePool{}
}
// SetNodes replaces the node list.
func (p *TranscodePool) SetNodes(nodes []*Node) {
p.mu.Lock()
defer p.mu.Unlock()
p.nodes = nodes
}
// Acquire returns the healthy node with the fewest active jobs.
// Returns nil if no healthy nodes are available.
func (p *TranscodePool) Acquire() *Node {
p.mu.RLock()
defer p.mu.RUnlock()
var best *Node
for _, n := range p.nodes {
if !n.Healthy || !n.Enabled {
continue
}
if best == nil || n.ActiveJobs < best.ActiveJobs {
best = n
}
}
return best
}
// FindByURL returns the node with the given URL, or nil if not found.
// Used for soft-affinity during quality switches.
func (p *TranscodePool) FindByURL(url string) *Node {
p.mu.RLock()
defer p.mu.RUnlock()
for _, n := range p.nodes {
if n.URL == url {
return n
}
}
return nil
}
// Nodes returns a copy of the current node list.
func (p *TranscodePool) Nodes() []*Node {
p.mu.RLock()
defer p.mu.RUnlock()
cp := make([]*Node, len(p.nodes))
copy(cp, p.nodes)
return cp
}
// ApplyHealth records a health check result by swapping the node for an
// updated copy, keeping published *Node values immutable.
func (p *TranscodePool) ApplyHealth(id int, healthy bool, activeJobs, egressKbps int, checkedAt time.Time) {
p.mu.Lock()
defer p.mu.Unlock()
applyNodeHealth(p.nodes, id, healthy, activeJobs, egressKbps, checkedAt)
}
// applyNodeHealth replaces the slice entry for id with an updated copy.
func applyNodeHealth(nodes []*Node, id int, healthy bool, activeJobs, egressKbps int, checkedAt time.Time) {
for i, n := range nodes {
if n.ID != id {
continue
}
clone := *n
clone.Healthy = healthy
clone.ActiveJobs = activeJobs
clone.EgressKbps = egressKbps
clone.LastHealthCheck = &checkedAt
nodes[i] = &clone
return
}
}