Files
XC_VM/docs/en/development/streaming-diagnostics.md
T
Divarion_D 175198a902 refactor(stream-check): merge checker and grapher into one stream_check.py
Consolidate the two stream-check tools into a single master script with
three subcommands:

- check <url>          verify one stream (or --live dashboard)
- playlist <path|url>  batch an .m3u list -> aggregate JSON (+ per-stream files)
- graph <inputs...>    render the JSON as SVG charts

Shared helpers (HTTP, slugify, m3u/JSON handling) are now defined once.
The old --playlist flag becomes the `playlist` subcommand and the bare-URL
form becomes `check <url>`; the streamtest harness is updated accordingly.

Also fix the per-stream SVG rendering black in viewers that do not support
8-digit #rrggbbaa hex: the bitrate area fill and not-PLAYING bands now use
6-digit hex plus a separate fill-opacity attribute.

Docs (English + tools READMEs + STREAMTEST) updated to the new invocation.
Removes stream_queue_check.py and stream_graph.py.
2026-09-06 11:11:10 +03:00

4.0 KiB
Raw Blame History

Streaming Diagnostics & Tooling

A standalone tool verifies that a stream delivers correctly — that segments arrive in order and the delivery queue does not break. It is independent of the request path; for that, see the Streaming Subsystem.


tools/stream-check/stream_check.py (Python, stdlib only)

A single dependency-free tool that both verifies stream delivery and renders the result as SVG. Auto-detects HLS vs MPEG-TS. Three subcommands:

Subcommand Purpose
check <url> verify one stream (or watch it live with --live)
playlist <path|url> test every stream in an .m3u channel list → aggregate JSON (+ per-stream files)
graph <inputs…> render the JSON from check/playlist as static SVG charts
python3 tools/stream-check/stream_check.py check "<url>" --duration 30        # batch check
python3 tools/stream-check/stream_check.py check "<url>" --json               # cron / monitoring
python3 tools/stream-check/stream_check.py check "<url>" --live --duration 0  # live dashboard
python3 tools/stream-check/stream_check.py playlist list.m3u --out-dir logs/  # batch a whole playlist
python3 tools/stream-check/stream_check.py graph logs/ --combined            # JSON → SVG charts

What "queue intact" means per stream type:

Stream Queue check
HLS (.m3u8) EXT-X-MEDIA-SEQUENCE monotonic and contiguous (no dropped or rewound segments), no EXT-X-DISCONTINUITY, every newly appearing segment downloadable. Master playlists are resolved to their first variant.
MPEG-TS (.ts, /play/<token>/ts) per-PID continuity_counter (lost / duplicated / reordered packets = queue break), sync-byte loss, transport-error indicator, and delivery stalls.

Key check options:

Flag Purpose
--duration N seconds to observe (0 = until Ctrl-C in --live)
--tolerance N allow N transient queue breaks before reporting BROKEN (ignores rare source glitches relayed by -c copy)
--stall-timeout S delivery gap counted as a stall; keep it above the segment duration (default 15)
--live colored TUI dashboard (below)
--prebuffer S / --buffer-target S live: virtual-player prebuffer and buffer-graph scale
--json / --no-color machine output / disable ANSI

Exit code: 0 healthy, 2 queue problem or stall, 1 usage.

Live dashboard (--live)

Models a virtual player: the playhead advances at wall-clock rate while content is "received". For TS the received timeline comes from PCR (the stream clock); for HLS from the segments' EXTINF durations. Buffered playtime ("cache") = received − played; if it reaches zero the playhead freezes (a rebuffer event).

  STREAM QUEUE / BUFFER MONITOR   TS   up 00:22
  cache buffer (s), last 60s:
  ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▄▄▇▇▇▆▆▆▅▅▅▄▄▇▇▇▆▆▆▅   <- burst-then-drain = delivery sawtooth
  IN CACHE : [█████████████████░░░░░░░░░░░░░]  11.6s / 20s
  PLAYING  : PLAYING     head 00:18   received 00:29
  rate 1000 kbit/s   received 4.1 MB   last data 7.0s ago
  QUEUE OK   cc:0 sync:0 gaps:0 disc:0   rebuffers:0

The buffer graph and gauge are colored green (healthy) / yellow (low) / red (starving). For HLS a row of blocks shows the segments still in cache ahead of the playhead.

Note — delivery pacing. Live client delivery is now handled by the xc_fanout daemon (see Streaming Subsystem → Daemon delivery), which pulls each source once and fans it out over a unix socket. stream_check.py check --live visualises the buffer behaviour a real player would see against the delivered stream.


File Purpose
tools/stream-check/stream_check.py queue-integrity checker (check), playlist batch (playlist), live buffer dashboard (check --live), and SVG grapher (graph)