Files
XC_VM/docs/en/development/streaming-diagnostics.md
T
Divarion_D 175198a902 refactor(stream-check): merge checker and grapher into one stream_check.py
Consolidate the two stream-check tools into a single master script with
three subcommands:

- check <url>          verify one stream (or --live dashboard)
- playlist <path|url>  batch an .m3u list -> aggregate JSON (+ per-stream files)
- graph <inputs...>    render the JSON as SVG charts

Shared helpers (HTTP, slugify, m3u/JSON handling) are now defined once.
The old --playlist flag becomes the `playlist` subcommand and the bare-URL
form becomes `check <url>`; the streamtest harness is updated accordingly.

Also fix the per-stream SVG rendering black in viewers that do not support
8-digit #rrggbbaa hex: the bitrate area fill and not-PLAYING bands now use
6-digit hex plus a separate fill-opacity attribute.

Docs (English + tools READMEs + STREAMTEST) updated to the new invocation.
Removes stream_queue_check.py and stream_graph.py.
2026-09-06 11:11:10 +03:00

71 lines
4.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Streaming Diagnostics & Tooling
A standalone tool verifies that a stream delivers correctly — that segments arrive in order and the delivery queue does not break. It is independent of the request path; for that, see the [Streaming Subsystem](streaming-subsystem.md).
---
## `tools/stream-check/stream_check.py` (Python, stdlib only)
A single dependency-free tool that both **verifies** stream delivery and **renders** the result as SVG. Auto-detects HLS vs MPEG-TS. Three subcommands:
| Subcommand | Purpose |
| --- | --- |
| `check <url>` | verify one stream (or watch it live with `--live`) |
| `playlist <path\|url>` | test every stream in an `.m3u` channel list → aggregate JSON (+ per-stream files) |
| `graph <inputs…>` | render the JSON from `check`/`playlist` as static SVG charts |
```bash
python3 tools/stream-check/stream_check.py check "<url>" --duration 30 # batch check
python3 tools/stream-check/stream_check.py check "<url>" --json # cron / monitoring
python3 tools/stream-check/stream_check.py check "<url>" --live --duration 0 # live dashboard
python3 tools/stream-check/stream_check.py playlist list.m3u --out-dir logs/ # batch a whole playlist
python3 tools/stream-check/stream_check.py graph logs/ --combined # JSON → SVG charts
```
What "queue intact" means per stream type:
| Stream | Queue check |
| --- | --- |
| HLS (`.m3u8`) | `EXT-X-MEDIA-SEQUENCE` monotonic and contiguous (no dropped or rewound segments), no `EXT-X-DISCONTINUITY`, every newly appearing segment downloadable. Master playlists are resolved to their first variant. |
| MPEG-TS (`.ts`, `/play/<token>/ts`) | per-PID `continuity_counter` (lost / duplicated / reordered packets = queue break), sync-byte loss, transport-error indicator, and delivery stalls. |
Key `check` options:
| Flag | Purpose |
| --- | --- |
| `--duration N` | seconds to observe (`0` = until Ctrl-C in `--live`) |
| `--tolerance N` | allow N transient queue breaks before reporting `BROKEN` (ignores rare source glitches relayed by `-c copy`) |
| `--stall-timeout S` | delivery gap counted as a stall; keep it above the segment duration (default 15) |
| `--live` | colored TUI dashboard (below) |
| `--prebuffer S` / `--buffer-target S` | live: virtual-player prebuffer and buffer-graph scale |
| `--json` / `--no-color` | machine output / disable ANSI |
Exit code: `0` healthy, `2` queue problem or stall, `1` usage.
### Live dashboard (`--live`)
Models a virtual player: the playhead advances at wall-clock rate while content is "received". For **TS** the received timeline comes from **PCR** (the stream clock); for **HLS** from the segments' `EXTINF` durations. Buffered playtime ("cache") = received − played; if it reaches zero the playhead freezes (a rebuffer event).
```text
STREAM QUEUE / BUFFER MONITOR TS up 00:22
cache buffer (s), last 60s:
▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▄▄▇▇▇▆▆▆▅▅▅▄▄▇▇▇▆▆▆▅ <- burst-then-drain = delivery sawtooth
IN CACHE : [█████████████████░░░░░░░░░░░░░] 11.6s / 20s
PLAYING : PLAYING head 00:18 received 00:29
rate 1000 kbit/s received 4.1 MB last data 7.0s ago
QUEUE OK cc:0 sync:0 gaps:0 disc:0 rebuffers:0
```
The buffer graph and gauge are colored green (healthy) / yellow (low) / red (starving). For HLS a row of blocks shows the segments still in cache ahead of the playhead.
> **Note — delivery pacing.** Live client delivery is now handled by the
> `xc_fanout` daemon (see [Streaming Subsystem → Daemon delivery](streaming-subsystem.md#daemon-delivery-xc_fanout)), which pulls each source once and fans it out over a unix socket. `stream_check.py check --live` visualises the buffer behaviour a real player would see against the delivered stream.
---
## Related files
| File | Purpose |
| --- | --- |
| `tools/stream-check/stream_check.py` | queue-integrity checker (`check`), playlist batch (`playlist`), live buffer dashboard (`check --live`), and SVG grapher (`graph`) |