The cs-fix auto-typing narrowed the pid parameters of the /proc-based
process checks to a non-nullable `int` (read from the `@param int $pid`
docblocks). But every one of these methods was written to tolerate a
missing pid: each opens with `$pid = (int) $pid;` and a `$pid <= 0`
guard, and their callers routinely pass nullable DB columns
(`tv_archive_pid`, `monitor_pid`, `vframes_pid`, `streams.pid`, …) and
runtime PIDs that can be null. In production this crashed the monitor
cron:
ProcessManager::isNamedProcessRunning(): Argument #1 ($pid) must be of
type int, null given, called in Cli/Commands/MonitorCommand.php:322
Make the pid parameter nullable (`?int`) on the eight affected public
checks — isRunning, isNamedProcessRunning, isStreamRunning,
producerKind, resourceSample, kill, getProcessAge, isStreamAlive,
isMonitorAlive — so the existing `(int) null === 0` cast and guard
return the safe "not running" result instead of a fatal TypeError. The
docblocks are corrected to `int|null` as well so a future cs-fix run
does not re-narrow the types. procExists stays `int` (internal, only
reached after the guard). No behaviour change for valid pids.
Mechanical, behaviour-preserving reformat produced by 'make cs-fix' under
the new build/phpcs.xml.dist ruleset: K&R braces, tab indentation, and the
other whitespace normalisations. No logic changes.
CPU is a difference between two /proc readings, and the previous one was kept
in the stream's progress_info row — so a figure depended on that value
surviving a round trip through a row other code also rewrites, and the first
pass of every producer showed a dash for a minute.
The previous reading now lives beside the stream's files, in
<streams>/<id>_.usage (tmpfs, removed with the rest of <id>_* when the stream
stops): node-local bookkeeping, like the pid file. Where there is no usable
previous reading — a producer's first pass, or a new pid after a restart — the
lifetime average stands in, as ps reports it, computed from the process's own
start time in /proc/PID/stat against /proc/uptime rather than /proc/PID's
mtime, which is only set when something first looks at the directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WwKmPG4RPK4cnJRAxQhPdL
A new "Resources" column between Stream Info and Actions: which process is
producing the channel (the fanout daemon's native remuxer, ffmpeg, or PHP for
the LLOD segmenter / loopback relay), the CPU it is burning and the memory it
holds. With the native remuxer now an option per stream, "what does this
channel actually cost" and "which backend is it on" are the two questions the
list could not answer.
ProcessManager reads both from /proc/PID/stat (fields 14/15 and 24, with the
page size derived rather than assumed — 64K pages are normal on arm64). CPU
there is cumulative, so a percentage needs two readings: cron:streams samples
each producer once a pass and folds cpu/mem/producer into the stream's
progress_info, carrying the previous reading in the same JSON to subtract from.
cpuPercent() returns null rather than a wild figure when the pair says nothing
— no previous sample, same instant, or a counter that went backwards because
the producer restarted. Only the node running a stream can read its own /proc,
so the sampling happens there and reaches the panel in the row the cron already
writes; MAIN just renders it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZP7fc2Hz36wbPmuLd9F9o
With fanout_supervise on, StreamProcess::startMonitor() no longer spawns a
PHP watchdog. It builds the stream's commands and hands them to the
daemon's supervisor (PUT /monitor/<id>), which starts, watches and
restarts them: failover, priority backup, forced source, stalled output,
audio loss, frame-rate drop and scheduled restart. PHP still builds every
command and makes every database write.
A copy-only live stream's command is the daemon's native remuxer,
`xc_fanout remux`, composed by the new buildNativeLive() exactly as
buildLive() composes an ffmpeg line. It reads the source natively and
writes the same on-disk HLS and daemon feed as the ffmpeg -f tee output,
with no ffmpeg process. fanout_source_backend decides: auto = remuxer with
the ffmpeg command as fallback_cmd (taken when the remuxer exits 3,
"cannot serve this source": fMP4 or encrypted HLS, rtmp, no keyframes),
native = remuxer only, ffmpeg = ffmpeg only. Eligibility is explicit
(isNativeEligible / isNativeSource): no transcode, custom ffmpeg, custom
map, RTMP output, external push, timestamp repair, read-native or forced
input codec; http(s)/udp/rtp sources only.
The rest of the panel learns who owns the producer:
- superviseStream() asks the daemon first and touches nothing unless it
is accepting; without a restart it adopts a running encoder, so
cron:streams moves PHP-monitored streams over with no blip. A producer
the daemon cannot adopt (PHP LLOD, PHP loopback) is replaced, never left
beside the new one. The row is marked watched before the hand-over, so
the reconcile cannot release a stream mid-start.
- reconcileSupervised() copies the daemon's state (status, pid, source,
codecs, resolution, measured bitrate) into streams_servers: every
cron:streams pass and every 5 s from the signals daemon. Supervised
streams whose row is gone or stopped are released.
- stopStream() and the on-demand reaper release before killing anything;
killing the producer first is what the supervisor restarts.
- isWatched() replaces bare isMonitorAlive() checks in live.php,
admin/live.php, rtmp.php and cron:streams: a supervised stream's
monitor_pid is the daemon's. MonitorCommand stands down for supervised
streams; startMonitor() releases one before falling back to PHP, so
turning supervision off does bring streams back on their next restart.
- force_stream goes through the daemon for a supervised stream (the .force
file is only read by the PHP monitor).
- ProcessManager::isStreamRunning() recognises the remuxer, so the
archive, thumbnail and delay workers follow it like ffmpeg.
- A supervised loopback child tees into the daemon (the supervisor judges
a stream by the bytes it receives); legacy loopback is unchanged.
Delay streams, created channels and yt-dlp platform sources stay on the
PHP monitor. A daemon without /monitors/state (older than this) is never
handed a stream, so the panel is safe against an un-upgraded node.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012QL93N6dzGkmKoQgA4oh16
Namespace-migration find/replace (XC_VM -> \XC_VM) leaked a backslash into
many plain string literals: CLI echo/log text, the MaxMind/GitHub outbound
User-Agents, the LB tarball temp path, process-match greps, the sysctl and
cron markers, the proxy ini section, and docblocks.
Most are cosmetic or self-consistent, but two carried upgrade hazards on
already-deployed servers, handled with migration guards:
- sysctl marker: writer + first-line check moved to `# XC_VM` together
(one-time benign rewrite on upgrade).
- cron marker: switched to `# XC_VM` and added a purge to both root-crontab
installers that drops lines with either the old `# \XC_VM` or the new
`# XC_VM` marker before re-adding, so upgrades don't duplicate cron jobs.
The xc_vm user crontab already does a full `crontab -r` rebuild.
ProxyCommand was removed in E1, so no process is ever titled XC_VMProxy[<id>]
anymore — the process-name checks in ProcessManager::isMonitorAlive and
StreamProcess::stopStream that matched it can never be true. Remove the dead
references; the real titles (XC_VM[], LLOD[], Loopback[], the m3u8/ts patterns)
are untouched.
Not touched: the live.php non-proxy .ts chase-read (E2) and HLSGenerator
generateHLS / segment.php tmpfs-serving (E3) — those are the active
daemon-reachability fallback (daemon down, re-feed windows, and the
proxy/restreamer/llod2 types not yet canary-validated), so deleting them now
would remove the rollback. Gated on Phase D stability across all types.
Audited the streaming subsystem for symbols with zero real callers (ruled out
dynamic dispatch: command-name strings, routes, #[ListensTo], self::/$this->
internal calls, bare-name string dispatch). Removals, each grep-verified across
all of src/:
- Whole class TS (src/Streaming/TimeshiftClient.php) — superseded by the inline
TS byte-parsing in LLOD/Loopback; only ref left was a stale comment. Drops its
2 require + 3 use lines too.
- StreamUtils::getTSInfo — 0 callers.
- SegmentReader::getLLODSegments — replaced by the LLOD-v3 daemon feed; 0 callers.
- ProcessChecker::isPIDRunning + isPIDsRunning (dead pair) — 0 callers; drops the
now-orphaned CurlClient import.
- ProcessManager: checkPidFile + matchesCmdline (dead pair), killByPattern,
countProcesses, currentPid, releaseCronLock — all 0 callers (acquireCronLock
stays; locks self-release on exit).
- AsyncFileOperations: checkFilesExists, awaitFileExistsAdaptive, awaitFileModified,
getCacheStats, filterExistingFiles — never-wired public helpers, 0 refs.
php -l clean on all touched files; make gates green; no phpstan-baseline entries
reference the removed symbols.
Phase E (ADR 0003), step 1: proxy live streams are now daemon-only. The
daemon owns the proxy puller (registered + off-air-probed in live.php), so
the legacy producer + relay are dead code and are removed:
- delete Cli/Commands/ProxyCommand.php (the XC_VMProxy producer).
- remove StreamProcess::startProxy and ProcessManager::startProxy (only
caller was live.php).
- live.php: drop the startProxy branch in the process block and the
AF_UNIX socket relay in the proxy delivery arm. A proxy stream whose
daemon is unreachable ($rFanout false) now shows not-on-air — the
keepalive restarts the daemon in ~2s — instead of falling back to the
legacy producer.
This removes the proxy fallback: the daemon is required for proxy delivery
(its own crash resilience is the keepalive). Non-proxy legacy stays for now.
PHPStan clean, 443 tests.
Refs ADR 0003 (Phase E).
Fixes five runtime errors seen in production panel logs (v2.3.9):
- HomeController: player home page fatally crashed (count(): null given)
when content/tmdb_popular was absent or corrupt. Guard the read +
unserialize, default movies/series to arrays, null-safe the counts.
- ShutdownHandler: "array offset on null" on every live session close
when $rChannelInfo was null (vod/timeshift or early exit). Use !empty().
- ProcessManager::acquireCronLock: TOCTOU race — a competing cron removed
the lock file between file_exists() and file_get_contents()/filemtime(),
emitting "failed to open stream"/"stat failed". Read content+mtime once
under @, and take the lock if the file vanished.
- AuthRepository::updateCodes: "foreach() argument must be array, null"
when a code's whitelist JSON decoded to null. Cast to (array).
- admin/review.php: "Undefined array key extension" for URLs without an
extension. Coalesce pathinfo()['extension'] to ''.
cron:servers only checked whether a watchdog process existed. A watchdog
blocked in poll() on a half-open MariaDB socket (CLOSE_WAIT) inside its DB
ping still exists, so cron kept trusting it indefinitely: last_check_ago
went stale and the panel marked the node offline while nginx, php-fpm,
redis and active streams were all healthy. Restarting only the watchdog
restored the heartbeat immediately.
Treat an abnormally old watchdog as stale. A normal generation lives only
a few seconds, so any process alive for more than 90s is killed and a
fresh generation is started; a plain presence check alone would keep
trusting the wedged one forever.
Add ProcessManager::getProcessAge(), which derives age from the mtime of
the /proc/PID directory (time() - filemtime) — the same figure as
`ps -o etimes` without shelling out, consistent with the rest of the class
reading /proc directly and with no HZ/CLK_TCK assumption. Reaping uses the
existing ProcessManager::kill() rather than a raw `kill -9`.
Recovery restarts only the watchdog; nginx, php-fpm, redis, ffmpeg and
running streams are untouched.
Two false positives/negatives in process detection kept the panel's
self-healing loop dead on any real installation:
- isNginxRunning() looked for "nginx: master" only among xc_vm-owned
processes, but the master runs as root on typical installs (workers run
as xc_vm), so cron:servers/cron:streams bailed out with "XC_VM not
running..." before reaching the daemon revival block. Now scans
/proc/*/cmdline user-agnostically, same as RootSignalsCronJob.
- The per-daemon "is it alive" checks piped ps through grep by bare words:
every live-stream ffmpeg carries -thread_queue_size/-max_muxing_queue_size
in its command line, so the "queue" check matched any running stream and
the encode queue daemon was never started while at least one channel was
up — created channels sat at "0% DONE" until console.php queue was run by
hand. All seven checks (signals, cache_handler, network, watchdog, queue,
ondemand, scanner) now use ProcessManager::findProcessPIDs(), a /proc
cmdline scan matching the daemon title (XC_VM[...]) or its exact
console.php invocation. Kill branches use the same PID list — the old
bare "ondemand"/"scanner" greps could kill an innocent ffmpeg whose
source URL contained those words.
Divarion_D