LivenessService::tick() runs every second in MAIN's signals daemon
(cron:cluster is the minute fallback) and judges the active nodes whose
TELEMETRY flow is on by their silence: ok, suspect after 10 s, offline
after cluster_offline_after_sec (30 s). The result, in
tmp/cluster/health.json, is read through Core\Cluster\ClusterHealth:
ServerRepository::getAll() takes server_online from it for those nodes
(instead of the 90 s last_check_ago rule) and getCapacity() doubles a
suspect node's weight. Each transition rewrites the servers cache and is
audited.
When over half of the judged nodes (at least two) fall silent together,
the fleet silence guard holds every node at its last state instead of
marking any offline, audits it and alerts on the Cluster Nodes page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DcNdxHieZuUh3EnMUKt7zf
Review fixes for the php_pids change.
- cron:users judged a remote PHP-served connection by date_start, but a
reused connection (VOD range/seek, TS or timeshift reconnect) gets a new
worker pid and keeps its original date_start. A pid assigned after the
node's last heartbeat was then looked up in a list that could not hold
it, and the live viewer was killed. The pid write also stamps
hls_last_read (MAIN's clock), so the later of the two now dates the pid.
The check moves into isRemoteWorkerRunning().
- phpFpmWorkerPIDs() returns null when no FPM master is visible, so an
idle ondemand pool ("[]") now reaps dead workers' rows while an unknown
list still keeps them. A server missing from either map no longer hits
count(null).
- getAllSimple/getStreamingSimple/getProxySimple and the admin API
get_server leave out php_pids; getById keeps the full row for REPLACE.
- Tests pin the watchdog and reaper wiring, exercise sort(), and reset
the ServerRepository db seam; array_unique() is dropped (unreachable).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TQHeypQpfAXTdCysmi7b6a
A connection row records the PHP-FPM worker serving it (getmypid() in
live.php, vod.php and timeshift.php), but the watchdog published as
servers.php_pids the contents of bin/php/sockets/*.pid: the FPM master
pids. In Redis mode MAIN's cron:users reaps every node's connections
and, for a remote PHP-served connection older than the node's last
heartbeat, checks the worker pid against that list. It never matched,
so the connection was closed and the LB told to kill its worker: every
VOD, timeshift and PHP-served live viewer on an LB was dropped about a
minute after connecting. MySQL mode was unaffected (each node probes its
own pids locally).
- ProcessManager::phpFpmWorkerPIDs() lists the processes titled
"php-fpm: pool xc_vm" (sorted, unique); findProcessPIDs() takes an
optional procfs root so it can run against a fixture tree.
- WatchdogCommand publishes that list instead of the pid files.
- ServerRepository::getAll() drops php_pids from the rows it returns and
caches, so the per-request servers cache does not carry every node's
worker list. CacheCronJob's unset($rServers['php_pids']) meant to do
that but hit a top-level key of an id-keyed map; it is removed.
- UsersCronJob reads php_pids with its own SELECT `id`, `php_pids`.
The `- 1` heartbeat margin is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TQHeypQpfAXTdCysmi7b6a
Claude