fix(servers): harden the OPENSSL_EXTRA repair after review

server:sync-openssl-extra now refuses to send when the main's own
published fingerprint differs from the value it read. Run as root, it
could read a config/openssl_extra that xc_vm cannot read, while the
main's php-fpm runs on the default, and push a value that breaks every
LB. The command and cron:root_signals now share one contract:
OpensslExtra::SIGNAL_ACTION, signal() and applySignal(), with
publish() for cron:servers.

Encryption::readToken now tries the sealed format with both values
before the legacy one. A wrong-value legacy decrypt() could return
garbage (about 1 in 256), so a sealed token minted with the replaced
value was refused. Legacy tokens keep that inherent miss rate.

proxy_api trusts the posted server_id and handed any registered proxy
an LB's signal rows, including the main's OPENSSL_EXTRA. It now skips
set_openssl_extra rows until the source-IP binding lands.

Tests cover sync() with TestDb, the signal contract and the case's own
break. The docs cover the check and the repair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TQHeypQpfAXTdCysmi7b6a
This commit is contained in:
Claude
2026-09-25 00:56:05 +00:00
parent 33d1aa7188
commit 6c391f25d9
13 changed files with 406 additions and 47 deletions
+27 -1
View File
@@ -32,6 +32,7 @@ Probes the target node from the outside and reads its panel-side state:
| HTTP `GET /api` | Does PHP actually answer behind nginx |
| Clock offset | `time_offset` vs the panel (skew > 30 s can flap the node online/offline) |
| Signal queue | Unconsumed rows in `signals` for this node (backlog > 120 s = the node's callback loop is stuck) |
| OPENSSL_EXTRA | Whether the node holds the main's `OPENSSL_EXTRA` (see [below](#repairing-an-openssl_extra-mismatch)). `unknown (node not updated)` / `unknown (main not updated)`: that side runs a build that does not publish it yet. Not checked for proxies |
The probe combinations map to causes:
@@ -58,11 +59,33 @@ Run this **on the silent LB/proxy node itself** — the causes usually live ther
- is the system **cron service** active (no cron → the crontab never fires);
- is a previous `cron:servers` instance **hung on its cron lock** — a hung instance blocks every subsequent run for up to 30 minutes (`acquireCronLock` stale timeout), which is exactly how one DB blip keeps a node offline for half an hour. The command prints the holding PID and the kill command.
7. **Clock skew** — `time_offset` vs the panel.
8. **OPENSSL_EXTRA** — whether this node holds the main's value, as in Mode A.
> **Note:** the iptables check requires passwordless sudo (`sudo -n`). Without it the check reports `cannot check (need sudo iptables)` instead of failing — run the command as `root` for a full picture.
---
## Repairing an OPENSSL_EXTRA mismatch
`OPENSSL_EXTRA` (`config/openssl_extra`, or a built-in default when that file is absent) keys the stream tokens the main mints for the redirects an LB serves (`/auth`, `/vauth`, `/tsauth`, `/thauth`, subtitles). A main and its LBs must hold the same value; an LB that holds another one rejects all of those tokens, so playback redirected from the main fails there. The current installer gives a new main a random value, and LBs added by `server:install` before it shipped that file to them run on the default.
Each node's `cron:servers` publishes a fingerprint of its value (never the value itself) in `servers.server_hardware`, and the OPENSSL_EXTRA check compares the node's with the main's. To repair a mismatch, run on the **MAIN**:
```bash
sudo /home/xc_vm/console.php server:sync-openssl-extra <server_id>
sudo /home/xc_vm/console.php server:sync-openssl-extra --all
```
- The command queues one root signal with the main's value for each streaming LB that reports another fingerprint. The LB's `cron:root_signals` applies it within a minute: it writes `config/openssl_extra` (0600, owned by `xc_vm`) and php-fpm uses it from the next request, with no restart. The value stays in the `signals` table until then.
- For **10 minutes** afterwards the LB still accepts tokens it minted with its old value (kept in `config/openssl_extra.prev`), so links it has just handed out keep playing. Legacy-format tokens (`secure_stream_tokens` off) can still be refused in that window, about 1 in 256.
- LBs already in sync, LBs that publish no fingerprint (not updated yet) and offline LBs are skipped. `--force` queues them anyway; an offline LB applies the signal if it comes back within a day (queued signals expire after 24 hours). Proxies and the main are never sent the value.
- When the main has **no** `config/openssl_extra`, it runs on the built-in default and there is nothing to send. An LB that still reports a mismatch has a stale `config/openssl_extra` of its own: delete it on that LB (`sudo rm -f /home/xc_vm/config/openssl_extra`); the change applies at the next request.
- The command sends nothing when the main publishes another fingerprint than its `config/openssl_extra` holds: either `xc_vm` cannot read that file, so the main's php-fpm runs on the default (`sudo chown xc_vm:xc_vm /home/xc_vm/config/openssl_extra && sudo chmod 600 /home/xc_vm/config/openssl_extra`), or the file changed within the last minute (wait for `cron:servers`). It also sends nothing when the main has not published a fingerprint yet.
Check the result with `server:diagnose <server_id>` a minute or two later, after the LB's next `cron:servers` run.
---
## Output & Exit Codes
Each check prints one aligned `[OK]`/`[WARN]` line, followed by a numbered **Probable cause(s)** summary with the exact fix command where one exists (e.g. the `iptables -D INPUT ... -j DROP` unblock line).
@@ -109,6 +132,7 @@ Probable cause(s):
| **All nodes drop at the same moment** | A MySQL restart/blip on the main hit every node's watchdog at once (fatal for pre-fix builds; current builds wait it out) | Check the main's MySQL error log around the drop time; update the nodes so the watchdog survives outages |
| Node flaps online/offline | Clock skew > 30 s (or recurring MySQL blips) | Sync NTP on the node; check MySQL stability on the main |
| Status = 4 | Install/provision errored | Re-run `server:install` from the main |
| Playback redirected from the main fails on one LB | `OPENSSL_EXTRA` mismatch (the check reports it) | `sudo /home/xc_vm/console.php server:sync-openssl-extra <server_id>` on the main (see [above](#repairing-an-openssl_extra-mismatch)) |
---
@@ -119,7 +143,9 @@ Probable cause(s):
| `src/Cli/Commands/ServerDiagnoseCommand.php` | The diagnostic command (both modes) |
| `src/Cli/Commands/WatchdogCommand.php` | The watchdog daemon — writes the heartbeat (`last_check_ago`) |
| `src/Cli/CronJobs/ServersCronJob.php` | Babysitter cron — relaunches a dead watchdog |
| `src/Cli/CronJobs/RootSignalsCronJob.php` | Applies iptables blocks (the false-positive source) |
| `src/Cli/CronJobs/RootSignalsCronJob.php` | Applies iptables blocks (the false-positive source) and the `OPENSSL_EXTRA` repair signal |
| `src/Cli/Commands/ServerSyncOpensslExtraCommand.php` | `server:sync-openssl-extra` — queues the main's `OPENSSL_EXTRA` for mismatched LBs (MAIN only) |
| `src/Core/Config/OpensslExtra.php` | `OPENSSL_EXTRA` fingerprint, repair signal and the 10-minute fallback |
| `src/Domain/Server/ServerRepository.php` | `servers` table access |
See also: [CLI Tools](../guides/cli-tools.md), [Updating a Server](../administration/server-update.md).
+12 -1
View File
@@ -54,6 +54,7 @@ To see all available commands:
| `db:migrate` | `DbMigrateCommand` | Apply pending database migrations from the `migrations/` directory | xc_vm |
| `server:install` | `ServerInstallCommand` | Install/configure server (Proxy/LB) via SSH | root |
| `server:diagnose` | `ServerDiagnoseCommand` | Diagnose why a proxy/LB node is silent to the main (heartbeat, reachability, iptables, service) | root |
| `server:sync-openssl-extra` | `ServerSyncOpensslExtraCommand` | Send the main's `OPENSSL_EXTRA` to load balancers that report another one (MAIN only) | root/xc_vm |
> `console.php` registers **every** class it discovers in `Cli/Commands/` and `Cli/CronJobs/` (glob + reflection) — there is **no** `file_exists()` guard. A command is "optional" only in that it may be **stripped from the LB build** (`Makefile` `LB_FILES_TO_REMOVE`) or **provided by an installed module**. `plex_item` and `watch_item` above are **module-provided** (Plex/Watch) — their command classes are not in the committed core tree and exist only when that module is installed.
@@ -373,7 +374,17 @@ sudo /home/xc_vm/console.php server:diagnose <server_id>
sudo /home/xc_vm/console.php server:diagnose
```
Finds out **why** a proxy/LB node shows offline in the panel: checks the heartbeat, reachability (ICMP/TCP/HTTP `/api`), clock skew, the signal queue, and — locally on the node — whether the node firewalled the main's IP in its own iptables, whether the `xc_vm` service/nginx are up, whether the `watchdog` heartbeat daemon is running, and whether `cron:servers` is in the `xc_vm` crontab. Read-only; exit code `0` = no problems found, `2` = probable causes printed. See the [Server Diagnostics guide](../administration/server-diagnostics.md) for details.
Finds out **why** a proxy/LB node shows offline in the panel: checks the heartbeat, reachability (ICMP/TCP/HTTP `/api`), clock skew, the signal queue, whether an LB holds the main's `OPENSSL_EXTRA`, and — locally on the node — whether the node firewalled the main's IP in its own iptables, whether the `xc_vm` service/nginx are up, whether the `watchdog` heartbeat daemon is running, and whether `cron:servers` is in the `xc_vm` crontab. Read-only; exit code `0` = no problems found, `2` = probable causes printed. See the [Server Diagnostics guide](../administration/server-diagnostics.md) for details.
### OPENSSL_EXTRA Sync
```bash
# On the MAIN — one load balancer, or every LB that reports another value
sudo /home/xc_vm/console.php server:sync-openssl-extra <server_id>
sudo /home/xc_vm/console.php server:sync-openssl-extra --all [--force]
```
Moves load balancers onto the main's `OPENSSL_EXTRA` when `server:diagnose` reports a mismatch (playback redirected from the main fails on that LB). Each LB applies it within a minute and still accepts its old value for 10 minutes. See [Repairing an OPENSSL_EXTRA mismatch](../administration/server-diagnostics.md#repairing-an-openssl_extra-mismatch).
### SSL Certificate
@@ -25,8 +25,10 @@ use XcVm\Infrastructure\Database\DatabaseAware;
* opens tokens it minted with the value it replaced. --force also sends to LBs
* that report the same value, none (not updated yet) or are offline.
*
* It refuses to send a value the main's php-fpm does not mint with (see sync()).
* The value passes through the signals table until the LB's root cron deletes
* the row (about a minute); a legacy LB reads that database in full anyway.
* the row (about a minute); a legacy LB reads that database in full anyway, and
* proxy_api never hands these rows to a proxy.
*
* @package XC_VM_CLI_Commands
* @author Divarion_D <https://github.com/Divarion-D>
@@ -54,9 +56,27 @@ class ServerSyncOpensslExtraCommand implements CommandInterface {
echo "Usage: php console.php server:sync-openssl-extra <server_id>|--all [--force]\n";
return 1;
}
if (!defined('SERVER_ID')) {
echo "Run this on the MAIN server.\n";
return 1;
}
$rServers = ServerRepository::getAll(true);
if (!defined('SERVER_ID') || empty($rServers[SERVER_ID]['is_main'])) {
return self::sync(ServerRepository::getAll(true), (int) SERVER_ID, $rAll, $rTargetID, $rForce, CONFIG_PATH . 'openssl_extra');
}
/**
* Queue the main's OPENSSL_EXTRA for the chosen nodes.
*
* $rFile is the main's config/openssl_extra: without a value there the main
* runs on the built-in default and nothing is sent. The value sent is this
* process's OPENSSL_EXTRA, and only when it is the one the main's php-fpm
* mints with, as the main's own cron:servers (xc_vm) publishes it: run as
* root, this process can read a file that xc_vm cannot.
*
* @param array<int,array<string,mixed>> $rServers ServerRepository::getAll() rows.
*/
public static function sync(array $rServers, int $rMainID, bool $rAll, int $rTargetID, bool $rForce, string $rFile): int {
if (empty($rServers[$rMainID]['is_main'])) {
echo "Run this on the MAIN server.\n";
return 1;
}
@@ -65,7 +85,6 @@ class ServerSyncOpensslExtraCommand implements CommandInterface {
return 1;
}
$rFile = CONFIG_PATH . 'openssl_extra';
$rValue = is_file($rFile) ? @file_get_contents($rFile) : '';
if ($rValue === false) {
echo "Cannot read {$rFile}: run this as xc_vm or root.\n";
@@ -78,19 +97,27 @@ class ServerSyncOpensslExtraCommand implements CommandInterface {
}
$rMainPrint = OpensslExtra::fingerprint(OPENSSL_EXTRA);
$rCustomData = json_encode(['action' => 'set_openssl_extra', 'value' => OPENSSL_EXTRA]);
$rPublished = OpensslExtra::reportedFingerprint($rServers[$rMainID]);
if ($rPublished === null) {
echo "The main has not published its OPENSSL_EXTRA fingerprint yet: wait for its next cron:servers run (within a minute) and try again.\n";
return 1;
}
if (!hash_equals($rPublished, $rMainPrint)) {
echo "The main publishes another OPENSSL_EXTRA than {$rFile} holds, so its php-fpm does not mint with that value; sending it would break every token the main mints.\n";
echo "Either xc_vm cannot read the file (chown xc_vm:xc_vm {$rFile} && chmod 600 {$rFile}), or it changed within the last minute (wait for cron:servers).\n";
return 1;
}
$rCustomData = OpensslExtra::signal(OPENSSL_EXTRA);
$rQueued = 0;
foreach (($rAll ? $rServers : [$rTargetID => $rServers[$rTargetID]]) as $rServerID => $rServer) {
if ($rAll && !empty($rServer['is_main'])) {
continue;
}
$rSkip = self::skipReason($rServer, $rMainPrint, $rForce);
foreach (self::targets($rServers, $rAll, $rTargetID, $rMainPrint, $rForce) as $rServerID => $rSkip) {
$rName = $rServers[$rServerID]['server_name'] ?? '';
if ($rSkip !== null) {
echo "#{$rServerID} " . ($rServer['server_name'] ?? '') . ": skipped, {$rSkip}\n";
echo "#{$rServerID} {$rName}: skipped, {$rSkip}\n";
continue;
}
self::db()->query('INSERT INTO `signals`(`server_id`, `time`, `custom_data`) VALUES(?, ?, ?);', $rServerID, time(), $rCustomData);
echo "#{$rServerID} " . ($rServer['server_name'] ?? '') . ": queued\n";
echo "#{$rServerID} {$rName}: queued\n";
$rQueued++;
}
echo "{$rQueued} node(s) queued. Each applies it at its next cron:root_signals run (within a minute) and reports it at its next cron:servers run; check with server:diagnose <id>.\n";
@@ -98,6 +125,25 @@ class ServerSyncOpensslExtraCommand implements CommandInterface {
return 0;
}
/**
* The nodes a run considers, each with why it is skipped (null: it gets the
* main's value). --all leaves the main out; a single target is always listed.
*
* @param array<int,array<string,mixed>> $rServers
* @return array<int,?string> server_id => skip reason
*/
public static function targets(array $rServers, bool $rAll, int $rTargetID, string $rMainPrint, bool $rForce): array {
$rTargets = [];
foreach (($rAll ? $rServers : [$rTargetID => $rServers[$rTargetID] ?? []]) as $rServerID => $rServer) {
if ($rAll && !empty($rServer['is_main'])) {
continue;
}
$rTargets[$rServerID] = self::skipReason($rServer, $rMainPrint, $rForce);
}
return $rTargets;
}
/**
* Why $rServer does not get the main's value, or null when it does.
*
+3 -3
View File
@@ -551,16 +551,16 @@ class RootSignalsCronJob implements CommandInterface {
$db->close_mysql();
shell_exec('sudo reboot');
break;
case 'set_openssl_extra':
case OpensslExtra::SIGNAL_ACTION:
// Sent by server:sync-openssl-extra on MAIN. install() keeps the value it
// replaces open for tokens minted just before; php-fpm reads the new one
// on its next request. The MAIN's own value (hmac_keys, image names) never
// changes this way.
if (!empty($rServers[SERVER_ID]['is_main'])) {
$rSet = OpensslExtra::applySignal($rData, !empty($rServers[SERVER_ID]['is_main']), CONFIG_PATH, time());
if ($rSet === null) {
break;
}
echo 'Setting OPENSSL_EXTRA...' . "\n";
$rSet = is_string($rData['value'] ?? null) && OpensslExtra::install($rData['value'], CONFIG_PATH, time());
if ($rSet) {
shell_exec('sudo chown xc_vm:xc_vm ' . CONFIG_PATH . 'openssl_extra ' . CONFIG_PATH . 'openssl_extra.prev 2>/dev/null');
}
+1 -1
View File
@@ -158,7 +158,7 @@ class ServersCronJob implements CommandInterface {
$rHardware = ['total_ram' => $rStats['total_mem'], 'total_used' => $rStats['total_mem_used'], 'cores' => $rStats['cpu_cores'], 'threads' => $rStats['cpu_cores'], 'kernel' => $rStats['kernel'], 'total_running_streams' => $rStats['total_running_streams'], 'cpu_name' => $rStats['cpu_name'], 'cpu_usage' => $rStats['cpu'], 'network_speed' => $rStats['network_speed'], 'bytes_sent' => $rStats['bytes_sent'], 'bytes_received' => $rStats['bytes_received']];
// Lets server:diagnose tell whether this node reads MAIN's tokens (never the value itself).
$rHardware[OpensslExtra::HARDWARE_KEY] = OpensslExtra::fingerprint(OPENSSL_EXTRA);
$rHardware = OpensslExtra::publish($rHardware);
if (@fsockopen($rServers[SERVER_ID]['server_ip'], $rServers[SERVER_ID]['http_broadcast_port'], $rErrNo, $rErrStr, 3) || @fsockopen($rServers[SERVER_ID]['server_ip'], $rServers[SERVER_ID]['https_broadcast_port'], $rErrNo, $rErrStr, 3)) {
$rRemoteStatus = true;
+29
View File
@@ -31,6 +31,9 @@ final class OpensslExtra {
/** Seconds a replaced value still opens tokens. */
public const PREVIOUS_WINDOW = 600;
/** The root-signal action server:sync-openssl-extra queues and cron:root_signals applies. */
public const SIGNAL_ACTION = 'set_openssl_extra';
/** Test seam: the previous-value file to read instead of CONFIG_PATH's. */
private static ?string $rPrevFile = null;
@@ -61,6 +64,32 @@ final class OpensslExtra {
return (is_string($rPrint) && $rPrint !== '') ? $rPrint : null;
}
/** $rHardware with this node's fingerprint added, as cron:servers publishes it. */
public static function publish(array $rHardware): array {
$rHardware[self::HARDWARE_KEY] = self::fingerprint(OPENSSL_EXTRA);
return $rHardware;
}
/** The signals.custom_data that brings a node onto $rValue (applied by applySignal()). */
public static function signal(string $rValue): string {
return (string) json_encode(['action' => self::SIGNAL_ACTION, 'value' => $rValue]);
}
/**
* Apply a decoded signal() payload on this node.
*
* @return bool|null Null on the main, which never takes its value this way
* (it keys hmac_keys and image names); else install()'s result.
*/
public static function applySignal(array $rData, bool $rIsMain, string $rConfigDir, int $rNow): ?bool {
if ($rIsMain) {
return null;
}
return is_string($rData['value'] ?? null) && self::install($rData['value'], $rConfigDir, $rNow);
}
/**
* Make $rNew this node's OPENSSL_EXTRA.
*
+19 -12
View File
@@ -130,25 +130,32 @@ class Encryption {
* A token made for the OPENSSL_EXTRA context that does not open is tried once
* more with the value this node replaced, while that is still accepted
* (OpensslExtra::previous()), so switching a node onto MAIN's value does not
* break the links it has just handed out. A token that opens pays nothing extra.
* break the links it has just handed out. A sealed token that opens pays
* nothing extra; a legacy one pays one more open() while a value is accepted.
* The sealed format is tried with both values before the legacy one, since a
* legacy decrypt() under the wrong value can return garbage instead of false
* (about 1 in 256, when the padding happens to hold): a legacy token minted
* with the replaced value can still be missed at that rate, a sealed one cannot.
*
* @return string|false
*/
public static function readToken($token, $key, $deviceId, bool $rAcceptLegacy) {
$rPlain = self::readTokenWith($token, $key, $deviceId, $rAcceptLegacy);
if ($rPlain === false && defined('OPENSSL_EXTRA') && $deviceId === OPENSSL_EXTRA && ($rPrevious = OpensslExtra::previous()) !== null) {
return self::readTokenWith($token, $key, $rPrevious, $rAcceptLegacy);
}
return $rPlain;
}
/** @return string|false */
private static function readTokenWith($token, $key, $deviceId, bool $rAcceptLegacy) {
$rPlain = self::open($token, $key, $deviceId);
if ($rPlain !== false || !$rAcceptLegacy || !is_string($token)) {
if ($rPlain !== false) {
return $rPlain;
}
return self::decrypt($token, $key, $deviceId);
$rPrevious = (defined('OPENSSL_EXTRA') && $deviceId === OPENSSL_EXTRA) ? OpensslExtra::previous() : null;
if ($rPrevious !== null && ($rPlain = self::open($token, $key, $rPrevious)) !== false) {
return $rPlain;
}
if (!$rAcceptLegacy || !is_string($token)) {
return false;
}
$rPlain = self::decrypt($token, $key, $deviceId);
if ($rPlain === false && $rPrevious !== null) {
return self::decrypt($token, $key, $rPrevious);
}
return $rPlain;
}
private const SEAL_NONCE = 12;
+4 -1
View File
@@ -1,5 +1,6 @@
<?php
use XcVm\Core\Config\OpensslExtra;
use XcVm\Core\Config\SettingsManager;
use XcVm\Core\Database\DatabaseHandler;
use XcVm\Domain\Security\BlocklistService;
@@ -66,7 +67,9 @@ if (BlocklistService::isProxy($_SERVER['REMOTE_ADDR'])) {
$db->query('INSERT INTO `servers_stats`(`server_id`, `cpu`, `cpu_cores`, `cpu_avg`, `total_mem`, `total_mem_free`, `total_mem_used`, `total_mem_used_percent`, `total_disk_space`, `uptime`, `total_running_streams`, `bytes_sent`, `bytes_received`, `bytes_sent_total`, `bytes_received_total`, `cpu_load_average`, `connections`, `total_users`, `users`, `time`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);', $rServerIDRequest, $rStats['cpu'], $rStats['cpu_cores'], $rStats['cpu_avg'], $rStats['total_mem'], $rStats['total_mem_free'], $rStats['total_mem_used'], $rStats['total_mem_used_percent'], $rStats['total_disk_space'], $rStats['uptime'], $rStats['total_running_streams'], $rStats['bytes_sent'], $rStats['bytes_received'], $rStats['bytes_sent_total'], $rStats['bytes_received_total'], $rStats['cpu_load_average'], $rConnections, $rAllUsers, $rUsers, time());
$db->query('UPDATE `servers` SET `connections` = ?, `users` = ?, `ping` = ?,`server_hardware` = ?,`whitelist_ips` = ?, `interfaces` = ?, `watchdog_data` = ?, `last_check_ago` = ? WHERE `id` = ?', $rConnections, $rUsers, $rPing, json_encode($rHardware), json_encode($rAddresses), json_encode($rStats['interfaces']), json_encode($rStats, JSON_PARTIAL_OUTPUT_ON_ERROR), time(), $rServerIDRequest);
if ($db->query("SELECT `signal_id`, `custom_data` FROM `signals` WHERE `server_id` = ? AND `custom_data` <> '' ORDER BY signal_id ASC;", $rServerIDRequest)) {
// A proxy is never sent OPENSSL_EXTRA; this endpoint trusts the posted server_id,
// so those rows (an LB's, carrying the main's value) must not be handed out here.
if ($db->query("SELECT `signal_id`, `custom_data` FROM `signals` WHERE `server_id` = ? AND `custom_data` <> '' AND `custom_data` NOT LIKE ? ORDER BY signal_id ASC;", $rServerIDRequest, '%"action":"' . OpensslExtra::SIGNAL_ACTION . '"%')) {
if (0 < $db->num_rows()) {
foreach ($db->get_rows() as $rRow) {
$rData = json_decode($rRow['custom_data'], true);
+25 -2
View File
@@ -25,8 +25,11 @@ final class EncryptionPreviousExtraTest extends TestCase {
}
protected function tearDown(): void {
OpensslExtra::usePrevFile(null);
@unlink($this->rFile);
try {
OpensslExtra::usePrevFile(null);
} finally {
@unlink($this->rFile);
}
}
private function previous(string $rValue, int $rValidUntil): void {
@@ -53,6 +56,26 @@ final class EncryptionPreviousExtraTest extends TestCase {
$this->assertFalse(Encryption::readToken($rToken, 'k', OPENSSL_EXTRA, false));
}
/**
* Where legacy tokens are accepted, a sealed token minted with the replaced
* value must not be read as a legacy one with the current value first: that
* decrypt() returns garbage instead of false about once in 256 tokens (when
* the padding happens to hold). 48- and 64-byte tokens can pass CBC's length
* check; 2000 of them miss none only when both sealed attempts come first.
*/
public function testASealedTokenWithTheReplacedValueOpensWhereLegacyTokensAreAccepted(): void {
$this->previous('old-extra', time() + 600);
$rMissed = 0;
for ($i = 0; $i < 2000; $i++) {
$rPlain = str_pad('live/' . $i . '/', ($i % 2) ? 20 : 36, 'x');
if (Encryption::readToken(Encryption::seal($rPlain, 'k', 'old-extra'), 'k', OPENSSL_EXTRA, true) !== $rPlain) {
$rMissed++;
}
}
$this->assertSame(0, $rMissed);
}
public function testOnlyTheOpensslExtraContextFallsBack(): void {
$this->previous('old-extra', time() + 600);
+17 -9
View File
@@ -80,18 +80,26 @@ final class LbInstallFlowOpensslExtraTest extends TestCase {
$this->assertNotContains(['run', 'sudo chmod 600 ' . CONFIG_PATH . 'openssl_extra'], $this->rCalls);
}
/** Shipped as root before config/ is handed to xc_vm, so FPM can read it. */
/**
* Shipped as root after config.enc and before config/ is handed to xc_vm, so
* FPM can read it; a failed upload marks the server errored (status 4).
* provisionConfig() needs the xcvm_core extension (XC_VM::config_pack), so
* this reads its source, located by reflection.
*/
public function testProvisionConfigShipsTheValueBeforeHandingConfigToXcVm(): void {
$rSource = (string) file_get_contents(MAIN_HOME . 'Cli/Commands/LbInstallFlow.php');
$rStart = strpos($rSource, 'function provisionConfig(');
$this->assertNotFalse($rStart);
$rEnd = strpos($rSource, "\n\t}\n", $rStart);
$rBody = substr($rSource, $rStart, $rEnd - $rStart);
$rMethod = new ReflectionMethod(LbInstallFlow::class, 'provisionConfig');
$rLines = file((string) $rMethod->getFileName());
$rBody = implode('', array_slice($rLines, $rMethod->getStartLine() - 1, $rMethod->getEndLine() - $rMethod->getStartLine() + 1));
$rProvision = strpos($rBody, 'self::provisionOpensslExtra(');
$rGuard = preg_quote("if (!self::provisionOpensslExtra(\$rConn, \$rRunSSH, \$rSendFileSSH, CONFIG_PATH . 'openssl_extra')) {", '/');
$rFail = preg_quote("\$db->query('UPDATE `servers` SET `status` = 4 WHERE `id` = ?;', \$rServerID);", '/');
$this->assertSame(1, preg_match('/' . $rGuard . '\s*' . $rFail . '\s*echo [^;]+;\s*return false;\s*\}/', $rBody, $rMatch, PREG_OFFSET_CAPTURE), 'a failed upload fails the install');
$rConfigEnc = strpos($rBody, "CONFIG_PATH . 'config.enc', false)");
$rChown = strpos($rBody, "'sudo chown -R xc_vm:xc_vm '");
$this->assertNotFalse($rProvision, 'provisionConfig ships OPENSSL_EXTRA');
$this->assertNotFalse($rConfigEnc);
$this->assertNotFalse($rChown);
$this->assertLessThan($rChown, $rProvision);
$this->assertGreaterThan($rConfigEnc, $rMatch[0][1]);
$this->assertLessThan($rChown, $rMatch[0][1]);
}
}
+45 -2
View File
@@ -11,6 +11,11 @@ use XcVm\Core\Config\OpensslExtra;
* servers.server_hardware, so the values can be compared without leaving the
* node.
*
* publish: what cron:servers adds to the node's server_hardware.
*
* signal / applySignal: the root signal server:sync-openssl-extra queues on
* MAIN and cron:root_signals applies on an LB; both sides share it here.
*
* install / previous: the root signal that brings an LB onto MAIN's value
* keeps the value it replaced for a short window, so tokens the LB minted with
* it still open (Encryption::readToken).
@@ -25,8 +30,11 @@ final class OpensslExtraTest extends TestCase {
}
protected function tearDown(): void {
OpensslExtra::usePrevFile(null);
exec('rm -rf ' . escapeshellarg($this->rDir));
try {
OpensslExtra::usePrevFile(null);
} finally {
exec('rm -rf ' . escapeshellarg($this->rDir));
}
}
/** Write a previous-value file (JSON unless given raw) and point previous() at it. */
@@ -59,6 +67,41 @@ final class OpensslExtraTest extends TestCase {
$this->assertNull(OpensslExtra::reportedFingerprint([]));
}
public function testPublishAddsThisNodesFingerprintToItsHardware(): void {
$rHardware = OpensslExtra::publish(['cores' => 4]);
$this->assertSame(OpensslExtra::fingerprint(OPENSSL_EXTRA), OpensslExtra::reportedFingerprint(['server_hardware' => json_encode($rHardware)]));
$this->assertSame(4, $rHardware['cores']);
}
/** An LB on an older or newer build must still recognise the action: its name is the wire contract. */
public function testTheSignalCarriesTheValueUnderAFixedAction(): void {
$this->assertSame('set_openssl_extra', OpensslExtra::SIGNAL_ACTION);
$this->assertSame(['action' => 'set_openssl_extra', 'value' => 'main-extra'], json_decode(OpensslExtra::signal('main-extra'), true));
// proxy_api.php leaves out the rows that match this, so no proxy is handed the value.
$this->assertStringContainsString('"action":"' . OpensslExtra::SIGNAL_ACTION . '"', OpensslExtra::signal('main-extra'));
}
public function testAnLbAppliesTheSignalAndKeepsItsOldValueForTheWindow(): void {
$this->assertTrue(OpensslExtra::applySignal(json_decode(OpensslExtra::signal('main-extra'), true), false, $this->rDir, 1000));
$this->assertSame('main-extra', file_get_contents($this->rDir . 'openssl_extra'));
$this->assertSame(['value' => OPENSSL_EXTRA, 'valid_until' => 1600], json_decode((string) file_get_contents($this->rDir . 'openssl_extra.prev'), true));
}
/** The main's value keys hmac_keys and image names: a signal never changes it. */
public function testTheMainIgnoresTheSignal(): void {
$this->assertNull(OpensslExtra::applySignal(json_decode(OpensslExtra::signal('main-extra'), true), true, $this->rDir, 1000));
$this->assertSame([], array_values(array_diff(scandir($this->rDir), ['.', '..'])));
}
public function testASignalWithoutAStringValueIsNotApplied(): void {
foreach ([['action' => OpensslExtra::SIGNAL_ACTION], ['action' => OpensslExtra::SIGNAL_ACTION, 'value' => 42], ['action' => OpensslExtra::SIGNAL_ACTION, 'value' => ['x']], ['action' => OpensslExtra::SIGNAL_ACTION, 'value' => '']] as $rData) {
$this->assertFalse(OpensslExtra::applySignal($rData, false, $this->rDir, 1000), json_encode($rData));
}
$this->assertSame([], array_values(array_diff(scandir($this->rDir), ['.', '..'])));
}
public function testInstallWritesTheNewValueAndKeepsTheOldOneForTheWindow(): void {
$this->assertTrue(OpensslExtra::install('new-extra', $this->rDir, 1000));
+24 -2
View File
@@ -9,6 +9,10 @@ use XcVm\Core\Config\OpensslExtra;
* server_hardware with the MAIN's. A mismatch means every token MAIN mints for
* a redirect is rejected by that node. A node that publishes none runs an
* older build; proxies publish none and are not checked.
*
* The section is called through reflection. That Mode A (diagnoseFromMain) and
* Mode B (diagnoseLocal) still call it is pinned from their source only: running
* either mode probes the network and the host.
*/
final class ServerDiagnoseOpensslExtraTest extends TestCase {
@@ -27,9 +31,21 @@ final class ServerDiagnoseOpensslExtraTest extends TestCase {
$rMethod->setAccessible(true);
$rProblems = [];
ob_start();
$rMethod->invokeArgs(new ServerDiagnoseCommand(), [$rServer, $rMain, &$rProblems]);
try {
$rMethod->invokeArgs(new ServerDiagnoseCommand(), [$rServer, $rMain, &$rProblems]);
} finally {
$rOutput = (string) ob_get_clean();
}
return [(string) ob_get_clean(), $rProblems];
return [$rOutput, $rProblems];
}
/** The source of one of the command's methods, as reflection locates it. */
private function body(string $rName): string {
$rMethod = new ReflectionMethod(ServerDiagnoseCommand::class, $rName);
$rLines = file((string) $rMethod->getFileName());
return implode('', array_slice($rLines, $rMethod->getStartLine() - 1, $rMethod->getEndLine() - $rMethod->getStartLine() + 1));
}
public function testAMismatchIsReportedAsAProblem(): void {
@@ -58,6 +74,12 @@ final class ServerDiagnoseOpensslExtraTest extends TestCase {
$this->assertSame([], $rProblems);
}
/** Both modes still report the check (see the class comment). */
public function testBothModesRunTheCheck(): void {
$this->assertStringContainsString('$this->opensslExtraSection($rServer, $rMain, $rProblems);', $this->body('diagnoseFromMain'));
$this->assertStringContainsString('$this->opensslExtraSection($rMe, $rMain, $rProblems);', $this->body('diagnoseLocal'));
}
public function testProxiesAndAMissingMainAreSkipped(): void {
$this->assertSame(['', []], $this->check($this->row(3, null, ['server_type' => 1]), $this->row(1, 'main-extra')));
$this->assertSame(['', []], $this->check($this->row(2, 'lb-extra'), null));
@@ -9,10 +9,151 @@ use XcVm\Core\Config\OpensslExtra;
* LBs that report another one. A node that does not report a fingerprint runs
* a build whose root cron cannot apply the value, and an offline node would
* leave it waiting in the signals table, so both need --force; proxies and the
* MAIN never get it.
* MAIN never get it. Nothing is sent when the main has no value of its own, or
* when the value this process reads is not the one the main publishes (a file
* root can read and xc_vm's php-fpm cannot).
*
* The LB side (cron:root_signals) applies the row through
* OpensslExtra::applySignal, covered in OpensslExtraTest.
*/
final class ServerSyncOpensslExtraCommandTest extends TestCase {
private TestDb $db;
private string $rFile;
protected function setUp(): void {
$this->db = new TestDb();
$this->db->exec('CREATE TABLE signals (signal_id INTEGER PRIMARY KEY, server_id INTEGER, `time` INTEGER, custom_data TEXT);');
ServerSyncOpensslExtraCommand::setDb($this->db);
$this->rFile = sys_get_temp_dir() . '/xcvm_sync_openssl_extra_' . uniqid();
file_put_contents($this->rFile, 'main-extra-on-disk');
}
protected function tearDown(): void {
try {
(new ReflectionProperty(ServerSyncOpensslExtraCommand::class, 'db'))->setValue(null, null);
} finally {
@unlink($this->rFile);
}
}
/** Main #1 (publishes this process's value), LBs #2 and #5 on another value, #3 in sync, proxy #4. */
private function cluster(?string $rMainValue = OPENSSL_EXTRA): array {
$rMainHardware = $rMainValue === null ? ['cores' => 8] : [OpensslExtra::HARDWARE_KEY => OpensslExtra::fingerprint($rMainValue)];
return [
1 => ['id' => 1, 'is_main' => 1, 'server_type' => 0, 'server_online' => true, 'server_name' => 'Main', 'server_hardware' => json_encode($rMainHardware)],
2 => $this->lb(['id' => 2]),
3 => $this->lb(['id' => 3, 'server_hardware' => json_encode([OpensslExtra::HARDWARE_KEY => OpensslExtra::fingerprint(OPENSSL_EXTRA)])]),
4 => $this->lb(['id' => 4, 'server_type' => 1, 'server_hardware' => json_encode(['cores' => 2])]),
5 => $this->lb(['id' => 5]),
];
}
/** @return array{0:int,1:string} [exit code, output] */
private function sync(array $rServers, bool $rAll, int $rTargetID, bool $rForce): array {
ob_start();
try {
$rCode = ServerSyncOpensslExtraCommand::sync($rServers, 1, $rAll, $rTargetID, $rForce, $this->rFile);
} finally {
$rOutput = (string) ob_get_clean();
}
return [$rCode, $rOutput];
}
/** @return array<int,string> server_id => custom_data of every queued signal */
private function queued(): array {
$this->db->query('SELECT server_id, custom_data FROM signals ORDER BY signal_id;');
$rQueued = [];
foreach ($this->db->get_rows() as $rRow) {
$this->assertArrayNotHasKey((int) $rRow['server_id'], $rQueued, 'one signal per node');
$rQueued[(int) $rRow['server_id']] = $rRow['custom_data'];
}
return $rQueued;
}
public function testEachMismatchedLbGetsOneSignalCarryingTheMainsValue(): void {
[$rCode] = $this->sync($this->cluster(), true, 0, false);
$this->assertSame(0, $rCode);
$rSignal = OpensslExtra::signal(OPENSSL_EXTRA);
$this->assertSame([2 => $rSignal, 5 => $rSignal], $this->queued());
$this->assertSame(['action' => OpensslExtra::SIGNAL_ACTION, 'value' => OPENSSL_EXTRA], json_decode($rSignal, true));
}
public function testForceNeverQueuesTheMainOrAProxy(): void {
[$rCode] = $this->sync($this->cluster(), true, 0, true);
$this->assertSame(0, $rCode);
$this->assertSame([2, 3, 5], array_keys($this->queued()));
}
public function testASingleTargetIsQueuedAlone(): void {
$this->assertSame(0, $this->sync($this->cluster(), false, 5, false)[0]);
$this->assertSame([5], array_keys($this->queued()));
[$rCode, $rOutput] = $this->sync($this->cluster(), false, 1, true);
$this->assertSame(0, $rCode);
$this->assertStringContainsString('#1 Main: skipped, the main', $rOutput);
$this->assertSame([5], array_keys($this->queued()));
}
/** Without config/openssl_extra the main runs on the built-in default, as its LBs do. */
public function testNothingIsQueuedWhenTheMainHasNoValueOfItsOwn(): void {
foreach (['', " \n"] as $rEmpty) {
file_put_contents($this->rFile, $rEmpty);
$this->assertSame(0, $this->sync($this->cluster(), true, 0, true)[0]);
}
@unlink($this->rFile);
$this->assertSame(0, $this->sync($this->cluster(), true, 0, true)[0]);
$this->assertSame([], $this->queued());
}
/** Run as root, the command can read a file xc_vm cannot: the main's php-fpm then mints with another value. */
public function testNothingIsQueuedUnlessTheMainPublishesTheValueToSend(): void {
[$rCode, $rOutput] = $this->sync($this->cluster('built-in-default'), true, 0, true);
$this->assertSame(1, $rCode);
$this->assertStringContainsString('chown xc_vm:xc_vm', $rOutput);
[$rCode, $rOutput] = $this->sync($this->cluster(null), true, 0, true);
$this->assertSame(1, $rCode);
$this->assertStringContainsString('cron:servers', $rOutput);
$this->assertSame([], $this->queued());
}
public function testItRunsOnlyOnTheMainAndForAKnownServer(): void {
$rServers = $this->cluster();
$rServers[1]['is_main'] = 0;
$this->assertSame(1, $this->sync($rServers, true, 0, true)[0]);
$this->assertSame(1, $this->sync($this->cluster(), false, 9, true)[0]);
$this->assertSame([], $this->queued());
}
public function testTargetsListsEveryNodeButTheMainWithItsSkipReason(): void {
$rTargets = ServerSyncOpensslExtraCommand::targets($this->cluster(), true, 0, OpensslExtra::fingerprint(OPENSSL_EXTRA), false);
$this->assertSame([2, 3, 4, 5], array_keys($rTargets));
$this->assertNull($rTargets[2]);
$this->assertSame('already in sync', $rTargets[3]);
$this->assertSame('a proxy', $rTargets[4]);
$this->assertNull($rTargets[5]);
}
/** Its own case with its own break: falling through would run the next action (restart_services). */
public function testTheLbAppliesTheSignalInItsOwnCase(): void {
$rSource = (string) file_get_contents(MAIN_HOME . 'Cli/CronJobs/RootSignalsCronJob.php');
$this->assertSame(1, preg_match('/case OpensslExtra::SIGNAL_ACTION:(.*?)\n\s*case /s', $rSource, $rMatch));
$this->assertStringContainsString('OpensslExtra::applySignal($rData, !empty($rServers[SERVER_ID][\'is_main\']), CONFIG_PATH, time())', $rMatch[1]);
$this->assertMatchesRegularExpression('/break;\s*$/', $rMatch[1]);
}
private function lb(array $rOverride = []): array {
return array_merge([
'id' => 2,