Files
XC_VM/tests/Unit/ClusterSchemaTest.php
T
obscuremind 4566ecbd52 feat(cluster): report settings reads the replica would not answer
Plan section 9, R1 `settings`: API nodes log unknown-key reads to
`audit.settings_misses`. Once a node's replica owns its settings, a key
outside lb_settings_keys.php is absent there, and the CI scan behind the
allowlist cannot see a key built at run time. The count names such reads
before they fail.

On the node (Core/Cluster/SettingsAudit): in mode 1 or 2, as the agent's
flows.json says (NodeFlows::declared, no database and no servers read),
each read through SettingsManager's getters (get, getBool, getInt,
getString, getArray, has) of a key outside the allowlist's `keys` and
`withheld` is counted. A process keeps its counts in memory and merges
them at exit, and every minute while it runs, into a per-day file under
its lock (STORAGE_PATH/cluster/settings_misses/YYYYMMDD.json). It then
writes config/cluster/audit.json, `{"settings_misses": {key: count}}`
over the last seven UTC days, which the agent is to send as the
heartbeat's `audit`. Everything is bounded: 64 names, the rest and any
key that is not a settings name under "*". cron:cleanup prunes the day
files after eight days and rewrites audit.json every hour, so old days
drop out; in mode 0 the file is removed. MAIN and legacy nodes have no
flows.json and count nothing; a read costs one lookup.

On MAIN (Domain/Cluster/NodeAudit): a heartbeat's `audit` is normalised
(names and positive integer counts only, at most 16 KiB, the least
missed past 64 names folded into "*") and kept in the new
cluster_nodes.audit (migration 045, mirrored in database.sql; 044 is taken
by work on another branch). It is written only when it differs from what
the row holds, which every request reads already, so an unchanged report
costs no query, on the cluster bus or not. A heartbeat without `audit`,
today's agent's, changes nothing. The Cluster Nodes page shows each mode 1
or 2 node's misses.

SettingsAuditTest covers the counting per getter, withheld keys not
counted, mode 0 and MAIN counting nothing, the merge across processes,
the caps, the seven-day window, the pruning, the empty report and the
minute's merge of a long-running process. ClusterApiTest covers the
heartbeat storing a report once, an unchanged, missing or malformed one
writing nothing, an empty one clearing the page, and the cap; and
ReplicaBootTest a child process's miss reaching audit.json at its exit,
while booting from the replica reads only allowlisted settings.
2026-09-27 02:26:55 +00:00

111 lines
5.3 KiB
PHP

<?php
use PHPUnit\Framework\TestCase;
/**
* The cluster schema exists twice: as the core migrations from 028 on for
* upgrades (028–045, then later ones that ALTER a cluster table) and in
* database.sql for fresh installs. Both were loaded into MariaDB 10.11 and
* compared column by column when written; this test keeps them from drifting
* where CI has no database.
*/
final class ClusterSchemaTest extends TestCase {
private const MIGRATIONS = ['028_add_cluster_settings', '029_create_cluster_nodes', '030_create_cluster_commands', '031_create_cluster_enrolment', '032_create_cluster_audit', '033_add_crontab_role', '034_create_cluster_changes', '035_add_cluster_epoch_eph', '036_add_cluster_enrol_request_eph', '037_enable_cluster_cron', '038_add_cluster_endpoint_settings', '039_add_cluster_node_root_ready', '040_add_cluster_db_allowlist', '041_add_cluster_node_features', '042_crontab_cleanup_role_all', '043_crontab_main_roles', '045_add_cluster_node_audit'];
private function src(string $rPath): string {
return (string) file_get_contents(dirname(__DIR__, 2) . '/src/' . $rPath);
}
/** @return array<string, string> table => column block */
private function tables(string $rSql): array {
preg_match_all('/CREATE TABLE IF NOT EXISTS `([a-z_]+)` \((.*?)\n\) ENGINE/s', $rSql, $rM);
return array_combine($rM[1], array_map('trim', $rM[2]));
}
/**
* The cluster migrations above, then every later core migration from 028
* on: one may ALTER a column into a cluster table (046 adds
* `cluster_nodes.main_port`) without being listed.
*
* @return list<string>
*/
private function migrations(): array {
$rNames = self::MIGRATIONS;
foreach (glob(dirname(__DIR__, 2) . '/src/migrations/database/up/*.sql') ?: [] as $rFile) {
if ((int) basename($rFile) >= 28) {
$rNames[] = basename($rFile, '.sql');
}
}
return array_values(array_unique($rNames));
}
public function testEveryMigrationHasADownFile(): void {
foreach (self::MIGRATIONS as $rName) {
$this->assertFileExists(dirname(__DIR__, 2) . '/src/migrations/database/up/' . $rName . '.sql');
$this->assertFileExists(dirname(__DIR__, 2) . '/src/migrations/database/down/' . $rName . '.sql');
}
}
public function testClusterTablesMatchDatabaseSql(): void {
$rInstall = $this->tables($this->src('bin/install/database.sql'));
// Columns a later migration ALTERs into a table: present in database.sql,
// absent from the migration that created the table.
$rAdded = [];
foreach ($this->migrations() as $rName) {
if (preg_match_all('/ALTER TABLE `([a-z_]+)` ADD COLUMN IF NOT EXISTS `([a-z_]+)`/', $this->src('migrations/database/up/' . $rName . '.sql'), $rM, PREG_SET_ORDER)) {
foreach ($rM as [, $rTable, $rColumn]) {
$rAdded[$rTable][] = $rColumn;
}
}
}
foreach ($rAdded as $rTable => $rColumns) {
if (!isset($rInstall[$rTable])) {
continue;
}
foreach ($rColumns as $rColumn) {
$this->assertStringContainsString('`' . $rColumn . '`', $rInstall[$rTable], $rTable . '.' . $rColumn);
$rInstall[$rTable] = (string) preg_replace('/\n\s*`' . $rColumn . '` [^\n]*/', '', $rInstall[$rTable]);
}
}
$rCount = 0;
foreach (self::MIGRATIONS as $rName) {
foreach ($this->tables($this->src('migrations/database/up/' . $rName . '.sql')) as $rTable => $rBody) {
$this->assertArrayHasKey($rTable, $rInstall, $rTable . ' missing from database.sql');
$this->assertSame($rBody, $rInstall[$rTable], $rTable);
$rCount++;
}
}
$this->assertSame(11, $rCount);
}
public function testSettingsColumnsMatchDatabaseSql(): void {
preg_match_all('/ADD COLUMN IF NOT EXISTS (`[a-z_]+` [^\n]*?),?\n/', $this->src('migrations/database/up/028_add_cluster_settings.sql') . "\n", $rM);
$this->assertCount(19, $rM[1]);
$rSettings = $this->tables($this->src('bin/install/database.sql'))['settings'];
foreach ($rM[1] as $rColumn) {
$this->assertStringContainsString(' ' . rtrim($rColumn, ';') . ',', $rSettings);
}
}
public function testCrontabRoles(): void {
$rSql = $this->src('bin/install/database.sql');
$this->assertStringContainsString("`role` enum('all','main','legacy')", $rSql);
// 033, then 043: the crons the plan's cron table gives MAIN alone.
$rMain = ['tmdb', 'tmdb_popular', 'update', 'epg', 'series', 'backups', 'cache_engine', 'providers', 'stats', 'proxy', 'watch', 'plex'];
foreach ($rMain as $rCron) {
$this->assertMatchesRegularExpression("/\\(\\d+, '" . $rCron . "', '[^']*', 1, 'main'\\)/", $rSql, $rCron);
}
$rUp = $this->src('migrations/database/up/043_crontab_main_roles.sql');
foreach (array_slice($rMain, 3) as $rCron) {
$this->assertStringContainsString("'" . $rCron . "'", $rUp, $rCron);
}
// maxmind keeps each node's own GeoIP databases current.
$this->assertMatchesRegularExpression("/\\(\\d+, 'maxmind', '[^']*', 1, 'all'\\)/", $rSql);
// cleanup prunes each node's own files; only its table rotation is MAIN's (042).
$this->assertMatchesRegularExpression("/\\(\\d+, 'cleanup', '[^']*', 1, 'all'\\)/", $rSql);
$this->assertStringContainsString("SET `role` = 'all' WHERE `filename` = 'cleanup'", $this->src('migrations/database/up/042_crontab_cleanup_role_all.sql'));
$this->assertStringContainsString("(30, 'cluster', '* * * * *', 1, 'main')", $rSql, 'cron:cluster runs on MAIN');
$this->assertFileExists(dirname(__DIR__, 2) . '/src/Cli/CronJobs/ClusterCronJob.php');
}
}