Leads of the 2026-10-10 scan, re-read and confirmed:
- transactional() over a DDL statement: the statement commits on its own,
PDO then refused the commit and the rollback, so a module migration that
had done its work threw "There is no active transaction", the callback's
own error was hidden, and the handler stayed "in a transaction" for the
rest of the process. commit() and rollback() leave the transaction first
and ask PDO only for one that is still open.
- A module updated at the second attempt stayed Failed, and unloaded.
- A module archive is told apart by its content: a stored one is named
.zip whatever it is and an upload has no extension, so a tar.gz update
could not be extracted again or fetched by a load balancer, and a plain
.tar upload failed.
- A 404 page, a redirect's body or a page answered 200 was cached under an
image's name and never fetched again: only a 2xx answer that is an image
is cached.
- A prebuffer shorter than a segment returned the whole playlist
(array_slice(..., -0)).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XC_VM_Fanout's interop harness runs MAIN's PHP as many processes over
one database. Since the unit suite moved to MariaDB, every TestDb opened
a throwaway schema of its own and dropped it with the instance, so the
harness could not share MAIN's state and every panel-backed agent test
failed at its setup.
new TestDb('xcvm_t<pid>_<n>') opens that schema, creating it when
missing, and no instance drops it. Named after a process that outlives
the others (the Go test), TestDb's orphan cleanup drops it once that
process has ended. ADR 0004's note on the interop test says it runs
again.
The suite ran on in-memory SQLite, an engine production never uses: it hid
column lengths, MySQL-only syntax and type coercion (the identity bug fixed in
the previous commit passed CI that way). SQLite is gone:
- TestDb connects only to MariaDB/MySQL (XCVM_TEST_DB_DSN/_USER/_PASS; root over
the local socket, else 127.0.0.1:3306). Each instance gets a throwaway
schema, dropped with it, in the installer's sql_mode, with an explicit
connect timeout (a boot under test can zero default_socket_timeout).
- InstallSchema and the tests take the production DDL as is: the SQLite
reducers, the keys they re-added, the faked information_schema and the
sqliteCreateFunction() stand-ins are removed; test DDL is MySQL DDL.
- ReplicaBootTest, DbConnectRefusalTest and LazyDatabaseHandlerTest reach the
test server instead of SQLite files; RecordingFinalizer no longer swallows
an information_schema failure "for tests on SQLite".
- `make test-db` starts a throwaway mariadb:11.4 (tmpfs) for a machine without
a server; the CI PHPUnit job gets the same as a service.
The suite also runs on a panel host with the bundled PHP and its own php.ini:
tests reach the code through MAIN_HOME, not `<repo>/src`; repository checks
and process-isolated tests (ionCube Loader + OPcache CLI segfault on a script
read from stdin) are in the skip-on-panel group; xcvm_test_child_php() starts
a child that fakes \XC_VM without the real extension; Redis-down tests use
RedisManager::useConnector() instead of an eval'd \XC_VM.
Stale opt-in tests fixed on the way: the real-agent provisioning test answers
the agent-started check, the extension integration test sends the typed starts
and reads the binary signature raw, a fake producer starts as PHP_BINARY, and
the certbot test waits for the line its background command writes.
A node's batch or snapshot holds the locks of its rows until it commits.
MAIN's close of an ended row (its sweep, its mint) waited for that lock,
up to innodb_lock_wait_timeout: a playlist request or the whole sweep
could stand still behind a node's long transaction. The DELETE now runs
with no lock wait: a locked row fails at once and stays for the next
sweep.
The mint closes the player's ended record in the table store only. In
Redis MAIN's sweep works from a list it read at the start of its pass,
so a record closed at the mint and opened by the node meanwhile would
be removed by that pass and logged twice. Redis is as it was: the sweep
closes every server's ended record within the minute.
A test pins the re-open check for a record resent unchanged, which
MariaDB reports as no row changed.
An HLS connection's id names the player, not the server. A player whose
record on one server had ended, and who then opened on a node whose
agent holds its viewers, was refused by MAIN's ingest until the old
record was swept, so the line's limit did not count it. With lines_live
as the store that lasted up to 30 minutes: each server sweeps its own
rows, a node in mode 2 has no database to sweep them in, and MAIN took
another server's row only as orphaned, after 1800 s.
MAIN now closes such a record on its own decision, never on a node's
event:
- its sweep closes what a node in mode 2 said has ended (the table
store), as it already does for every server in Redis mode;
- at its token mint for a live HLS request it closes the same player's
ended record on another server, in both stores and for unlimited
lines too, so the node's record finds nothing in its way.
The record goes only while it is still ended (a guarded DELETE, a WATCH
in Redis), its activity line is written at once, and its server is told
to drop the viewer's marker, once per sweep.
The ingest is as strict as before: a node writes only its own
connections, and an open record never makes way. One check is added: an
update that changed no row because MAIN closed it in between is not
acknowledged, and the node sends the event again.
Not covered: the old server's record is still open and the line has
room. MAIN leaves it (it may be a second identical device on the line),
so the viewer is counted there until that record ends.
Database::get_row()/get_rows() pass every value through clean_row(), whose
HTML escaping (< > to entities, \r to \n) and trim() corrupt binary data.
The cluster read its node keys, epoch records, sealed tokens, enrol-code
keys and MACs, and command payloads and signatures that way, so:
- about half of all commands reached the agent with a signature that no
longer verified ("refused: bad signature");
- an epoch record (up to 2 KB) almost always came back altered, so a node's
first token could not be opened (TOKEN_EXPIRED, RECORD:epoch_record);
- a node whose signing or box key held one of those bytes could never
authenticate or be sealed to.
Database gains get_raw_rows()/get_raw_row(), which return the rows as
stored, and every cluster read of a binary column or a signed document uses
them. The admin pages keep the escaped fetch.
TestDb now escapes like Database, so the tests see what production sees:
before this fix, 70 cluster tests failed under it.
- Migration 047's counter never restarts below a row: up starts it at the
highest version a row holds (1 on an empty table), down empties
cluster_stream_ver (only this code writes it) and removes the counter,
the floors and the pruning's place, and a counter found missing starts
past every row. A rollback then an upgrade no longer loops nodes
through `full` or loses edits below their cursor.
- StreamReplica::prune walks the table in key order from a cursor in
cluster_meta (stream_ver_prune), 10,000 rows a run, so rows of held
streams no longer keep it from the rows no node holds. Held checks and
deletes go in chunks of 1000; the count is what was deleted.
- ReplicaStreams reads no section without a readable streams/ directory
(GLOB_ERR), and the contract has the agent keep it.
- A panel migration seeds a version-0 row for every holder it made
(StreamVersions::seedHolders) before its reset, and post.php's movie
title tool dispatches for the titles it changed.
- The streams op takes `resync: null` as a delta and null or missing
`hashes` as none, and refuses stream id 0.
- Tests: TestDb runs transactions over its PDO and QueryLogDb hands them
on, so the bump's rollback is exercised; migration 047 down then up; a
missing counter; the import's seeding; the pruning past more held rows
than a run reads, the floor at the newest lost version and the delete's
guard; resync paging past MAX_EXAMINE and a delta's `more` at MAX_ROWS;
MAIN's cluster entry point subscribing StreamVersions; a mass edit and
the settings' argument defaults dispatching. The writer scan now works
function by function (and per statement at top level), ignores `use`
imports and statements that set only local columns, and lists its
exceptions with reasons.
- ADR 0004 and the plan: the node-written carried columns
(recordings.status, movie_properties) as a difference from the plan,
the VOD check's runtime columns, the pruning walk, the bus sections
naming `streams`, and the agent's contract (dot-named temp files, the
streams/ directory, `hashes` as an object taken from the stored etag,
`resync` omitted, 409 NOT_ACTIVE, 404 UNKNOWN_OP, the rollback).
- storeSettings() counts its conditional UPDATE as stored only when it
changed the row (the affected-row count; the version always changes on
a match). The read-back it replaces took another admin's save of the
same transport and name for this one: the save's other columns were
never written, yet it reported success and audited a second change.
- Every settings save that posts cluster_transport or cluster_main_host
goes through storeSettings() on MAIN, which compares them with the
stored row instead of the request's settings cache. A stale cache let
a save move the stored name back with no version bump and nothing
kept. An unchanged save is stored without a bump, also before
migration 044.
- cluster:endpoint lists the kept ports and URLs, and "drop <url|host>"
drops a kept URL at once (ClusterEndpoint::drop(): over the lists as
read, the version raised, audited as cluster.endpoint_dropped), for an
old name or address MAIN gives up. MAIN only, stripped from LB builds.
- ADR 0004: state what today's agent does with a kept URL whose host now
answers for someone else (a 200 octet-stream reply that fails the MAC
ends the request and clears the URL's back-off, so the agent can stay
off MAIN for up to 10 minutes after each outage), and make backing off
such a URL and trying the next one a required agent change.
- Tests: the version guard and the affected-row check alone, a stale
cache, the unchanged-save path, the MAX_URLS cap on settings saves,
recordMainChange()'s retry limit (failing instead of hanging), and the
command. TestDb::num_rows() now counts the rows a write changed, as
Database::num_rows() does.
- Docs: the MAIN Host Name change and cluster:endpoint on the server
diagnostics page; the plan's Phase 3 status line.
live.php, vod.php, timeshift.php and rtmp.php no longer touch
lines_live or Redis inline: they record, find, refresh and check in
their viewers through ConnectionTracker only (openRecord, findByUuid
with the table path's HTTP Range fallback, updateLive, heartbeat,
acceptedIP; createLive now goes through openRecord). Behaviour is
unchanged on both stores: each caller still writes exactly the
lines_live columns it wrote (VOD leaves external_device NULL, RTMP keeps
its own date_start on the table path). The seam resolves the database
through the current handle (DatabaseFactory::get()), which follows the
endpoints' connectLazy()/close(), not a handle injected at boot.
ConnectionStoreTest pins both stores (lines_live on SQLite, and a real
redis-server started for the test); TestDb answers ping() for
connectLazy(). The accepted-IP source test follows the seam.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DcNdxHieZuUh3EnMUKt7zf
Rollback previously swapped in older code but never touched schema
changes newer migrations applied — the rollback's own comment admitted
"migrations are forward-only, so a downgrade cannot undo them". Of the
25 core migrations, two actually DROP a column older code still reads
(013_drop_asn_allocated, 016_per_repo_update_channel); rolling back
past either without restoring it would crash the restored old code.
Add MigrationRunner::rollback(): diffs the currently-applied migration
set (the `migrations` tracking table) against the TARGET version's own
migrations/database/up/ file list — read straight out of the
already-downloaded rollback tarball via the new
ReleaseArchiveInspector (PharData/tar, no extra network call, no full
extraction) — and reverses, in descending order, every migration the
target version doesn't carry. Only migrations with a down.sql
counterpart are reversed; migrations lacking one are left applied and
logged as skipped, which is safe by design since every migration
without a down file only ever ADDs schema or business data that older
code simply never references.
Restructured src/migrations/ into database/up/ (the 25 existing
forward migrations, unchanged content) and database/down/ (18 new
reverse migrations: the two critical drops plus every mechanically-safe
ADD-column/cron-row inverse; excluded are three migrations touching
real business data — activation_codes, category_templates, lines'
custom_data — and four data UPDATEs with unrecoverable prior values).
Kept as two directories rather than a `.down.sql`-suffixed sibling of
the up file: glob('*.sql') in run() would otherwise pick up the down
file too (it still ends in .sql) and try to run it forward.
Wired into UpdateCommand's rollback case right after the tarball
download+MD5 verification (MAIN only, aborts the rollback on failure —
the pre-rollback DB backup remains the recovery path either way).
Also extends the shared TestDb harness (tests/Support/TestDb.php) to
strip MySQL-only ENGINE=/CHARSET=/COLLATE= table-option clauses under
its default SQLite backend, so MigrationRunner's real production DDL
can be exercised in tests without a live MariaDB.
TestDb can now run the DB-touching unit tests against a real MariaDB as
well as the default in-memory SQLite, so the suite can be exercised on
the panel host (which ships pdo_mysql, not pdo_sqlite). When
XCVM_TEST_DB_DSN is set it connects there and translates the SQLite test
DDL on the fly: AUTOINCREMENT -> AUTO_INCREMENT, a bare INTEGER PRIMARY
KEY gains AUTO_INCREMENT, and each CREATE TABLE is preceded by DROP TABLE
IF EXISTS (a MariaDB schema persists across the per-test connections that
:memory: starts fresh). DDL is routed through exec() on both backends so
the ModuleMigrator path (which runs DDL via query()) works too, and the
MySQL session uses a permissive sql_mode to match SQLite's leniency.
AuthRepositoryTest back-quotes the reserved column `key`.
Three env-fragile guards are tagged #[Group('skip-on-panel')] so the
deployed-panel run can exclude them (--exclude-group skip-on-panel):
ArchitectureTest and StreamTokenCallSitesTest scan the repo src/ tree
(absent / polluted in a flat deploy; they also self-skip when it is
missing), and LoginSessionFixationTest runs in isolated child processes
that the panel's ionCube/OPcache PHP cannot reconstitute.
Verified green on all three backends: SQLite (local, 721), MariaDB 11.4
(container, 721), and the panel's bundled PHP 8.1 + server MariaDB
(713, with the group excluded).
Two regressions surfaced when running the suite (9 + 3 errors):
- setDb(DatabaseHandler) rejects the SQLite double: the DatabaseAware
refactor gave setDb() a strict DatabaseHandler hint, but TestDb was a
standalone class. Make TestDb extend DatabaseHandler (a real subtype) so
it satisfies the seam; its own constructor wires sqlite::memory: and never
calls the MySQL-connecting parent constructor. Overridden methods widen
parameter types (contravariant) and add return types, so LSP holds.
- validateHMAC(int|string $rExpiry) rejects null: HmacTokenTest passed null
for an intentionally-empty expiry. Pass '' — identical HMAC input, and the
strict type stays correct (the sole production caller always passes a value).
Suite is green again: 580 tests, 0 errors.
Move Core/Database into XcVm\Core\Database (DatabaseHandler, Database,
MigrationRunner, QueryHelper). First of the Core hub sub-layers.
- Namespace the 4 classes; DatabaseHandler extends Database (same namespace);
built-ins/ioncube qualified (\PDO, \PDOException, \Exception, \Throwable,
\XC_VM); Database keeps its 'use XcVm\Core\Logging\FileLogger;'.
- Add 'use XcVm\Core\Database\...;' to referencing files (DatabaseHandler 51,
Database 64, QueryHelper 29, MigrationRunner 3) + 2 test files (PHPStan does not
analyse tests/, so PHPUnit is the gate there).
- Rewrite pre-existing leading-backslash global refs (\DatabaseHandler etc., e.g.
in @param docblocks of ResellerApiDispatcher/ResellerTableRenderer) to the full
FQCN \XcVm\Core\Database\... — a 'use' import does not cover a leading-\
reference. Done with a lookbehind so FQCN continuations and use-lines are intact.
- phpstan-baseline.neon regenerated (292→292; pre-existing Database/migration_logic
findings re-anchored after class names in messages gained the namespace).
Verified: php -l clean; PHPStan no errors; PHPUnit 295/295.
Adds tests/Support/TestDb.php — a Database-compatible wrapper over
sqlite::memory: injected via setDb() — and StreamConfigRepositoryTest as a
worked example (SELECT/list/keyed/DELETE-cascade/not-found). 286 -> 292 tests.
