XC_VM/docs/en/development/caching-and-redis.md

# Caching and Redis Strategy

XC_VM uses a dual-layer caching strategy:

- **File-based cache (igbinary)** — primary layer, used by both streaming and admin paths
- **Redis/KeyDB** — optional high-performance layer for connection state and advanced operations

The streaming path reads exclusively from file cache (no database queries).
The admin path reads from the database with optional short-lived cache.

---

## Cache Interface

File: `src/Core/Cache/CacheInterface.php`

```php
get($key, $maxAge = null)
set($key, $data, $ttl = 0)
delete($key)
has($key, $maxAge = null)
flush()
```

- `$ttl = 0` means cache forever (until manual deletion or flush).
- `$maxAge` checks file modification time for freshness (FileCache only).

---

## FileCache

File: `src/Core/Cache/FileCache.php`

Default cache implementation. Stores igbinary-serialized data as flat files.

```php
$cache = new FileCache(CACHE_TMP_PATH);
$cache->set('my_key', $data, 3600);
$data = $cache->get('my_key', 120);     // only if < 2 min old
```

Static convenience API (backward compatibility):

```php
FileCache::setCache($key, $data)
FileCache::getCache($key, $maxAge = null)
```

Characteristics:

- Serialization: igbinary (if available) or PHP serialize fallback.
- Locking: `LOCK_EX` on write to prevent corruption.
- File location: `{basePath}/{key}` (no subdirectories for core keys).
- Corruption recovery: detects bad data, auto-deletes corrupted files.

---

## RedisCache

File: `src/Core/Cache/RedisCache.php`

Optional high-performance implementation.

```php
$redis = new RedisCache('127.0.0.1', 6379, $password, 'prefix:');
$redis->set($key, $data, 600);          // 10-minute TTL via SETEX
$redis->getConnection();                 // raw phpredis for sorted sets, pipelines
```

- Lazy connection: connects on first operation.
- Native TTL support via Redis `SETEX`.
- Used primarily for `ConnectionTracker` (sorted sets for live connection state).

---

## Redis Connection Management

File: `src/Infrastructure/Redis/RedisManager.php`

Singleton lifecycle:

```php
RedisManager::instance()                 // get active Redis or null
RedisManager::ensureConnected()          // connect if not already
RedisManager::isConnected()              // health check
RedisManager::closeInstance()            // disconnect
```

Health check pings Redis every 30 seconds (debounced). Auto-reconnects on failure. Returns null on connection failure (graceful degradation).

Configuration:

| Setting | Source | Default |
| --- | --- | --- |
| `hostname` | `config.ini` | — |
| `port` | hardcoded | `6379` |
| `password` | `settings.redis_password` | — |
| `read_timeout` | hardcoded | `2.0s` |
| `tcp_keepalive` | hardcoded | `60s` |

### Surviving idle disconnects (long-lived daemons)

Short-lived requests (PHP-FPM stream/admin) open a fresh connection per process
and are unaffected by idle timeouts. Long-lived daemons — the `watchdog` loop,
`fanout_sync` — instead hold **one** connection through the singleton for their
whole lifetime, which exposes two failure modes on a busy or cross-server
(LB → MAIN) link:

- **Server idle-close.** Redis closes any client idle past its `timeout` (`300s`
  in the bundled `bin/redis/redis.conf`). phpredis then transparently re-opens
  the socket on the next command **without replaying AUTH**, so a later command
  answers `NOAUTH` — or simply returns `false`.
- **Debounced health-check gap.** `instance()` only pings every 30s, so between
  pings a dropped connection is not yet noticed.

Guards in place:

- `instance()` treats any non-`PONG` ping reply (the silent-reconnect / `NOAUTH`
  state) as a dead connection and forces a full, **re-authenticated** reconnect
  via `\XC_VM::redis_connect()` — not just a socket-level retry.
- Call sites that pipeline commands validate the pipeline object. For example
  `ConnectionTracker::getCapacity()` checks that `$redis->multi()` returned a
  `\Redis` (a broken socket returns `false`, and calling `zCard()` on that bool
  would fatal outside the reconnect path) and throws so its retry loop reconnects.

The server-side alternative (`timeout 0`) is deliberately **not** used — the
client is made resilient instead, and `tcp-keepalive` still reaps dead peers.

---

## Cache Population

Cache files are generated by two cron jobs:

### Lightweight cache (CacheCronJob)

Runs every cron cycle. Rebuilds fast-changing data (~1 second):

- `settings` — panel settings
- `servers` — server list
- `bouquets` — channel packages
- `categories` — stream categories
- Blocklists: `blocked_isp`, `blocked_ua`, `blocked_ips`, `blocked_servers`
- `allowed_ips`, `output_formats`, `hmac_keys`, `rtmp_ips`

### Heavy cache (CacheEngineCronJob)

Rebuilds stream, line, and series data. Throttled to once per 5 minutes via `heavy_cache_built` marker:

- `STREAMS_TMP_PATH/stream_{id}` — individual stream metadata
- `LINES_TMP_PATH/line_i_{user_id}` — user account data
- `LINES_TMP_PATH/line_c_{username_password}` — username → user_id lookup
- `LINES_TMP_PATH/line_t_{access_token}` — token → user_id lookup
- `SERIES_TMP_PATH/series_{id}` — series metadata

Change detection mode (if `cache_changes` enabled): compares DB `updated` timestamp vs file `mtime`, rebuilds only changed items.

Full rebuild mode: regenerates all entries. Controlled by `cache_thread_count` setting.

### Cache readiness

A `cache_complete` file is written after each full cache build. The streaming path checks for this file and exits with an error if missing.

### Cold-cache safety

The streaming bootstrap (`LegacyInitializer::initStreaming()`) reads `servers`,
the blocklists and `proxy_servers` from the file cache. Before the first build
(fresh boot, cleared tmp) those files do not exist and `CacheReader::get()`
returns `null`, so every such global is defaulted to an empty array. A cold cache
therefore **fails closed** — a request finds no servers and shows "not on air" —
instead of a `foreach(null)` warning or an `in_array($x, null)` fatal (PHP 8)
downstream. A genuinely broken cache *build* still surfaces separately via
`FileCache`'s write-failure warning, so this default masks only the transient
cold-start window, not a real failure.

---

## Cache Key Conventions

### System keys (CACHE_TMP_PATH)

| Key | Contents |
| --- | --- |
| `settings` | panel settings array |
| `servers` | `array[server_id]` → server config |
| `bouquets` | `array[bouquet_id]` → bouquet definition |
| `categories` | `array[category_id]` → category data |
| `bouquet_map` | `array[stream_id]` → `array[bouquet_id]` |
| `category_map` | `array[bouquet_id]` → `array[category_id]` |
| `permissions_{group_id}` | group permission set |
| `cache_complete` | `time()` timestamp of last full build |

### Stream keys (STREAMS_TMP_PATH)

| Key | Contents |
| --- | --- |
| `stream_{id}` | stream info + bouquets + per-server state |
| `channels_categories` | `array[stream_id]` → `array[category_id]` |

### Line keys (LINES_TMP_PATH)

| Key | Contents |
| --- | --- |
| `line_i_{user_id}` | full user record |
| `line_c_{username_password}` | user_id (credential lookup) |
| `line_t_{access_token}` | user_id (token lookup) |

### Series keys (SERIES_TMP_PATH)

| Key | Contents |
| --- | --- |
| `series_{id}` | series metadata |
| `series_map` | `array[stream_id]` → series_id |
| `episodes_{series_id}` | `array[season_num]` → episode list |

---

## Invalidation Patterns

| Trigger | Affected keys | Mechanism |
| --- | --- | --- |
| Admin edits stream | `stream_{id}`, `bouquet_map` | signal → next `cron:cache_engine` |
| Admin edits line | `line_i_*`, `line_c_*`, `line_t_*` | next `cron:cache_engine` |
| Settings changed | `settings`, categories, blocklists | `SettingsManager::clearCache()` + cron |
| Server list updated | `servers`, `bouquet_map` | cron |
| Stream start (FFprobe) | `{md5(source)}` | 5-minute TTL via file mtime check |
| Admin flush button | all files in `CACHE_TMP_PATH` | `rm -rf` |

---

## Streaming vs Admin Path

### Streaming path (`www/stream/*`)

- `cached: true` by default.
- Reads from file cache exclusively (no DB queries).
- Raw igbinary deserialization: `igbinary_unserialize(file_get_contents(...))`.
- If `cache_complete` missing: exit with error.

### Admin path (`Public/Controllers/Admin/*`)

- `cached: false` by default.
- Reads from database directly via domain services.
- Optional short-lived cache (example from `BouquetService::getAll()`):

```php
$rCache = FileCache::getCache('bouquets', 60);  // only if < 60s old
if (!empty($rCache)) {
    return $rCache;
}
// miss: query database and write cache
FileCache::setCache('bouquets', $rOutput);
```

---

## Cache File Layout

```text
/home/xc_vm/tmp/cache/
├── settings
├── servers
├── bouquets
├── categories
├── bouquet_map
├── category_map
├── cache_complete
├── heavy_cache_built
├── streams/
│   ├── stream_{id}
│   └── channels_categories
├── lines/
│   ├── line_i_{user_id}
│   ├── line_c_{username_password}
│   └── line_t_{access_token}
└── series/
    ├── series_{id}
    ├── series_map
    └── episodes_{series_id}
```

---

## Related files

| File | Purpose |
| --- | --- |
| `src/Core/Cache/CacheInterface.php` | cache contract |
| `src/Core/Cache/FileCache.php` | file-based cache implementation |
| `src/Core/Cache/RedisCache.php` | Redis cache implementation |
| `src/Infrastructure/Redis/RedisManager.php` | Redis connection singleton |
| `src/Infrastructure/Cache/CacheReader.php` | legacy cache reader bridge |
| `src/Cli/CronJobs/CacheCronJob.php` | lightweight cache generation |
| `src/Cli/CronJobs/CacheEngineCronJob.php` | heavy cache generation (streams, lines, series) |
| `src/Domain/Bouquet/BouquetService.php` | example of admin-path caching |
| `src/Domain/Stream/ConnectionTracker.php` | Redis sorted sets for connection state |