# 🌊 Reverse Proxy (Streaming, Caching & Rate Limits) This section documents the `reverse_proxy:` block inside `config.yml`. It is the most critical block for determining runtime behavior when Tuliprox actively proxies video streams to clients (Reverse Proxy Mode), rather than just redirecting them. It manages how Tuliprox establishes upstream connections, buffers video frames, handles sudden client disconnects, and caches static resources like EPG images and channel logos. It also contains the optional telemetry pipeline for stream reliability analysis: * `stream_history` stores raw stream lifecycle events on disk. * `qos_aggregation` periodically condenses those raw events into compact QoS snapshots. * The Web UI can then inspect both the raw history view and the aggregated QoS summaries. ## Read This First: Connection Handling Handbook The `reverse_proxy.stream` block now interacts with a broader connection-handling model: * user limits * soft connections * priorities * admission strategies * grace periods * HLS and catchup sessions * shared streams * provider-side preemption If you want to understand why a stream starts, waits, gets reused, gets preempted, or is rejected, read these pages first: * [Connection Handling](./connection-handling.md) * [Connection Handling: Priorities, Soft Connections and Preemption](./connection-handling-priorities-and-preemption.md) * [Connection Handling: Sessions, HLS, Catchup and Reconnects](./connection-handling-sessions-and-reconnects.md) * [Connection Handling: Failures and User-Visible Behavior](./connection-handling-failures-and-user-visible-behavior.md) ## Top-level entries ```yaml reverse_proxy: resource_rewrite_disabled: false rewrite_secret: A1B2C3D4E5F60718293A4B5C6D7E8F90 stream: cache: hls_cache: rate_limit: disabled_header: resource_retry: geoip: stream_history: qos_aggregation: ``` > **Note:** Reverse Proxy mode can be activated for each user individually. ### General Parameters | Parameter | Type | Default | Technical Impact & Background | | :--- | :--- | :--- | :--- | | `resource_rewrite_disabled` | Bool | `false` | Normally, Tuliprox rewrites all image URLs in playlists to point to itself (e.g., `http://tuliprox:8901/resource/...`). If set to `true`, original URLs are kept (clients load images directly from the provider). **Warning:** Local caching will stop working if this is enabled! | | `rewrite_secret` | String | `""` | A 32-character Hex string (16 bytes). Tuliprox encrypts/signs the original image URLs during the rewrite process. To prevent image URLs from becoming invalid after a server restart, you MUST enter a static secret here. | | `stream_history` | Block | `null` | Optional stream telemetry block that persists raw connect/disconnect/startup-failure events to daily history files. | | `qos_aggregation` | Block | `null` | Optional background worker that aggregates stream history into compact per-stream QoS snapshots. | > **Note:** You can generate a random secret using: ```bash openssl rand -hex 16 # or node -e "console.log(require('crypto').randomBytes(16).toString('hex').toUpperCase())" ``` --- ## 1. Stream Management (`stream`) This sub-block defines how Tuliprox maintains stream stability, buffers data, and handles HLS/Catchup session affinity. Admission strategies in this block can optionally evict an existing stream after normal and soft user admission are already exhausted: * `evict_user_same_ip_oldest` * `evict_user_same_ip_latest` * `evict_user_oldest` * `evict_user_latest` * `grace_instant_stream` * `grace_hold_stream` Order matters. If you want same-IP eviction to be preferred, place the same-IP rule before the broader user-wide rule with the same oldest/latest policy. Examples: * valid: `evict_user_same_ip_oldest`, then `evict_user_oldest` * invalid: `evict_user_oldest`, then `evict_user_same_ip_oldest` * valid: `evict_user_same_ip_latest`, then `evict_user_latest` * invalid: `evict_user_latest`, then `evict_user_same_ip_latest` ```yaml reverse_proxy: stream: retry: true buffer: enabled: true size: 1024 throttle_kbps: 12500 grace_period_millis: 2000 grace_period_timeout_secs: 4 grace_period_hold_stream: true hls_session_ttl_secs: 15 catchup_session_ttl_secs: 45 shared_burst_buffer_mb: 12 cleanup_queue_capacity: 4096 recent_eviction_reentry_ttl_ms: 3000 metrics_enabled: false ``` ### Stream Parameters in Detail | Parameter | Type | Default | Technical Impact & Background | | :--- | :--- | :--- | :--- | | `retry` | Bool | `true` | Retries connecting to the upstream provider during the initial stream open when the provider returns a transient failure or no usable stream. Once a stream has started, Tuliprox does not transparently replace that live upstream inside the same client response. | | `buffer.enabled` | Bool | `false` | Enables an asynchronous ring-buffer in RAM between the provider download stream and the client upload stream. Necessary if the provider stream is faster than the consumer can process. | | `buffer.size` | Int | `0` | The size of the buffer in *Chunks* (1 Chunk = 8192 Bytes). A value of `1024` equals approximately 8 Megabytes of RAM per active stream. | | `buffer.max_bytes_mb` | Int | `5` | Byte-level backpressure cap of the buffer in Megabytes. The producer stops reading from the provider once this many bytes are queued for the client, regardless of chunk count. Increase for high-bitrate streams with slow consumers; decrease on memory-constrained hosts. | | `throttle_kbps` | Int | `0` | **Background:** Some players download VODs (Movies) at maximum line speed ("Bursting"). Providers often view this as abuse or scraping and will ban the IP. By throttling (e.g., to `12500` kbps), you force the download into a constant, inconspicuous flow. Supports units like `KB/s`, `MB/s`, `kbps`, `Mibps`. | | `metrics_enabled` | Bool | `false` | **Monitoring:** If active, Tuliprox samples the live bandwidth (in kbps) and transferred bytes for every active reverse-proxied stream and pushes them via WebSockets to the Web UI. It adds a tiny bit of CPU overhead but is invaluable for debugging buffering issues. | | `grace_period_millis` | Int | `2000` | The exact time window in ms where a temporary over-allocation is allowed (see notes on [The VLC Seek Problem](#the-vlc-seek-problem--grace-periods) for details). | | `grace_period_timeout_secs` | Int | `4` | A hard timeout limit for overlapping "ghost sessions" to expire. | | `shared_subscriber_idle_timeout_secs` | Int | `300` | How long a subscriber of a shared (multi-client) stream may consume no data before it is dropped. Lower values reclaim slots from stalled clients faster; higher values tolerate longer player pauses. | | `grace_period_hold_stream` | Bool | `true` | Tuliprox artificially holds back video data to the client, waiting for grace check to finish, so it doesn't trigger provider prematurely. | | `hls_session_ttl_secs` | Int | `15` | Keeps virtual provider slot open between HLS segment (`.ts`) requests to prevent provider bans for "Account Hopping". | | `catchup_session_ttl_secs` | Int | `45` | Same session-holding principle applied to Archive/Catchup TV. See notes on section [Session TTLs for HLS & Catchup](#session-ttls-for-hls-m3u8--catchup) for details. | | `shared_burst_buffer_mb` | Int | `12` | Minimum burst buffer size (in MB) used for shared live streams to immediately synchronize new clients without Keyframe dropouts. See notes on section [Shared Live Streams](#shared-live-streams) for details. | | `cleanup_queue_capacity` | Int | `4096` | Maximum number of concurrent cleanup permits available to active response bodies and shared subscribers. Must be at least `1`. If all permits remain held, new stream admission waits for a bounded interval and then returns `503 Service Unavailable` instead of growing memory without limit. | | `recent_eviction_reentry_ttl_ms` | Int | `3000` | Time window after an eviction during which a retry of the evicted playback must not evict its replacement. The guard is scoped to the user, client address and channel, so unrelated clients are not blocked. Suppressed retries end quietly (no `user_connections_exhausted` video and no `ConnectionDenied` event), since they are not a real connection-limit refusal. Raise this for players with slow automatic retries; `0` disables the guard. The cumulative suppression count is exposed as `reentry_suppressed_total` in the server status (`GET /api/v1/status`). | ### 1.1 `retry` & `buffer` (Deep Dive) Tuliprox handles streams differently based on these settings: * **Option A:** Both `retry: false` and `buffer.enabled: false` âž” The provider stream is piped directly to the client with minimal overhead. * **Option B:** `retry: true` retries transient failures while opening the provider stream. `buffer.enabled: true` adds buffered streaming with a higher memory footprint. Stream-type provider behavior: * Plain TS live is socket/request-oriented. A new TS reconnect is a new provider allocation and may use another provider account if the selected account is unavailable. * Initial stream-open retries may also rotate configured provider URLs/accounts when failover is enabled. * HLS, DASH, VOD, series, and catchup are provider-affine after a session exists. Follow-up segment, seek, range, or reopen requests must stay on the provider account pinned in that session; if that account is unavailable, Tuliprox fails the follow-up instead of silently migrating it. * `retry: false` disables stream-open retry/failover for stream requests. * **DASH delivery scope:** DASH streams (`LiveDash`, `.mpd`) are delivered exclusively via HTTP redirect to the upstream provider. Tuliprox does not reverse-proxy or cache DASH segments. Even when reverse-proxy mode is active, DASH requests are not routed into the HLS proxy pipeline. DASH sessions retain provider-account affinity during reopen/refresh within `hls_session_ttl_secs`. #### Ring-Buffer Calculation `buffer.size` is defined in chunks of **8192 bytes (8 KB)**. * A size of `1024` equals approx. **8 MB** of RAM per active stream. * **Shared Streams Impact:** If `share_live_streams.mpeg_ts` is enabled, each channel consumes at least **12 MB** regardless of client count. Increasing `size` above 1024 (e.g., 2048) increases this to **24 MB** per shared channel. ### 1.2 `throttle_kbps` Prevents provider bans by limiting "bursting" players. Supported units: `KB/s`, `MB/s`, `KiB/s`, `MiB/s`, `kbps`, `mbps`, `Mibps`. **Reference Table for Throttling:** | Resolution | Framerate | Bitrate (kbps) | Quality | | :--- | :--- | :--- | :--- | | 480p (854x480) | 30 fps | 819 – 2,457 | Low-Quality | | 720p (1280x720) | 30 fps | 2,457 – 5,737 | HD-Streams | | 1080p (1920x1080) | 30 fps | 5,737 – 12,288 | Full-HD | | 4K (3840x2160) | 30 fps | 20,480 – 49,152 | Ultra-HD | ### 1.3 `grace_period` (The VLC Seek Problem) If `max_connections` is > 0, seeking can trigger a 509/401 error because the old connection isn't closed yet. * `grace_period_millis` (Default: `2000`): Grants a temporary over-allocation during switchover. * `grace_period_timeout_secs` (Default: `4`): How long a grace grant lasts before a new one can be made. * `grace_period_hold_stream`: If `true`, Tuliprox waits for the check to complete before sending data, preventing player timeouts on "exhausted" switches. ### 1.4 HLS & Catchup Session TTLs HLS/Catchup clients connect and disconnect repeatedly. Tuliprox uses a **Virtual Reservation** to maintain account affinity: * **HLS (`15s`):** The real provider slot is only held during active requests. The reservation keeps the provider account stable between segment fetches. * **Catchup (`45s`):** Keeps the reservation alive during seeking and reconnects. * **Note:** Channel switches from the same client immediately take over the reservation, bypassing the TTL. Important boundary: * These TTLs are for HLS/catchup style session continuity. * Plain TS live playback is socket-bound. A new TS socket is a separate playback attempt. * VOD, series and local playback use logical session identities so size, range, seek and reopen requests can continue the same playback across sockets. A genuinely separate playback still consumes another connection or soft slot. * A provider reservation becomes capacity-relevant only after Tuliprox forwards real media bytes. Manifests, HEAD requests, keys, maps and error responses do not confirm a provider slot lease. ### 1.5 Cleanup admission and overload behavior Every admitted streaming response must own a terminal cleanup permit before it can publish user or provider state. The permit follows the response body and releases the exact request when the body completes, is dropped without being polled, or fails while streaming. Shared subscribers use the same ownership rule without reserving a second permit for the same body. `cleanup_queue_capacity` bounds how many of these cleanup owners may exist at once. If the queue remains saturated for the admission deadline, Tuliprox rejects the new request with `503 Service Unavailable`; it does not admit an unprotected stream. HLS origin-control cleanup has reserved capacity so ordinary body churn cannot prevent a provider handle from being released. Changing the setting requires a server restart. --- ## 2. Resource Caching (`cache`) Tuliprox maintains a local disk cache for channel logos, posters, and EPG images. This reduces the load on provider servers and significantly speeds up playlist loading times for your clients. ```yaml reverse_proxy: cache: enabled: true size: 1GB directory: ./cache ``` ### Cache Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `enabled` | Bool | `false` | Global switch for resource caching. **Note:** Requires `resource_rewrite_disabled: false`. | | `size` | Size | `1GB` | Maximum disk space allocation. Supported units: `KB`, `MB`, `GB`, `TB`. | | `directory` | String | `./cache` | Storage location. Relative paths are resolved against the `storage_dir`. | ### Technical Background * **LRU Logic:** Operates as a **Least Recently Used** cache. When the `size` limit is reached, Tuliprox automatically evicts the oldest/least accessed images to make room for new content. * **Encryption:** Cached resources are indexed using a hash derived from the `rewrite_secret`. If the secret changes, the old cache becomes orphaned. * **Client Delivery:** Instead of the client downloading directly from the provider, Tuliprox serves the local file, acting as a high-speed CDN for your media metadata. --- ## 2.1 HLS Cache (`hls_cache`) This block configures the Live HLS cache proxy. It only defines operating parameters. For an operator-friendly explanation, start with [Shared HLS Sessions](./shared-hls-sessions.md). For the configuration reference, see [Shared HLS Configuration](./shared-hls-configuration.md). For the shared session, access lease, and transient delivery state machines, see [HLS Cache State Machines](./hls-cache-state-machine.md). ```yaml reverse_proxy: hls_cache: cache_path: "/tmp/tuliprox/cache/hls" strip: mode: "segments" value: 0 cache_duration: 300 cache_bytes: "10GB" cache_bytes_per_session: "512MB" max_segments_prefetch: 6 max_concurrent_segment_fetches_per_session: 2 max_concurrent_segment_fetches_global: 64 origin_manifest_timeout_ms: 3000 origin_segment_timeout_ms: 10000 session_idle_timeout: 300 segment_repair: max_level: "off" apply_to_first_segments: 1 max_parallel_repairs: 1 postprocess_timeout_ms: 2000 corrupt_segment_watchdog: mode: "off" max_parallel_jobs: 1 ``` ### HLS Cache Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `cache_path` | Path | `/tmp/tuliprox/cache/hls` | Root directory for future HLS segment and MAP cache objects. | | `strip.mode` | String | `segments` | Interprets `strip.value` as either a segment count (`segments`) or accumulated `#EXTINF` duration (`seconds`). | | `strip.value` | Int | `0` | Initial tail holdback for the first rendered HLS view. | | `cache_duration` | Seconds | `300` | Retention baseline for unprotected HLS cache objects. | | `cache_bytes` | Byte size | `10GB` | Global HLS cache byte budget. | | `cache_bytes_per_session` | Byte size | `512MB` | Per-session HLS cache byte budget. | | `max_segments_prefetch` | Int | `6` | Maximum session-local segment prefetch queue depth. | | `max_concurrent_segment_fetches_per_session` | Int | `2` | Maximum concurrent future segment fetches for one HLS session. | | `max_concurrent_segment_fetches_global` | Int | `64` | Maximum concurrent future segment fetches across all HLS sessions. | | `origin_manifest_timeout_ms` | Milliseconds | `3000` | Timeout for future Origin manifest fetches. | | `origin_segment_timeout_ms` | Milliseconds | `10000` | Timeout for future Origin segment fetches. | | `initial_manifest_wait_timeout_secs` | Seconds | `90` | How long a client may wait for the initial manifest decision (session bootstrap window). Lower values fail unhealthy sessions faster; higher values tolerate slow providers. | | `session_idle_timeout` | Seconds | `300` | Idle timeout before a future HLS cache session may be collected. | | `segment_repair.max_level` | String | `off` | Maximum repair level allowed by the codec-aware MPEG-TS segment repair policy (`off`, `low`, `medium`, `high`). | | `segment_repair.apply_to_first_segments` | Int | `1` | Number of visible TS objects checked per access-lease activation. | | `segment_repair.max_parallel_repairs` | Int | `1` | Maximum concurrent repair jobs. Must not exceed `max_segments_prefetch` when repair is enabled. | | `segment_repair.postprocess_timeout_ms` | Milliseconds | `2000` | Shared timeout for the complete segment post-processing chain, including repair and watchdog work. | | `segment_repair.corrupt_segment_watchdog.mode` | String | `off` | Optional watchdog for residual TS packet-corrupt warnings after regular repair (`off`, `detect_only`, `sanitize`, `diagnostic`). | | `segment_repair.corrupt_segment_watchdog.max_parallel_jobs` | Int | `1` | Maximum concurrent watchdog sanitize jobs. | Supported byte-size units: * `B` * `KB`, `MB`, `GB`, `TB` as decimal 1000-based units * `KiB`, `MiB`, `GiB`, `TiB` as binary 1024-based units * no suffix means bytes Important boundaries: * `reverse_proxy.hls_cache` only prepares the global cache engine. A target must also set `options.share_live_streams.hls: true` in `source.yml` before generated HLS live entries use the shared HLS path. * HLS cache retry behavior is fixed internally and is not user-configurable. * `reverse_proxy.rewrite_secret` must stay stable. It is used for future HLS `proxy_session_id` values and transient resource IDs. * `session_idle_timeout` controls HLS cache access-lease validity and idle cleanup. It is independent from `reverse_proxy.stream.hls_session_ttl_secs`, which belongs to the non-cache HLS request continuity path. * This block does not enable legacy resource caching; image/logo/EPG caching remains controlled by `reverse_proxy.cache`. --- ## 3. Rate Limiting (`rate_limit`) This block implements an IP-based **Token-Bucket** rate limiter. It protects your Tuliprox instance and upstream providers from DDoS attacks, malfunctioning scrapers, or aggressive players by restricting the frequency of incoming HTTP requests. ```yaml reverse_proxy: rate_limit: enabled: true period_millis: 500 burst_size: 10 ``` ### Rate Limit Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `enabled` | Bool | `false` | Global switch for the rate limiting engine. | | `period_millis` | Int | `500` | The refill rate. Defines how many milliseconds it takes to replenish exactly one request token. | | `burst_size` | Int | `10` | The bucket capacity. Allows a client to send this many requests instantly before the rate limit kicks in. | ### Technical Background * **Mechanism:** A client starts with a full "bucket" of `burst_size` tokens. Every request consumes one token. Once empty, the client must wait `period_millis` for a new token to be generated. * **IP-Detection:** Tuliprox identifies clients by their IP address. If you are running Tuliprox behind a proxy (Nginx, Traefik), ensure headers like `X-Forwarded-For` or `X-Real-IP` are passed correctly. * **Behavior:** When a client exceeds the limit, Tuliprox returns an `HTTP 429 (Too Many Requests)` status, protecting your CPU and provider bandwidth. --- This block controls which HTTP headers Tuliprox removes before forwarding a client request to the upstream provider. Stripping these headers is essential to prevent the provider from detecting proxy usage, internal IP addresses, or specific player fingerprints. ```yaml reverse_proxy: disabled_header: referer_header: true # Removes 'Referer' (prevents leaking your Tuliprox URL/Domain) x_header: true # Removes all 'X-*' headers (e.g., X-Forwarded-For, X-Real-IP) cloudflare_header: true # Removes 'CF-*' headers (hides Cloudflare origin details) custom_header: # List of additional specific headers to be dropped - "X-Powered-By" - "my-custom-tracker" ``` ### Header Stripping Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `referer_header` | Bool | `false` | Suppresses the source of the request. Prevents your Tuliprox instance from being logged at the provider. | | `x_header` | Bool | `false` | **Critical:** Wildcard-strips all headers starting with `X-`. These are the most common indicators used for proxy detection. | | `cloudflare_header` | Bool | `false` | Removes Cloudflare-specific headers. Essential if Tuliprox itself is running behind a Cloudflare proxy. | | `custom_header` | List | `[]` | Manual blacklist for headers not covered by the automatic toggles above (e.g., vendor-specific trackers). | > **Security Note:** Many providers analyze headers for "restreaming" patterns. > Combining `x_header: true` with a neutral `user_agent` in your `source.yml` provides the best protection against account flags. ## 5. Resource Retries (`resource_retry`) This block defines the retry behavior when Tuliprox proxies static resources (logos, EPG images). It ensures transient network issues or temporary provider timeouts don't result in broken images for your clients. ```yaml reverse_proxy: resource_retry: max_attempts: 3 backoff_millis: 250 backoff_multiplier: 1.5 failover_redirect_patterns: - "service-abuse" ``` ### Resource Retry Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `max_attempts` | Int (u8) | `3` | Maximum number of download tries before giving up. Minimum is `1`. | | `backoff_millis` | Duration | `250` | Initial wait time (ms) after the first failure. | | `backoff_multiplier` | Float | `1.5` | Factor by which the delay grows. `> 1.0` creates an exponential backoff. | | `failover_redirect_patterns` | List | `[]` | Regex patterns to identify "Abuse" or "Blocked" redirect URLs from providers. | ### Technical Background * **Exponential Backoff:** The wait time for each attempt is calculated as: $$ \text{delay} = \text{backoff\_millis} \times (\text{backoff\_multiplier}^{\text{attempt}-1}) $$ * **Failover Logic:** Providers sometimes redirect blocked requests (HTTP 302) to a "service-abuse.png" image. If a redirect URL matches a pattern in `failover_redirect_patterns`, Tuliprox treats it as a hard failure and triggers a retry instead of serving the abuse image. * **Smart Handling:** If an upstream server sends a `Retry-After` header, Tuliprox prioritizes that value over the local backoff calculation. --- ## 6. GeoIP Resolution (`geoip`) This block enables local IP-to-Country mapping. It allows the Tuliprox Web UI to display country flags in the **"Active Streams"** tab, providing immediate visual feedback on client locations. ```yaml reverse_proxy: geoip: enabled: true url: "https://raw.githubusercontent.com/sapics/ip-location-db/refs/heads/main/asn-country/asn-country-ipv4.csv" ``` ### GeoIP Parameter Details | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `enabled` | Bool | `false` | Global switch for GeoIP resolution. | | `url` | String | *(Optional)* | Source URL for the GeoIP CSV database. | ### Technical Background * **Data Format:** Tuliprox requires a CSV format with exactly three columns: `range_start, range_end, country_code` *Example:* `1.0.0.0, 1.0.0.255, AU` * **Performance:** The database is loaded into a high-speed memory-mapped structure to ensure that resolving client locations adds zero latency to stream processing. * **Automation:** To keep the data accurate, use the `GeoIpUpdate` task type within the `schedules` block. This periodically downloads and rebuilds the local binary lookup file. * **Privacy:** All resolution happens locally on your server; no client IPs are ever sent to external third-party APIs for location lookups. The CSV file must have exactly 3 columns: `range_start,range_end,country_code`. (The DB is periodically updated via the `schedules` block using the `GeoIpUpdate` task type). --- ## 7. Stream History (`stream_history`) This block enables persistent stream lifecycle telemetry. Tuliprox writes daily binary history files containing successful connects, startup failures, disconnect reasons, provider-open failures, reconnect counts, latency signals, and stable stream identity fields. When the same channel exists multiple times across different providers, stream history gives Tuliprox the raw event-level view for each variant. It records which variant failed, how it failed, how often it reconnects, and whether the problem happened during startup or while streaming. **User impact:** * Gives you a reliable answer to questions like "Which stream failed?", "Was this a provider abort or a capacity problem?", and "How often does this stream reconnect?" * Builds the raw data foundation for later QoS analysis and failover optimization. * Adds asynchronous background disk writes, but does not put synchronous work into the live streaming hotpath. ```yaml reverse_proxy: stream_history: stream_history_enabled: true stream_history_batch_size: 128 stream_history_retention_days: 30 stream_history_directory: ./stream_history ``` ### Stream History Parameters | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `stream_history_enabled` | Bool | `false` | Master switch for stream telemetry. If disabled, no stream history files are written. | | `stream_history_batch_size` | Int | `128` | Number of records buffered before the writer flushes a block to disk. Higher values reduce write frequency; lower values make new records visible sooner. | | `stream_history_retention_days` | Int | `14` | Number of UTC day partitions retained before old history files are deleted. | | `stream_history_directory` | String | `stream_history` under `storage_dir` | Directory for daily stream history files. Relative paths are resolved against `storage_dir`. | ### Technical Background * Stream history records are written asynchronously in append-friendly daily files. * Recorded event types include: * successful connect * startup failure (`connect_failed`) * disconnect * Recorded QoS metadata includes: * disconnect reason * startup failure reason * failure stage * provider error class / HTTP status * reconnect counts * first-byte latency * stream identity fields for later aggregation --- ## 8. QoS Aggregation (`qos_aggregation`) This block enables the periodic QoS aggregation worker. It reads stream history in the background and builds compact per-stream QoS snapshots that summarize recent reliability. In practice, QoS turns raw stream history into a compact health view for each channel variant. This makes it possible to compare equivalent variants of the same channel across providers, rank them by reliability, and use that information for channel selection and failover decisions. **User impact:** * Lets you see a condensed reliability view without manually scanning raw history files. * Prepares the data that the later failover feature can use to prioritize the most reliable stream/provider candidates. * Keeps heavy analysis work out of request handling and out of the streaming hotpath. ```yaml reverse_proxy: stream_history: stream_history_enabled: true qos_aggregation: enabled: true interval_secs: 300 compaction_interval_secs: 86400 ``` ### QoS Aggregation Parameters | Parameter | Type | Default | Technical Impact | | :--- | :--- | :--- | :--- | | `enabled` | Bool | `false` | Enables the background QoS worker. Only effective if `stream_history.stream_history_enabled` is also `true`. | | `interval_secs` | Int | `300` | Polling interval for the aggregation loop. Lower values update snapshots faster but increase background disk and CPU work. | | `compaction_interval_secs` | Int | `86400` | Rebuilds `qos_snapshot.db` at this interval to reclaim B+Tree storage from expired snapshots. Set to `0` to disable automatic compaction; this does not change the rolling 30-day QoS summaries. | ### Technical Background * The worker reads raw stream history by UTC day and stores snapshots in dedicated B+Tree databases. * Snapshots are keyed by stable stream identity and contain rolling `24h`, `7d`, and `30d` windows. * QoS data is persisted locally, so it survives restarts and can later be consumed by the failover feature without rescanning all history every time. * If stream history is disabled, QoS aggregation is automatically disabled during config preparation. ---   ## Additional Information ### Session TTLs for HLS (`.m3u8`) & Catchup HLS streams do not consist of an endless TCP pipe. Instead, the player downloads small `.ts` segments every few seconds (e.g., `seg1.ts`, `seg2.ts`). If Tuliprox selected a different provider account for every segment, providers could treat the traffic as account hopping. Tuliprox therefore keeps a logical, confirmed slot lease for the playback: * `hls_session_ttl_secs: 15`: After Tuliprox has forwarded actual media bytes, a cleanly finished HLS request may keep its provider account reserved for this playback for up to 15 seconds. A manifest-only or abandoned start does not create a capacity reservation. * Archive/Catchup playback uses the same confirmed-lease principle with `catchup_session_ttl_secs: 45`. * Provider errors, preemption, kicks and timeouts release the lease immediately rather than keeping the reconnect window. --- ### Archive Proxy Behavior & Live Playback Invariant When Tuliprox operates as a reverse proxy, it can securely proxy upstream archive/catchup requests. * **Live Playback Invariant:** A standard live stream request (e.g., `/m3u-stream/live/user/password/1234.m3u8`) **always** starts current live playback. Normal live requests never activate archive mode, even if arbitrary query parameters are added by the player. * Archive mode is only activated through a dedicated catchup endpoint (such as the generated Tuliprox catchup URLs) or a signed marker generated by Tuliprox itself. * Live and archive requests are completely isolated and never share the same session key. --- ### Shared Live Streams Tuliprox can share a live stream (`share_live_streams.mpeg_ts: true` in the target options of `source.yml`). If 5 users watch the same Live-TV channel, Tuliprox pulls the stream only 1x from the provider and multicasts the bytes locally to 5 clients. To ensure a user who tunes in 10 seconds later doesn't get player errors due to missing I-Frames/Keyframes, Tuliprox continuously keeps the last X Megabytes (`shared_burst_buffer_mb`, default `12`) in RAM. It fires this burst buffer at new subscribers so their decoders can instantly synchronize. Each viewer has a distinct subscriber identity even when several viewers arrive through the same reverse-proxy socket. The per-subscriber delivery queue is bounded by both chunk count and retained bytes. A client that stops making progress is disconnected after `shared_subscriber_idle_timeout_secs`; it cannot indefinitely retain the burst buffer or stall other viewers. If initial burst replay cannot complete, that subscriber ends instead of silently skipping missing data and continuing with the live tail. --- ### The "VLC Seek Problem" & Grace Periods When a user fast-forwards or rewinds a VOD, the player calculates the new byte offset, drops the old TCP connection, and immediately fires a new HTTP GET request (with a `Range` header) to Tuliprox. **The Problem:** It takes milliseconds to seconds for the upstream provider to realize the old connection is dead. If you have a `max_connections: 1` limit at the provider, they will view this new seek-request as a *second concurrent stream* and reject it with an HTTP 509 (Bandwidth Exceeded) or HTTP 401 error. **The Tuliprox Solution:** * `grace_period_millis: 2000`: Tuliprox grants the user a temporary over-allocation (Grace) for exactly this duration. * `grace_period_hold_stream: true`: Tuliprox artificially holds back the video data to the client, waiting for the grace check to finish, so it doesn't trigger the provider prematurely. * After the milliseconds expire, Tuliprox checks internally: Is the old connection truly gone now?
If Yes âž” Data flows.
If No âž” The new connection is hard-killed (serving the `user_connections_exhausted.ts` video) because the user is actually illegally watching twice. * `grace_period_timeout_secs: 4`: A hard timeout limit for overlapping "ghost sessions" to expire.