90f5d117c8 feat(events): let websocket clients declare channels on connect (#525)
* feat(events): let websocket clients declare channels on connect

Observing the events hub over `/api/v1/events/ws` costs more than it
should. A read-only consumer has to send a `subscribe` frame within five
seconds or be closed with a policy violation, which means implementing
the handshake and holding the write half of the socket open purely to
satisfy it. That cost is contract, not transport: `subscribe` is the only
inbound message this endpoint accepts.

Accept the selection on the URL instead. `?channels=catalog,user_state`
subscribes on connect, answers with the same `subscribed` frame and
per-channel snapshots the handshake produces, and is never put on the
grace-period clock. A connection that declares nothing is unchanged —
it still owes a subscribe frame within five seconds.

Two supporting changes:

- Channel selection now resolves through one shared function used by both
  paths, so the URL and handshake cannot drift on who may subscribe to
  what. Role, profile-binding, and validity checks are unchanged.
- An unrecognized channel name is reported in the existing `rejected`
  array as `unknown_channel` rather than closing the connection. Closing
  took down every other channel the client held over one bad name, and a
  client cannot always know which channels its role allows before asking.
  Forbidden and profile-scoped channels were already handled this way.

`required_action` in the hello frame is `"none"` for a declared
connection and `"subscribe"` otherwise; the web type is widened to match.
No wire field changes type or disappears, so this stays additive under
the v1 rules.

Part of #523

AI disclosure: tool Claude Code, model claude-opus-5, fully AI-generated,
reviewed and verified by the author before submission.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(events): address review findings on declared-channel subscriptions

Three findings from automated review, all verified against the code
before acting on them.

**Start the reader before declared-channel snapshots (regression).**
configureWebSocket installs an absolute read deadline that only pongs
extend, and gorilla processes pongs solely inside ReadMessage
(conn.go:950, reached only via advanceFrame). The declared path built its
snapshots before starting the reader goroutine, so a snapshot slower than
the deadline — a loaded jobs/sessions/scans/history query — would kill an
otherwise healthy connection the instant reading began. The handshake
path never had this problem because its snapshots run downstream of an
active reader. Regression test stalls a tasks snapshot past the deadline;
it fails with the previous ordering.

**Advertise the feature through a capability endpoint.** Adding a
client-visible subscription mode without one leaves a read-only client
unable to tell, before connecting, whether ?channels= will be honored: an
older server ignores it and closes the connection after the grace period,
so the client must retain the very handshake this removes. GET
/api/v1/events/capability reports both modes, the grace period the
handler actually enforces, and the known channels, following the existing
per-subsystem convention.

**Deduplicate rejections, not just acceptances.** Asking twice for one
forbidden channel produced two identical `rejected` entries. Pre-existing
— the dedup check sat after the rejection branches — but cheap to correct
in the function this PR extracted.

Part of #523

AI disclosure: tool Claude Code, model claude-opus-5, fully AI-generated,
reviewed and verified by the author before submission.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(events): frame the capability endpoint as a staleness probe

Clients are expected to run a current build rather than negotiate down to
an old server, so the endpoint is not a branch-on-capability contract.
Its value is letting a client distinguish "this server does not do that"
from "the connection failed" — the two are indistinguishable from the
socket alone, since an older server ignores ?channels= and then closes on
the grace period — so it can tell the user the deployment is out of date
instead of failing opaquely.

Comment-only; no behavior or wire change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(events): bound the subscribed answer and reap unsubscribed connections

Second review pass on the declared-channel path. Four issues, all at the
edges of the new URL surface rather than in the design itself.

Rejections amplified the request. Making an unknown channel non-fatal
removed the brake that used to close the connection on the first bad
name, and every refusal quotes the name it refuses — so a large
?channels= of distinct garbage produced a far larger `subscribed` frame,
buffered server-side. Cap a selection at 32 distinct channels, report the
overrun once instead of per name, truncate an echoed name at 64 bytes,
and set a 64 KiB read limit on the socket so an oversize frame cannot be
buffered whole before it is rejected.

The grace period was disarmed by declaring, not by subscribing. Both
`?channels=` with no names and a non-admin naming only an admin channel
came up subscribed to nothing and were never reaped, each holding a hub
subscriber, two goroutines, and an envelope channel that every published
event fans into. Disarm on holding a subscription instead. Selection now
resolves before the hello frame — it is pure, so nothing moves ahead of
the reader — which lets required_action say "subscribe" when the
connection really does still owe one.

Repeating the parameter dropped channels silently.
`?channels=a&channels=b` honored only the first and reported nothing
rejected. Read every occurrence.

The capability endpoint advertised `plugins`, which is host-to-plugin
runtime dispatch and is granted to no role. An admin following the
endpoint's stated purpose got `forbidden` while already being admin, and
the hardcoded "Admin access required" made it a dead end rather than a
soft failure. Split evt.ClientChannels out of AllChannels, advertise
that, and word the refusal so it does not promise a remedy that does not
exist. A test pins that an admin can subscribe to everything the
endpoint names.

Each guard was verified to bite by reverting it and watching the test
fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 10:59:50 -04:00
2026-07-25 18:37:51 +00:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-07-25 18:37:51 +00:00

Silo

Silo is a self-hosted media streaming server for your movies, shows, music, and books. Point it at your media folders and stream to your devices — at home or away — with direct play, remuxing, and hardware-accelerated transcoding handled automatically.

Join the community on Discord. If Silo is useful to you, consider sponsoring the project — see Supporting Silo.

Highlights

  • Plays your media, your way — direct play when the device supports it, remux or hardware-accelerated transcode (including NVENC) when it doesn't.
  • Web app included — a full-featured web client and admin interface ship with the server.
  • Works with apps you already use — optional Jellyfin/Emby-compatible API supports clients such as VidHub, Findroid, and Infuse.
  • Household profiles — multiple profiles per account, with per-profile watch state and parental controls.
  • Plugin-driven metadata — match and enrich your libraries with providers like TMDB and TVDB, installed as plugins.
  • Fast setup — one docker compose up -d brings up the whole stack; everything else is configured in the admin UI.

The easiest way to run Silo is with Docker Compose. The default stack assumes you do not already have PostgreSQL and Redis available, so it bundles PostgreSQL, Redis, FFmpeg, and the application for a one-command start.

  1. Create a .env file

    cp .env.example .env
    
  2. Set your media path

    Edit .env and set:

    MEDIA_ROOT=/path/to/your/media
    

    MEDIA_ROOT is the one value most users need to change. You can also override SILO_DATA_ROOT if you do not want bind mounts under /opt/silo, and change ports if the defaults conflict with something else on the host.

  3. Start the default integrated stack

    docker compose up -d
    

    This starts PostgreSQL, Redis, and the integrated Silo server. The app is available at http://localhost:8090. Jellyfin-compatible app support is disabled until an administrator enables it in onboarding or admin settings.

    If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing DATABASE_URL and REDIS_URL instead.

    Optional NVIDIA/NVENC

    GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.

    Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.

    Use the optional override file when you want NVENC:

    docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
    

    If you want this controlled from .env, set COMPOSE_FILE:

    COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
    NVIDIA_GPU_COUNT=1
    

    Windows uses ; instead of : between compose files.

    Then docker compose up -d will include the NVIDIA override automatically.

  4. Configure through the admin UI

    Add libraries, users, metadata providers, and playback settings from the web interface.

Bind Mount Layout

The deploy-oriented compose files use host folder mappings rather than Docker-managed volumes.

By default, data is stored under /opt/silo:

  • /opt/silo/postgres
  • /opt/silo/redis
  • /opt/silo/transcode
  • /opt/silo/catalog-seeds

Media is mounted into the container at /mnt/media from the host path you set in MEDIA_ROOT.

Optional Profiles

The main compose file is integrated-first. These profiles exist for operators testing distributed mode or mirroring a split deployment shape. Most single-host installs should stay on the default integrated service, because it already includes proxying and transcoding.

Profile Command Description
default docker compose up -d Integrated server plus bundled PostgreSQL and Redis
proxy docker compose --profile proxy up -d Start a standalone proxy service for distributed-mode testing
transcode docker compose --profile transcode up -d Start a standalone transcode service for distributed-mode testing

You can enable both optional examples together:

docker compose --profile proxy --profile transcode up -d

If you are splitting workers across multiple hosts, use the separate remote worker example instead of trying to stretch the main compose file across machines.

Advanced Remote Node Example

For a dedicated remote transcode worker, use docker-compose.remote-transcode.yml. That file is intended for a separate worker host that connects back to an existing Silo deployment using shared PostgreSQL and Redis.

Deployment Notes

The default compose stack intentionally bundles PostgreSQL and Redis for ease of setup and assumes a fresh install without those services already available. If you already operate PostgreSQL and Redis, omit those examples from compose and point Silo at your existing infrastructure instead. For serious installs, PostgreSQL is better on a separate VM or a managed service so upgrades, tuning, and backups are isolated from the app host. Redis can stay local for many installs, but externalizing it is also reasonable if you already operate shared infrastructure.

Silo is externally stateful by default rather than fully stateless. Durable application state lives in PostgreSQL. Redis only stores coordination and cache-style data. Silo still writes transient transcode output locally under /tmp/silo-transcode. If you switch userdb.backend=sqlite, Silo also becomes locally stateful at /var/lib/silo/userdb.

Migrating an existing Continuum Docker install should be done with the preflight helper and cutover guide in docs/continuum-to-silo-docker-migration.md.

Configuration

Silo requires only a DATABASE_URL when running from source or against external infrastructure. In the default Docker Compose path, the stack wires the database and Redis URLs for you. All other settings — libraries, metadata providers, transcoding, users — are managed through the admin UI after first launch.

Server Modes

Mode Description
integrated Full server: API + frontend + scanner + transcode (default)
api API server only, no local transcoding
proxy Stream proxy node that connects to the shared deployment database and Redis
transcode HLS transcode worker node that connects to the shared deployment database and Redis

PostgreSQL Auto-Tuning

The default Docker Compose stack does not require a checked-in postgresql.conf. It enables Silo's pgtune-style OLTP tuning by default:

POSTGRES_TUNE: auto

When enabled, Silo connects with DATABASE_URL and applies recommendations with ALTER SYSTEM, which writes to PostgreSQL's postgresql.auto.conf inside the database data directory. Reloadable settings are applied immediately with pg_reload_conf(). Settings that PostgreSQL marks as restart-only are written too, and Silo logs the setting names so you can restart PostgreSQL once:

docker compose restart postgres

The default Compose database user has the required PostgreSQL permissions. If you use an external PostgreSQL server, make sure the configured DATABASE_URL user can run ALTER SYSTEM, or set POSTGRES_TUNE=off and manage PostgreSQL yourself.

For POSTGRES_TUNE_MEMORY=auto, Silo uses the first trustworthy memory source: a finite Docker cgroup limit, the read-only /host/proc/meminfo mount supplied by the bundled Compose file, then /proc/meminfo with container safety guards. Auto-detected memory is treated as a PostgreSQL budget, defaulting to 75% of detected RAM so Silo, Redis, plugins, transcodes, and the OS retain headroom. POSTGRES_TUNE_DB_SIZE=auto queries pg_database_size(current_database()) and classifies the workload by comparing the database size to that memory budget.

Optional tuning overrides:

Variable Default Description
POSTGRES_TUNE_PROFILE oltp Tuning profile. Only oltp is currently supported.
POSTGRES_TUNE_MEMORY auto Server/container RAM, such as 8GB or 32GB; explicit values are used as-is.
POSTGRES_TUNE_MEMORY_BUDGET_PERCENT 75 Percent of auto-detected RAM used for PostgreSQL recommendations.
POSTGRES_TUNE_CPUS auto CPU count used for worker recommendations.
POSTGRES_TUNE_STORAGE ssd One of hdd, ssd, san, or nvme.
POSTGRES_TUNE_DB_SIZE auto Use less_ram when the database comfortably fits in RAM, mid_ram, or greater_ram for very large databases.
POSTGRES_TUNE_CONNECTIONS 100 PostgreSQL max_connections; automatically raised if Silo's app pool is configured higher.
POSTGRES_SHM_SIZE 8gb Docker /dev/shm size for the bundled PostgreSQL container.

Advanced operators can still supply their own PostgreSQL configuration or override these env vars. Set POSTGRES_TUNE=off when you do not want Silo to change PostgreSQL server settings. Settings already written with ALTER SYSTEM remain in postgresql.auto.conf; reset those PostgreSQL parameters if you later move fully to a custom postgresql.conf.

Build from Source

If you prefer running Silo without Docker:

  1. Install prerequisites: Go 1.24+, Bun 1.0+, PostgreSQL 18+, and FFmpeg.

  2. Start PostgreSQL and Redis (skip if you already have them running)

    docker compose up -d postgres redis
    

    The main compose file still expects MEDIA_ROOT to be set even if you only want the bundled PostgreSQL and Redis services, so set that in .env first.

  3. Configure the database connection

    cp .env.example .env
    

    Edit .env and set DATABASE_URL to point to your PostgreSQL instance.

  4. Build and run

    make build
    ./silo
    

    The server starts at http://localhost:8080 by default. All other settings are configured through the admin UI.

Reporting Issues

If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.

Please include:

  • What you were trying to do
  • Exact steps you took
  • What you expected to happen
  • What actually happened
  • What exact action is slow or broken (save, scan, browse, import, playback, etc.)
  • Whether it happens every time or only sometimes
  • The library, media type, filter, setting, or value involved
  • Version, branch, commit, and deployment details if you know them
  • Screenshots, recordings, or log snippets if relevant

If you used Claude/Codex for debugging, put that under Technical notes at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.

Use this template:

Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:

Contributing & Development

Silo is open source and contributions are welcome. See DEVELOPMENT.md for building from source in a dev workflow, running tests, database migrations, and project layout, and CONTRIBUTING.md for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.

Supporting Silo

Silo is an open-source hobby project, developed in spare time and funded out of pocket. If you'd like to support development, you can sponsor via GitHub Sponsors.

Donations go directly toward the costs of building and running the project:

  • AI development tooling subscriptions (Claude, Codex) used to build and maintain Silo
  • Push notification relay infrastructure
  • Future development costs

Sponsoring is entirely optional — Silo is and will remain free and open source. Bug reports, contributions, and feedback are just as valuable.

License & Trademarks

Silo's source code is licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) — see LICENSE.

The Silo name, logo, and wordmark are trademarks of Silo Media L.L.C. and are not covered by the AGPL. You're free to fork and redistribute the code, but forks and redistributions must not use the Silo brand as their identity and must remove or replace the brand assets. Publishing a Silo-branded app to an app store requires written permission. See TRADEMARK.md for what's permitted — including referential use like "compatible with Silo."

S
Description
Self-hosted media streaming server with a Go backend, React web UI, Docker deployment, transcoding, and Jellyfin-compatible APIs.
Readme
340 MiB
Languages
Go 72.6%
TypeScript 26.3%
PLpgSQL 0.4%
CSS 0.3%
Python 0.2%
Other 0.1%