edd919c5f7 fix(notifications): restore store capabilities and batch series resolution (#647)
* fix(notifications): restore store capabilities and batch series resolution

Marking or unmarking a large series spent minutes in the interest-tracking
layer. Two independent defects, both in internal/notifications.

The decorator embeds the userstore.UserStore *interface*, which promotes only
that interface's methods. Every optional capability the backing store
implements was therefore invisible through the wrapper, and because callers
reach these by type assertion with a working fallback, the loss was silent:
no error, no test failure, just the slow path. cmd/silo wraps the provider
unconditionally when notifications are enabled, so in production
userstore.MarkWatchedBatch's assertion failed and #645's transactional batch
write never ran. AddVisibleHistory, VisibleHistoryTimestamps, and the
jellycompat series rollup were degraded the same way.

Forward all four capabilities explicitly, and add compile-time assertions so a
future capability is a build error rather than a silent slowdown.

Separately, the interest flush resolved each queued item to its series with one
query apiece, then deduped the results. A whole-series mark queues one mutation
per episode, so thousands of lookups collapsed to a single series after paying
for all of them. Resolve the batch in one query and dedupe from that; the
single-item resolveSeriesID had no other callers and is removed rather than
left to drift.

Measured on the dev server against a 6,375-episode series:

  mark    5.2s  -> ~0.9s
  unmark  116s  -> ~8.6s
  episodes index scans  18.5M -> 19,298

Fixes #646.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(notifications): queue interest recomputes on the batch fallback path

Review catch (CodeRabbit on #647). When the backing store lacks
WatchedBatchWriter, the forward handed the work to the generic helper against
s.UserStore — the inner store — so this decorator's own MarkWatched hook never
fired and nothing queued an interest recompute. Marking a series watched on
such a backend updated progress and history but left profile_series_interest
stale until an unrelated mutation or the rebuild task touched the series.

Queue by requested target on that path, and do so even when the helper returns
an error: the fallback is a per-target loop, so a mid-loop failure still leaves
earlier targets written. A redundant queue costs one recompute; a missing one
is silent staleness. The transactional path keeps queuing from written entries
only on success, because on error nothing landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(notifications): advertise the series rollup only when the store has it

Review catch (Codex on #647). Forwarding SeriesEpisodeWatchCounts
unconditionally made the wrapper always satisfy SeriesEpisodeRollupStore, even
over the per-user SQLite backend, which has no catalog tables and cannot answer
the query. Callers read "implements the interface" as "can do this", so every
jellycompat series detail and browse would enter the fast path, take the error,
log "series watch rollup query failed", and only then fall back — turning an
expected capability absence into recurring warning noise on requests that
succeeded.

Make the capability conditional on the backing store, the way DeviceRegistry
already is, via wrapper types composed in ForUser. The remaining capabilities
stay unconditional: those have real generic fallbacks and every store can
perform them.

Tests cover both directions — a SQLite-backed wrapper must not advertise the
rollup, and a rollup-capable store must keep it (and reach it) through the
wrapper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 00:00:02 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-07-25 18:37:51 +00:00

Silo

Silo is a self-hosted media streaming server for your movies, shows, music, and books. Point it at your media folders and stream to your devices — at home or away — with direct play, remuxing, and hardware-accelerated transcoding handled automatically.

Join the community on Discord. If Silo is useful to you, consider sponsoring the project — see Supporting Silo.

Highlights

  • Plays your media, your way — direct play when the device supports it, remux or hardware-accelerated transcode (including NVENC) when it doesn't.
  • Web app included — a full-featured web client and admin interface ship with the server.
  • Works with apps you already use — optional Jellyfin/Emby-compatible API supports clients such as VidHub, Findroid, and Infuse.
  • Household profiles — multiple profiles per account, with per-profile watch state and parental controls.
  • Plugin-driven metadata — match and enrich your libraries with providers like TMDB and TVDB, installed as plugins.
  • Fast setup — one docker compose up -d brings up the whole stack; everything else is configured in the admin UI.

The easiest way to run Silo is with Docker Compose 2.24 or newer. The default stack assumes you do not already have PostgreSQL and Redis available, so it bundles PostgreSQL, Redis, FFmpeg, and the application for a one-command start.

  1. Create a .env file

    cp .env.example .env
    printf '\nPOSTGRES_PASSWORD=%s\nSECRET_KEY=%s\n' \
      "$(openssl rand -hex 24)" "$(openssl rand -base64 48)" >> .env
    

    This replaces the development database password from .env.example and creates the key Silo uses to encrypt stored credentials. Back up .env separately from PostgreSQL; losing SECRET_KEY makes those credentials unrecoverable.

  2. Set your media path

    Edit .env and set:

    MEDIA_ROOT=/path/to/your/media
    

    MEDIA_ROOT is the one value most users need to change. You can also override SILO_DATA_ROOT if you do not want bind mounts under /opt/silo, and change ports if the defaults conflict with something else on the host.

  3. Start the default integrated stack

    docker compose up -d
    

    This starts PostgreSQL, Redis, and the integrated Silo server. The app is available at http://localhost:8090. Jellyfin-compatible app support is disabled until an administrator enables it in onboarding or admin settings.

    If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing DATABASE_URL and REDIS_URL instead.

    Optional Intel/AMD VA-API or Intel Quick Sync

    The default stack is CPU-only so it starts on hosts without /dev/dri. On a Linux host with /dev/dri, enable the device overlay:

    docker compose -f docker-compose.yml -f docker-compose.vaapi.yml up -d
    

    To make that the default for this installation, set:

    COMPOSE_FILE=docker-compose.yml:docker-compose.vaapi.yml
    

    Optional NVIDIA/NVENC

    GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.

    Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.

    Use the optional override file when you want NVENC:

    docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
    

    If you want this controlled from .env, set COMPOSE_FILE:

    COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
    NVIDIA_GPU_COUNT=1
    

    Windows uses ; instead of : between compose files.

    Then docker compose up -d will include the NVIDIA override automatically.

  4. Configure through the admin UI

    Add libraries, users, metadata providers, and playback settings from the web interface.

Bind Mount Layout

The deploy-oriented compose files use host folder mappings rather than Docker-managed volumes.

By default, data is stored under /opt/silo:

  • /opt/silo/postgres
  • /opt/silo/redis
  • /opt/silo/plugins
  • /opt/silo/compat
  • /opt/silo/transcode
  • /opt/silo/catalog-seeds

The optional search profile also stores its index under /opt/silo/meilisearch.

Media is mounted into the container at /mnt/media from the host path you set in MEDIA_ROOT.

Optional Search Profile

PostgreSQL full-text search works without any optional services. Meilisearch is available when you want its search provider:

Profile Command Description
default docker compose up -d Integrated server plus bundled PostgreSQL and Redis
search docker compose --profile search up -d Add the optional Meilisearch service

Before starting the search profile, set MEILI_MASTER_KEY in .env to the output of openssl rand -hex 32. After Silo starts, choose Meilisearch under Admin > Settings > Search, set the URL to http://meilisearch:7700, enter the same key as the API key, test the connection, and save. Restart Silo, then rebuild the catalog search index from the same page. Silo continues to use PostgreSQL full-text search until you select Meilisearch.

Distributed Examples

The main Compose file includes commented proxy and transcode service examples. Most single-host installs should leave them commented because the integrated service already includes proxying and transcoding.

Multi-host operators can use those examples as a starting point for a dedicated worker Compose file connected to the deployment's shared PostgreSQL and Redis services.

Proxy nodes serve source downloads from the same absolute media paths used by direct playback. Prepared-download work can also run on transcode nodes. Each selected transcode node retains its result on node-local storage and exposes it only through Silo's authenticated internal artifact API; the paired proxy relays those bytes, so no shared artifact mount is required. Dedicated transcode nodes default to retaining prepared downloads in a protected directory inside the transcode volume captured at process startup. download.artifact_dir overrides that location for both dedicated transcode nodes and the integrated/API-local fallback, so mount the configured path on every process that prepares downloads. Changing either artifact-path setting requires a restart. Downloads with a configured server-wide or per-user bandwidth limit remain API-local so those aggregate limits stay exact. Clients discover distributed delivery through proxy_delivery on the download capability response. When it is true, they may opt into GET or HEAD /api/v1/downloads/{id}/file-proxy and /api/v1/direct-download-proxy; those routes may return a temporary redirect to a proxy node. The established /file and /direct-download routes keep serving bytes directly with their existing status-code contract; for a node-local prepared artifact, the API itself performs the authenticated relay on that fallback route.

Deployment Notes

The default compose stack intentionally bundles PostgreSQL and Redis for ease of setup and assumes a fresh install without those services already available. If you already operate PostgreSQL and Redis, omit those examples from compose and point Silo at your existing infrastructure instead. For serious installs, PostgreSQL is better on a separate VM or a managed service so upgrades, tuning, and backups are isolated from the app host. Redis can stay local for many installs, but externalizing it is also reasonable if you already operate shared infrastructure.

Silo is externally stateful by default rather than fully stateless. Durable application state lives in PostgreSQL. Redis only stores coordination and cache-style data. Silo still writes transient transcode output locally under /tmp/silo-transcode. If you switch userdb.backend=sqlite, Silo also becomes locally stateful at /var/lib/silo/userdb.

Migrating an existing Continuum Docker install should be done with the preflight helper and cutover guide in docs/continuum-to-silo-docker-migration.md.

Configuration

Silo requires DATABASE_URL and SECRET_KEY when running from source or against external infrastructure. In the default Docker Compose path, the stack wires the database and Redis URLs for you. All other settings — libraries, metadata providers, transcoding, users — are managed through the admin UI after first launch.

Server Modes

Mode Description
integrated Full server: API + frontend + scanner + transcode (default)
api API server only, no local transcoding
proxy Stream proxy node that connects to the shared deployment database and Redis
transcode HLS and prepared-download worker node that connects to the shared deployment database and Redis

PostgreSQL Auto-Tuning

The default Docker Compose stack does not require a checked-in postgresql.conf. It enables Silo's pgtune-style OLTP tuning by default:

POSTGRES_TUNE: auto

When enabled, Silo connects with DATABASE_URL and applies recommendations with ALTER SYSTEM, which writes to PostgreSQL's postgresql.auto.conf inside the database data directory. Reloadable settings are applied immediately with pg_reload_conf(). Settings that PostgreSQL marks as restart-only are written too, and Silo logs the setting names so you can restart PostgreSQL once:

docker compose restart postgres

The default Compose database user has the required PostgreSQL permissions. If you use an external PostgreSQL server, make sure the configured DATABASE_URL user can run ALTER SYSTEM, or set POSTGRES_TUNE=off and manage PostgreSQL yourself.

For POSTGRES_TUNE_MEMORY=auto, Silo uses the first trustworthy memory source: a finite Docker cgroup limit, the read-only /host/proc/meminfo mount supplied by the bundled Compose file, then /proc/meminfo with container safety guards. Auto-detected memory is treated as a PostgreSQL budget, defaulting to 75% of detected RAM so Silo, Redis, plugins, transcodes, and the OS retain headroom. POSTGRES_TUNE_DB_SIZE=auto queries pg_database_size(current_database()) and classifies the workload by comparing the database size to that memory budget.

Optional tuning overrides:

Variable Default Description
POSTGRES_TUNE_PROFILE oltp Tuning profile. Only oltp is currently supported.
POSTGRES_TUNE_MEMORY auto Server/container RAM, such as 8GB or 32GB; explicit values are used as-is.
POSTGRES_TUNE_MEMORY_BUDGET_PERCENT 75 Percent of auto-detected RAM used for PostgreSQL recommendations.
POSTGRES_TUNE_CPUS auto CPU count used for worker recommendations.
POSTGRES_TUNE_STORAGE ssd One of hdd, ssd, san, or nvme.
POSTGRES_TUNE_DB_SIZE auto Use less_ram when the database comfortably fits in RAM, mid_ram, or greater_ram for very large databases.
POSTGRES_TUNE_CONNECTIONS 100 PostgreSQL max_connections; automatically raised if Silo's app pool is configured higher.
POSTGRES_SHM_SIZE 8gb Docker /dev/shm size for the bundled PostgreSQL container.

Advanced operators can still supply their own PostgreSQL configuration or override these env vars. Set POSTGRES_TUNE=off when you do not want Silo to change PostgreSQL server settings. Settings already written with ALTER SYSTEM remain in postgresql.auto.conf; reset those PostgreSQL parameters if you later move fully to a custom postgresql.conf.

Build from Source

If you prefer running Silo without Docker:

  1. Install prerequisites: Go 1.26.4+, Node.js 22+, pnpm 10.32.1, PostgreSQL 18 with pgvector, Redis, and FFmpeg.

  2. Configure the source process

    cp .env.example .env
    printf '\nSECRET_KEY=%s\nDATABASE_URL=%s\nREDIS_URL=%s\n' \
      "$(openssl rand -base64 48)" \
      'postgres://silo:silo@localhost:5432/silo?sslmode=disable' \
      'redis://localhost:6379' >> .env
    

    Change the URLs when you use existing services instead of the bundled development defaults.

  3. Start PostgreSQL and Redis (skip if you already have them running)

    docker compose up -d postgres redis
    
  4. Build and run

    make build
    ./silo
    

    The server starts at http://localhost:8080 by default. All other settings are configured through the admin UI.

Reporting Issues

Client implementers can use the Canonical Settings API guide for contract discovery, contextual headers, remote scopes, effective reads, and the admin projection.

If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.

Please include:

  • What you were trying to do
  • Exact steps you took
  • What you expected to happen
  • What actually happened
  • What exact action is slow or broken (save, scan, browse, import, playback, etc.)
  • Whether it happens every time or only sometimes
  • The library, media type, filter, setting, or value involved
  • Version, branch, commit, and deployment details if you know them
  • Screenshots, recordings, or log snippets if relevant

If you used Claude/Codex for debugging, put that under Technical notes at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.

Use this template:

Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:

Contributing & Development

Silo is open source and contributions are welcome. See DEVELOPMENT.md for building from source in a dev workflow, running tests, database migrations, and project layout, and CONTRIBUTING.md for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.

Supporting Silo

Silo is an open-source hobby project, developed in spare time and funded out of pocket. If you'd like to support development, you can sponsor via GitHub Sponsors.

Donations go directly toward the costs of building and running the project:

  • AI development tooling subscriptions (Claude, Codex) used to build and maintain Silo
  • Push notification relay infrastructure
  • Future development costs

Sponsoring is entirely optional — Silo is and will remain free and open source. Bug reports, contributions, and feedback are just as valuable.

License & Trademarks

Silo's source code is licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) — see LICENSE.

The Silo name, logo, and wordmark are trademarks of Silo Media L.L.C. and are not covered by the AGPL. You're free to fork and redistribute the code, but forks and redistributions must not use the Silo brand as their identity and must remove or replace the brand assets. Publishing a Silo-branded app to an app store requires written permission. See TRADEMARK.md for what's permitted — including referential use like "compatible with Silo."

S
Description
Self-hosted media streaming server with a Go backend, React web UI, Docker deployment, transcoding, and Jellyfin-compatible APIs.
Readme
340 MiB
Languages
Go 72.6%
TypeScript 26.3%
PLpgSQL 0.4%
CSS 0.3%
Python 0.2%
Other 0.1%