2dedf3ff26 feat(metadata): user-triggered trailer refresh with weekly per-item cooldown (#531)
* feat(metadata): user-triggered trailer refresh with weekly per-item cooldown

Adds POST /api/v1/items/{id}/trailers/refresh so any viewer with access to a
movie or series can ask the server to fetch its remote trailers, bounded by a
one-week per-item cooldown enforced server-side.

The cooldown lives in a new nullable media_items.trailers_refresh_requested_at
column rather than the refresh debt queue, whose last_attempt_at evaporates on
success (MarkTargetSuccess deletes the row when the reason mask clears). The
gate is a single UPDATE that writes NOW() only when the stored timestamp is
NULL or older than the window, so concurrent viewers cannot both win it; a
losing caller reads the stored timestamp back to compute next_allowed_at.

MetadataService.RequestTrailersRefresh resolves the per-library trailer_kinds
allow-list first: a non-nil empty map means every containing library disabled
remote videos, which answers "disabled" without consuming the cooldown slot
(a nil map is allow-all and must not short-circuit). On winning the gate it
reuses startOnDemandMetadataRefresh, whose scheduled mode merges fill-empty,
so this non-admin trigger cannot clobber unlocked admin edits while found
videos still persist.

The handler checks item access before calling the service, so an unauthorized
caller can never burn an item's slot, and rejects non movie/series types since
those detail responses never carry videos. cooldown and disabled are expected
client-rendered states and answer 200; 429 is reserved for the per-user
in-memory limiter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(metadata): release trailer-refresh slot on failed refresh; resolve episode ids to 400

Three review findings on the viewer-facing trailer fetch.

The weekly per-item slot was consumed unconditionally on winning the gate,
but the refresh it started ran detached and only logged on failure — nothing
ever put the slot back. A brief TMDb outage therefore answered 202 queued,
failed 30s later, and then answered cooldown for seven days over work that
never happened. The repository gains an equality-guarded release
(trailers_refresh_requested_at = NULL only while it still equals the
timestamp this request wrote, so a later claim is never clobbered), and
TryClaimTrailersRefresh now RETURNINGs the timestamp it stored so a winner
holds the key to its own slot. startOnDemandMetadataRefresh splits into a
claim step and runOnDemandMetadataRefresh, which takes an optional failure
hook; only the trailer path passes one, so the existing callers are
unchanged. A timeout counts as failure. A refresh that succeeds but finds
nothing still keeps the slot — that semantics was chosen deliberately.

The in-process dedup claim (shared with the item-detail view's stale nudge)
silently dropped the start while the slot had already been consumed, so the
caller was told queued for a refresh that never began. It is now taken
before the durable slot: a request landing while an equivalent refresh is
already in flight reports queued without consuming the slot, which is both
honest and retryable if that refresh fails.

Real episode and season content IDs answered 404 rather than the contracted
400, because neither is a media_items row and GetByID queries media_items
alone. The handler now falls through to the same season/episode lookups
HandleTranslateOnView uses, authorizing through the parent series, so a
genuine episode ID reports unsupported-type and only unknown content 404s.
The type-check test no longer fabricates a MediaItem{Type: "episode"} row
that production never writes; it covers the types that do exist as
media_items rows, with the episode and season paths tested through the
lookups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(metadata): address PR review on the trailer refresh action

Six review findings on the viewer-facing trailer fetch, all verified against
the current code before changing anything.

Durable claim no longer rides the request context. A cancellation landing
after Postgres commits the gate UPDATE but before pgx returns would consume
the item's weekly slot with no refresh started and nothing holding the
timestamp needed to release it. The claim now runs on
context.WithoutCancel with its own deadline, mirroring the release.

The cooldown gate retries once when the follow-up read finds the slot free.
Classification spans two statements, so a concurrent failure-release can
land between them; the old code reported that as a cooldown with no
next_allowed_at while the slot was in fact free. A NULL read now retries the
claim, and the doubly-lost case answers "queued" (an equivalent refresh is
running) rather than an undateable cooldown.

A failed item_videos write now releases the slot. mergeAndPersist logs and
continues when the write fails, so the refresh reported success and the
viewer was locked out for a week having stored nothing. A context-scoped
observer, installed only by this action, surfaces that failure to the
existing release hook.

Winning the gate also records durable refresh debt, so a restart that kills
the detached goroutine leaves work the refresh worker picks up instead of a
consumed slot and no fetch. Uses a new reason bit rather than the generic
failure reason: nothing is wrong with the item, so it must not sit in the
failure band ahead of real debt or count as a failure in operator metrics.

Any library lookup failure now degrades the video-kind scope to unknown. An
item in two libraries where one resolved with trailers off and the other
could not be read reported "disabled" — a guess made on behalf of a library
that might be the one enabling trailers. A library that is genuinely gone is
still skipped.

Adds GET /api/v1/items/trailers/capability, following the existing
per-subsystem probe convention. The action route is registered conditionally,
so "this build has the feature" is not the same question as "this deployment
serves it", and a 404 on the POST is indistinguishable from a missing item.
The probe is registered unconditionally and answers refresh:false when
unwired.

Not changed: content-ID canonicalization mid-refresh stranding the cooldown
on the old row. The re-anchor path is manual-refresh only and this action
runs in scheduled mode, so only local-skeleton promotion can fire, and the
rename carries the timestamp and the debt row to the new id along with
everything else.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(web): format overlays schema after merging main

The line came in over-length from main's card_overlays merge and the Web
CI format check runs prettier across all of src, not just changed files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(metadata): address second review round — recovery-debt lease, locked-videos preflight, shared limiter

The restart mitigation added in the first round reintroduced two of the
problems it was closing, and the reviewer was right to push again.

Lease the recovery debt behind the fast path. The row was enqueued due
now, so refresh_metadata could claim it while the detached goroutine was
still running the same refresh — RefreshScheduledTarget does not consult
the in-process claim, so both would fetch the item at once. It is now due
5 minutes out, comfortably past the 2-minute on-demand timeout, and the
goroutine settles the row on success so it fires only when the fast path
really did not finish. Settling clears just the trailers-requested bit,
keeping any real debt the item still carries.

Release the cooldown after a failed recovery. A recovery runs in a worker
that never saw the claim, so a failure left the viewer blocked for the
week having stored nothing. RefreshScheduledTarget now adopts the claim
when the debt row carries the trailers-requested reason, reading the
stored timestamp so the release stays equality-guarded, and hands the
slot back on the same failures the fast path's hook covers — including a
videos write that failed and was only logged.

Preflight the videos lock. locked_fields containing FieldVideos makes
mergeAndPersist skip the item_videos write, so the refresh "succeeded"
and kept the cooldown while never being able to save trailers. It now
answers disabled before consuming the claim; reusing that status rather
than adding one is deliberate, since clients treat an unknown status as a
dead end and "trailers cannot be fetched for this item" is what disabled
already means to a viewer.

Use the shared limiter. A private MemoryLimiter gave every instance an
independent per-user allowance on Redis deployments, and the per-item
cooldown cannot compensate — it bounds one item, while this budget bounds
how many distinct items a user can start refreshes for. The action now
takes the middleware's configured limiter, with namespaced keys, and
falls back to a private one only when rate limiting is off.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 20:53:21 -04:00
2026-07-25 18:37:51 +00:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-07-25 18:37:51 +00:00

Silo

Silo is a self-hosted media streaming server for your movies, shows, music, and books. Point it at your media folders and stream to your devices — at home or away — with direct play, remuxing, and hardware-accelerated transcoding handled automatically.

Join the community on Discord. If Silo is useful to you, consider sponsoring the project — see Supporting Silo.

Highlights

  • Plays your media, your way — direct play when the device supports it, remux or hardware-accelerated transcode (including NVENC) when it doesn't.
  • Web app included — a full-featured web client and admin interface ship with the server.
  • Works with apps you already use — optional Jellyfin/Emby-compatible API supports clients such as VidHub, Findroid, and Infuse.
  • Household profiles — multiple profiles per account, with per-profile watch state and parental controls.
  • Plugin-driven metadata — match and enrich your libraries with providers like TMDB and TVDB, installed as plugins.
  • Fast setup — one docker compose up -d brings up the whole stack; everything else is configured in the admin UI.

The easiest way to run Silo is with Docker Compose. The default stack assumes you do not already have PostgreSQL and Redis available, so it bundles PostgreSQL, Redis, FFmpeg, and the application for a one-command start.

  1. Create a .env file

    cp .env.example .env
    
  2. Set your media path

    Edit .env and set:

    MEDIA_ROOT=/path/to/your/media
    

    MEDIA_ROOT is the one value most users need to change. You can also override SILO_DATA_ROOT if you do not want bind mounts under /opt/silo, and change ports if the defaults conflict with something else on the host.

  3. Start the default integrated stack

    docker compose up -d
    

    This starts PostgreSQL, Redis, and the integrated Silo server. The app is available at http://localhost:8090. Jellyfin-compatible app support is disabled until an administrator enables it in onboarding or admin settings.

    If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing DATABASE_URL and REDIS_URL instead.

    Optional NVIDIA/NVENC

    GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.

    Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.

    Use the optional override file when you want NVENC:

    docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
    

    If you want this controlled from .env, set COMPOSE_FILE:

    COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
    NVIDIA_GPU_COUNT=1
    

    Windows uses ; instead of : between compose files.

    Then docker compose up -d will include the NVIDIA override automatically.

  4. Configure through the admin UI

    Add libraries, users, metadata providers, and playback settings from the web interface.

Bind Mount Layout

The deploy-oriented compose files use host folder mappings rather than Docker-managed volumes.

By default, data is stored under /opt/silo:

  • /opt/silo/postgres
  • /opt/silo/redis
  • /opt/silo/transcode
  • /opt/silo/catalog-seeds

Media is mounted into the container at /mnt/media from the host path you set in MEDIA_ROOT.

Optional Profiles

The main compose file is integrated-first. These profiles exist for operators testing distributed mode or mirroring a split deployment shape. Most single-host installs should stay on the default integrated service, because it already includes proxying and transcoding.

Profile Command Description
default docker compose up -d Integrated server plus bundled PostgreSQL and Redis
proxy docker compose --profile proxy up -d Start a standalone proxy service for distributed-mode testing
transcode docker compose --profile transcode up -d Start a standalone transcode service for distributed-mode testing

You can enable both optional examples together:

docker compose --profile proxy --profile transcode up -d

If you are splitting workers across multiple hosts, use the separate remote worker example instead of trying to stretch the main compose file across machines.

Advanced Remote Node Example

For a dedicated remote transcode worker, use docker-compose.remote-transcode.yml. That file is intended for a separate worker host that connects back to an existing Silo deployment using shared PostgreSQL and Redis.

Deployment Notes

The default compose stack intentionally bundles PostgreSQL and Redis for ease of setup and assumes a fresh install without those services already available. If you already operate PostgreSQL and Redis, omit those examples from compose and point Silo at your existing infrastructure instead. For serious installs, PostgreSQL is better on a separate VM or a managed service so upgrades, tuning, and backups are isolated from the app host. Redis can stay local for many installs, but externalizing it is also reasonable if you already operate shared infrastructure.

Silo is externally stateful by default rather than fully stateless. Durable application state lives in PostgreSQL. Redis only stores coordination and cache-style data. Silo still writes transient transcode output locally under /tmp/silo-transcode. If you switch userdb.backend=sqlite, Silo also becomes locally stateful at /var/lib/silo/userdb.

Migrating an existing Continuum Docker install should be done with the preflight helper and cutover guide in docs/continuum-to-silo-docker-migration.md.

Configuration

Silo requires only a DATABASE_URL when running from source or against external infrastructure. In the default Docker Compose path, the stack wires the database and Redis URLs for you. All other settings — libraries, metadata providers, transcoding, users — are managed through the admin UI after first launch.

Server Modes

Mode Description
integrated Full server: API + frontend + scanner + transcode (default)
api API server only, no local transcoding
proxy Stream proxy node that connects to the shared deployment database and Redis
transcode HLS transcode worker node that connects to the shared deployment database and Redis

PostgreSQL Auto-Tuning

The default Docker Compose stack does not require a checked-in postgresql.conf. It enables Silo's pgtune-style OLTP tuning by default:

POSTGRES_TUNE: auto

When enabled, Silo connects with DATABASE_URL and applies recommendations with ALTER SYSTEM, which writes to PostgreSQL's postgresql.auto.conf inside the database data directory. Reloadable settings are applied immediately with pg_reload_conf(). Settings that PostgreSQL marks as restart-only are written too, and Silo logs the setting names so you can restart PostgreSQL once:

docker compose restart postgres

The default Compose database user has the required PostgreSQL permissions. If you use an external PostgreSQL server, make sure the configured DATABASE_URL user can run ALTER SYSTEM, or set POSTGRES_TUNE=off and manage PostgreSQL yourself.

For POSTGRES_TUNE_MEMORY=auto, Silo uses the first trustworthy memory source: a finite Docker cgroup limit, the read-only /host/proc/meminfo mount supplied by the bundled Compose file, then /proc/meminfo with container safety guards. Auto-detected memory is treated as a PostgreSQL budget, defaulting to 75% of detected RAM so Silo, Redis, plugins, transcodes, and the OS retain headroom. POSTGRES_TUNE_DB_SIZE=auto queries pg_database_size(current_database()) and classifies the workload by comparing the database size to that memory budget.

Optional tuning overrides:

Variable Default Description
POSTGRES_TUNE_PROFILE oltp Tuning profile. Only oltp is currently supported.
POSTGRES_TUNE_MEMORY auto Server/container RAM, such as 8GB or 32GB; explicit values are used as-is.
POSTGRES_TUNE_MEMORY_BUDGET_PERCENT 75 Percent of auto-detected RAM used for PostgreSQL recommendations.
POSTGRES_TUNE_CPUS auto CPU count used for worker recommendations.
POSTGRES_TUNE_STORAGE ssd One of hdd, ssd, san, or nvme.
POSTGRES_TUNE_DB_SIZE auto Use less_ram when the database comfortably fits in RAM, mid_ram, or greater_ram for very large databases.
POSTGRES_TUNE_CONNECTIONS 100 PostgreSQL max_connections; automatically raised if Silo's app pool is configured higher.
POSTGRES_SHM_SIZE 8gb Docker /dev/shm size for the bundled PostgreSQL container.

Advanced operators can still supply their own PostgreSQL configuration or override these env vars. Set POSTGRES_TUNE=off when you do not want Silo to change PostgreSQL server settings. Settings already written with ALTER SYSTEM remain in postgresql.auto.conf; reset those PostgreSQL parameters if you later move fully to a custom postgresql.conf.

Build from Source

If you prefer running Silo without Docker:

  1. Install prerequisites: Go 1.24+, Bun 1.0+, PostgreSQL 18+, and FFmpeg.

  2. Start PostgreSQL and Redis (skip if you already have them running)

    docker compose up -d postgres redis
    

    The main compose file still expects MEDIA_ROOT to be set even if you only want the bundled PostgreSQL and Redis services, so set that in .env first.

  3. Configure the database connection

    cp .env.example .env
    

    Edit .env and set DATABASE_URL to point to your PostgreSQL instance.

  4. Build and run

    make build
    ./silo
    

    The server starts at http://localhost:8080 by default. All other settings are configured through the admin UI.

Reporting Issues

If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.

Please include:

  • What you were trying to do
  • Exact steps you took
  • What you expected to happen
  • What actually happened
  • What exact action is slow or broken (save, scan, browse, import, playback, etc.)
  • Whether it happens every time or only sometimes
  • The library, media type, filter, setting, or value involved
  • Version, branch, commit, and deployment details if you know them
  • Screenshots, recordings, or log snippets if relevant

If you used Claude/Codex for debugging, put that under Technical notes at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.

Use this template:

Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:

Contributing & Development

Silo is open source and contributions are welcome. See DEVELOPMENT.md for building from source in a dev workflow, running tests, database migrations, and project layout, and CONTRIBUTING.md for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.

Supporting Silo

Silo is an open-source hobby project, developed in spare time and funded out of pocket. If you'd like to support development, you can sponsor via GitHub Sponsors.

Donations go directly toward the costs of building and running the project:

  • AI development tooling subscriptions (Claude, Codex) used to build and maintain Silo
  • Push notification relay infrastructure
  • Future development costs

Sponsoring is entirely optional — Silo is and will remain free and open source. Bug reports, contributions, and feedback are just as valuable.

License & Trademarks

Silo's source code is licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) — see LICENSE.

The Silo name, logo, and wordmark are trademarks of Silo Media L.L.C. and are not covered by the AGPL. You're free to fork and redistribute the code, but forks and redistributions must not use the Silo brand as their identity and must remove or replace the brand assets. Publishing a Silo-branded app to an app store requires written permission. See TRADEMARK.md for what's permitted — including referential use like "compatible with Silo."

S
Description
Self-hosted media streaming server with a Go backend, React web UI, Docker deployment, transcoding, and Jellyfin-compatible APIs.
Readme
340 MiB
Languages
Go 72.6%
TypeScript 26.3%
PLpgSQL 0.4%
CSS 0.3%
Python 0.2%
Other 0.1%