2c714e4ef2 feat(requests): pluginize request fulfillment behind request_router.v1 (#104)
* docs: design spec for pluginizing requests fulfillment

Pluginize the requests fulfillment backend behind an agnostic
request_router.v1 capability (high seam: whole-request fulfiller).
Host keeps lifecycle/quota/policy/quality-governance and a generic
two-tier connection registry; plugins own routing+submission+status.
First plugin extracts multi-instance Sonarr/Radarr; Seerr follows in
a separate spec. Preserves autoscan reuse of arr connection rows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for requests pluginization

Three-phase plan: (1) request_router.v1 SDK capability, (2) new
silo-plugin-requests-arr plugin extracting multi-instance Sonarr/Radarr,
(3) host refactor routing fulfillment through the plugin while keeping
quality governance, target records, and autoscan connection reuse host-side.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(db): generalize request_integrations into a two-tier connection registry

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): add generic connection fields to Integration + repo mapping

* feat(pluginhost): typed RequestRouter capability client + resolver

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): plugin-backed RequestRouterProvider seam

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): route fulfillment through RequestRouterProvider; host keeps quality governance

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): base auto-approve gate on router connection model

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(api): wire plugin-backed request router at both service sites

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(requests): remove in-host Sonarr/Radarr fulfillment code

* test(autoscan): lock request-integration reuse after connection generalization

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): plugin-driven request integration config form

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(api): echo router connection fields in integration response

* fix(requests): retry dropped qualities, contain to one router installation, dedupe targets

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): harden plugin trust boundary (validate targets, contain bad connections, media-type routing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): tighten auto-approve gate, restore default/4k validation, propagate config-encode error, drop itoa wrapper, test status/options translation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(requests): resolve integrations/settings/secrets once per reconcile cycle

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): dedupe config helpers, preserve zero profile id, stabilize installation default, drop redundant options write

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: design spec for schema-driven plugin config form

Extends AdminFormDescriptor into a full form-description language (dynamic
options, multi-select, conditional visibility, sections, validation) + a
plugin Validate RPC, rendered by one reusable SchemaForm engine. Retires the
bespoke arr connection form and integrationOptionsFromRouter so any
request_router backend renders its config UI from manifest data with zero
host changes. Addresses code-review finding #9.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for schema-driven plugin config form

Six phases: SDK AdminFormDescriptor extensions + Validate RPC; reusable
SchemaForm renderer (refactor PluginConfigForm onto it); host Validate
plumbing + generic options + legacy-column derivation + retire
integrationOptionsFromRouter; requests admin page swap to SchemaForm with
per-plugin grouping; arr manifest enrichment + Validate impl; verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): extend plugin admin-form TS types (sections, conditions, validation, multi-select)

* feat(web): schema-form pure utils (show_when, validation, value coercion)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): SchemaForm renderer (controls, sections, show_when, dynamic options, errors)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(web): render PluginConfigForm via the shared SchemaForm engine

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): RequestRouter Validate client + provider seam

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): plugin Validate on save, generic options, derive legacy columns from plugin_config

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(api): generic options response + 400 field_errors on plugin validation failure

* feat(web): generic request-integration options type + surface validation field_errors

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): render request connections via SchemaForm; per-plugin grouping; retire bespoke arr form

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): silent connection-options probe with inline failure status (no toast spam)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(api): serialize admin_form sections/show_when/dynamic_options/validation to the client

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): drop show_when-hidden fields from buildSchemaValues payload

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): pass requester user id as int64 (no truncation)

* refactor(requests): drop legacy arr columns; plugin_config is sole source of truth

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): backfill api key in plugin validate; centralize validation 400; drop duplicate host cross-field check; guard admin-form serializer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): refuse stored api key reuse when base_url changes (security hardening)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): SchemaForm regex-guard, default_value, type-driven coercion, validity callback

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): connection-options latest-wins + narrowed deps + clear stale errors; auto-select; type-driven persist; reuse types

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: design spec for silo-plugin-requests-seerr (request_router.v1 backend)

* docs: implementation plan for silo-plugin-requests-seerr

* docs(spec): FindExistingRequest uses /api/v1/request (carries request id)

* docs(spec): seerr hardening — id-recovery, 404 terminal, media-status, sort pin, single missing-tmdb message

* docs: design spec for shared plugin-platform SDK helpers (code-review #10)

* docs: plan for plugin-platform SDK helpers (#10) + spec fix (inline broker wiring, no import cycle)

* docs: design spec for typed 4K quality-tier signal (code-review #9)

* docs: implementation plan for typed 4K quality-tier signal (#9)

* feat(requests): stamp is4k per quality (host owns the 4K-tier fact)

* fix(requests): store capability sub-id, not the type, in request_integrations

request_integrations.capability_id carried the capability TYPE
("request_router.v1") instead of the capability sub-id ("arr"/"seerr").
The host resolves a router plugin via
requireCapability("request_router.v1", id), which keys on (type, id), so
storing the type resolved to no capability: every save/options/fulfill
500'd ("Request operation failed" / "no fulfillment backend configured")
in ~1ms, before the arr/Seerr API was ever contacted. The path was
internally split-brained (the fulfillment filter matched the type while
the dispatcher needed the sub-id), so it never worked end-to-end; the
unit tests hid it behind a fake provider that skips requireCapability.

Align capability_id with the scan_source/metadata convention (sub-id):
- validateInstance: require a non-empty sub-id; drop the default-to-type
  and the "!= request_router.v1" reject.
- resolveRouterConnections / integrationConfigured / unbound-guidance:
  match on a non-empty capability, not type equality.
- repository: persist capability_id verbatim (never default to the type).
- web AdminRequests: send the selected plugin's capability.id in both the
  options probe and the save payload (was a hardcoded type constant).
- migration 20260608131649: backfill capability_id from each bound
  installation's request_router.v1 capability and drop the column's
  misleading default. Unbound legacy rows are left for admin re-save.

Tests: validateInstance now requires the sub-id, and the selected sub-id
must reach the plugin Validate RPC (fakeRouterProvider records it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): polish request connection cards (grouped toggles + option loading states)

The schema-driven connection cards rendered each boolean as its own
bordered, double-labeled box and showed dynamic SELECTs (root folder,
quality profile, tags) as empty controls with a single "Loading options…"
line while the host probed the service.

- Toggles render as a cohesive settings list: consecutive switches collapse
  into one bordered, divided container; each row is toggle-first with the
  label + description hugging beside it (no stranded whitespace between a
  short label and its switch). Honors show_when, so conditional toggles
  still group correctly.
- Dynamic SELECT/MULTI_SELECT fields show a per-field spinner + shimmer
  skeleton while options load, and only when there's nothing to show yet —
  a background re-probe never flashes over the operator's current value.
- Sections get a softer surface and clearer titles; the card's enable
  switch is labeled Enabled/Disabled; the options-load failure is a proper
  inline alert with retry guidance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): treat "Any"/no-cap playback ceiling as 4K-allowed

allowedQualities decided whether to also request 2160p with
`CompareQuality(ceiling, PlaybackQuality4K) >= 0`. But an "Any" max
playback quality resolves to an empty ceiling ("no cap"), and in
qualityRank "" is the LOWEST rank (0) — so CompareQuality("", "2160p")
returns -1 and 4K was dropped. A requester with unlimited playback quality
only got a 1080p request, never the 4K one.

Use access.QualityAllowed(PlaybackQuality4K, ceiling), which already
encodes "empty ceiling == no cap == allows everything". Now:
- "" / "Any"  -> 1080p + 2160p
- "2160p"     -> 1080p + 2160p
- "1080p"     -> 1080p only
- resolver error still fails safe to the HD ceiling.

Tests: add an "any/no-cap ceiling adds 2160p" case; the unknown-quality,
status-coercion, dedup, and per-quality-idempotency submit tests now pin
an explicit HD ceiling (they relied on the old empty-default == HD-only
behavior and were not about 4K entitlement).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: design spec for collapsible Library + anime gate/nesting (request card UI, Spec A)

Spec A of two for the request connection card UX: Library section becomes
collapsible/collapsed (auto-expanding on validation errors) and the anime
override fields move into a single gated section below Library instead of
popping out as a detached sibling card. Single-default enforcement is Spec B.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for collapsible Library + anime gate/nesting (Spec A)

Task-by-task TDD plan: SchemaForm auto-expand-on-error + nested-field
affordance (silo-server), arr manifest regroup (collapsible Library, anime
gate section), then build/deploy/reinstall + manual verify.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): auto-expand collapsible schema sections that have validation errors

SchemaFormSection now accepts a forceOpen prop; when any field in the section
has a mergedError (client validation or server error), the section expands
automatically so required-field setup can never be hidden behind a collapsed
accordion. The operator's manual toggle is preserved via a nullable userOpen
state that only takes effect when forceOpen is false.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): indent show_when-revealed schema fields to read as nested

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: design spec for schema-driven single-default exclusivity enforcement (Spec B)

At most one connection per service_kind may be the HD default (is_default) or
4K default (is_default_4k). Generic exclusivity: a new AdminFormField
exclusive_group_field declares the rule, the plugin Validate enforces it
against host-supplied siblings (config only, no creds), and the admin UI
auto-clears conflicts as you toggle. Host stays plugin-agnostic. Forward-only;
no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for single-default exclusivity enforcement (Spec B)

Five TDD tasks across 3 repos: SDK proto (siblings + exclusive_group_field)
+ buf regen; arr Validate cross-sibling + manifest; host gathers siblings
(config-only) into Validate; frontend generic mutual-exclusion helper; then
re-vendor/rebuild/redeploy + plugininstall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(requests): pass sibling connections to plugin Validate for cross-connection rules

Adds siblings []ResolvedRouterConnection to RequestRouterProvider.Validate so
the plugin can enforce cross-connection invariants (e.g. one default per
service_kind) without the host resolving sibling credentials. The new
siblingConnections helper gathers other connections on the same installation,
carrying only ID + PluginConfig. Vendor updated to the Task 1 SDK version that
carries ValidateRequest.Siblings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web): auto-clear mutually-exclusive defaults across request connection cards

Adds generic applyExclusivity helper and wires it into updateCardConfig so
turning on a field with exclusive_group_field proactively clears the same
field on sibling cards sharing the same group value, matching server-side
enforcement with a proactive UX.

* docs: design spec for single-flighting plugin client launch (cold-start herd fix)

Concurrent ensureClient calls for a cold installation each spawn a redundant
plugin process (Host.Start releases its lock during launch). Wrap ensureClient
in a per-installation singleflight.Group so concurrent first-use collapses to
one launch. Host-only fix; surfaced while testing the request-router feature.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for single-flighting plugin client launch

TDD: concurrency tests (herd collapses to one launch, warm-cache reuse,
distinct installations stay parallel, failed launch propagates) + the
singleflight wrapper around ensureClient; then rebuild/redeploy + verify.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(plugins): single-flight ensureClient to prevent cold-start launch herd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(requests): harden capability containment + dedupe eligibility; UI/migration cleanups

Addresses /code-review high findings on the previously-unreviewed commits:
- resolveRouterConnections contains fulfillment to the first chosen
  (installation, capability) and locks only after a connection's key resolves,
  so a plugin exposing >1 request_router capability never mixes connections and
  a skipped bad-key connection never pins the capability (+ test).
- extract eligibleRouterConnection, shared by resolveRouterConnections and
  integrationConfigured so the auto-approval gate and fulfillment filter can't
  drift.
- SchemaForm: shared FieldDescription helper (field/switch/section); key switch
  groups by position so a show_when reveal doesn't remount the group (focus loss).
- migration backfill uses a deterministic correlated subquery instead of a join
  cross-product when an installation exposes multiple request_router capabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: design spec for opt-in Seerr per-user requester mapping

Per-connection requester_mode (admin default | mapped). In mapped mode the host
pushes the requester email/username into the Fulfill descriptor and the seerr
plugin resolves/creates the matching Seerr user by email with operator-chosen
default permissions, attributing the request (and gating Seerr-side approval via
the auto-approve permission). Spans SDK (descriptor fields), host (extend
UserIdentityLookup with email + a requester resolver), and the seerr plugin
(Seerr user API + mapping). Fallback to admin on any failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: implementation plan for Seerr per-user requester mapping

Five TDD tasks across 3 repos: SDK descriptor fields (requester_email/username);
host resolves identity (UserIdentityLookup+email, RequesterIdentityResolver,
populate descriptor at both Fulfill sites); seerr config+user API (find/create
by email, exported PermissionBits); seerr Fulfill mapping + admin_form; then
re-vendor/rebuild/redeploy + plugininstall (installation 6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: make Seerr unmapped-requester behavior a toggle (admin fallback | fail request)

Per user feedback: require_mapped_user switch (default off = admin fallback,
on = fail the request). Updates spec + plan Tasks 3/4 (config field, Fulfill
honoring the toggle via a mapFailed signal, a new test, and the manifest switch).

* feat(requests): resolve requester email/username into the Fulfill descriptor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: design spec for simplified Seerr mapped-user permissions

Reduce the 5 permission toggles to two (request_4k_all + auto_approve);
1080p always granted; remove manage_requests; 4K eligibility per-user from the
request's qualities (host-decided, same as arr) with a blanket override toggle.
Seerr-plugin-only; permission-only override (host still gates 4K requests).

* docs: implementation plan for simplified Seerr mapped-user permissions

Two tasks (seerr-plugin-only): replace the 5 perm toggles with request_4k_all +
auto_approve (1080p always; 4K from request qualities via userPermissions;
remove PermManageRequests/PermissionBits; manifest + json_schema), then rebuild
+ reinstall (installation 6). No host/SDK change.

* docs: design spec for host rebase onto main + #95 credential-model adoption

Per-commit rebase of our 68 request-router commits onto the force-pushed
origin/main (drops 188 patch-equivalent). At the credential-path conflicts, adopt
#95's inline secret.Cipher model: keep our plugin columns + #95's encrypt/decrypt
in repository.go; drop our SecretResolver and read in.APIKeyRef directly in
service.go; wire NewRepository(pool, dataCipher). #39-area conflicts take ours
(our pluginization supersedes it). Security review + SECRET_KEY deploy note.

* docs: implementation plan for host rebase + #95 credential adoption

Four tasks: (1) guided per-commit rebase onto origin/main, take-ours on
credential files so it builds; (2) TDD integration commit adopting #95's
secret.Cipher (encrypt/decrypt in repository.go, drop SecretResolver, read
APIKeyRef directly, wire NewRepository(pool, cipher)); (3) security review;
(4) pin published SDK v0.6.0, push fork, open host PR with SECRET_KEY deploy note.

* chore(rebase): restore scan-source service methods + temp requests-repo arity

Post-rebase conflict fixups: take-ours on internal/plugins/service.go dropped
origin's ScanSourceClientByPluginID (independent upstream capability) — restored.
mediarequests.NewRepository temporarily 1-arg to match our pre-#95 repo; Task 2
restores the cipher arg when adopting #95's at-rest credential model.

* feat(requests): adopt at-rest credential cipher (#95) for plugin api keys; drop SecretResolver

* build: pin published silo-plugin-sdk v0.6.0 (drop local replace)

* test(requests): guard at-rest cipher round-trip + empty-key auto-approval (code-review)

Max-effort code review of the #95 credential integration. Fixes the actionable
findings:
- TestEncryptAPIKeyRoundTripAndAAD: pins encryptAPIKey<->DecryptIfEncrypted
  inversion, the id-bound apiKeyAAD == secret.RowAAD(...) match (so #95's backfill
  rows decrypt), the blank-key "" sentinel, and row-bound AAD — the security-
  critical invariants had no automated guard (no DB harness for scanIntegration).
- TestCreateRequestAutoApprovalEmptyKeyTreatedAsUnconfigured: pins that a keyless
  connection reads as unconfigured (request stays pending, never submitted), so
  integrationConfigured and resolveRouterConnections can't drift.
- Fix stale fulfillContext comment (referenced a resolved-API-key cache removed
  with SecretResolver).

Assessed-not-changed (documented): decrypt-error-fails-closed and failed-backfill
behaviors are origin/main #95 design we adopt; nil-cipher is unreachable in prod
and matches the codebase-wide no-guard pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* build: drop stale machine-local SDK replace comment from go.mod

The replace directive was already removed when v0.6.0 was pinned (3410df7);
this leftover comment falsely claimed a local replace still existed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(web): prettier-format schema-form utils to 100-col width

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: drop internal superpowers specs/plans from PR

These design specs and implementation plans are internal development
artifacts; keep them out of the upstream PR diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(metadata): exclude providers from content levels they don't declare

ResolveChain falls back to every enabled metadata provider when a library
+ content-level has no enabled chain entry. That fallback was media-type
blind: a provider declaring default_priority only for an unrelated level
(e.g. an audiobook provider declaring {"audiobook": N}) was kept in the
list (merely sorted last) and invoked for video content levels.

In production this made silo.audiobook-metadata hammer external audiobook
APIs with anime/movie/series titles every scheduled enrichment pass
(MatchWorker, 30s) for the season/episode levels that had no enabled chain
entry. Disabling the chain entries did not help because the fallback never
consults them; only disabling the installation removed it from the global
set.

Treat a non-empty default_priority map as the provider enumerating the
content levels it supports: in resolveEnabledProvidersByPriority, exclude
providers whose declared map omits the requested level instead of ranking
them last. Providers that declare no default_priority make no claim and
stay eligible everywhere (legacy behavior).

Fixes #105

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(plugins): isolate singleflight launch from leader ctx cancellation

The deduped ensureClient launch ran doEnsureClient under the leader caller's
ctx, so if that caller's request was canceled/timed out mid-launch the shared
plugin start was torn down and the error propagated to every waiter. Run the
launch under context.WithoutCancel so a single caller cannot cancel work the
other waiters depend on (values preserved for tracing/auth). (CodeRabbit)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(api): nil-guard request-router wiring

RequestRouterClient dereferenced a.Svc unconditionally and AttachRequestRouter
called SetRouterProvider even with nil deps, so a build without the plugin
service would panic instead of degrading. Guard both: the adapter returns a
controlled error and AttachRequestRouter no-ops, leaving fulfillment to fail
with the existing "no backend configured" path. (CodeRabbit)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): correct value coercion + track capability sub-id in request form

- schemaForm: Boolean("false") was true; parse string booleans explicitly.
  array:num now coerces decimals ("1.5"), array:int stays integer-only.
- AdminRequests: track capability_id alongside installation_id (composite
  <Select> value) so a multi-capability installation resolves the exact
  backend; reset pluginConfig when the selected plugin changes so plugin A's
  keys never reach plugin B's options probe/save. (CodeRabbit)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(requests): address request-router review findings

* fix(requests): handle router review edge cases

* fix(web): resolve schema form build casing

* fix(requests): skip unconfigured 4k fulfillment targets

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-06-09 13:00:48 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00
2026-05-22 23:26:56 -04:00

Silo

A self-hosted media streaming server with a React frontend and Go backend. Supports direct play, remuxing, and hardware-accelerated transcoding. Compatible with Jellyfin/Emby clients (VidHub, Findroid) via a built-in compatibility layer.

Join the community on Discord.

The easiest way to run Silo is with Docker Compose. The default stack assumes you do not already have PostgreSQL and Redis available, so it bundles PostgreSQL, Redis, FFmpeg, and the application for a one-command start.

  1. Create a .env file

    cp .env.example .env
    
  2. Set your media path

    Edit .env and set:

    MEDIA_ROOT=/path/to/your/media
    

    MEDIA_ROOT is the one value most users need to change. You can also override SILO_DATA_ROOT if you do not want bind mounts under /opt/silo, and change ports if the defaults conflict with something else on the host.

  3. Start the default integrated stack

    docker compose up -d
    

    This starts PostgreSQL, Redis, and the integrated Silo server. The app is available at http://localhost:8090 and the Jellyfin-compatible endpoint at http://localhost:8096.

    If you already have PostgreSQL and Redis available, omit those bundled service examples from compose and point Silo at your existing DATABASE_URL and REDIS_URL instead.

    Optional NVIDIA/NVENC

    GPU support is kept out of the default compose file so hosts without NVIDIA drivers work unchanged.

    Install the NVIDIA Container Toolkit and use a Docker Compose version with GPU reservation support before enabling this override.

    Use the optional override file when you want NVENC:

    docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d
    

    If you want this controlled from .env, set COMPOSE_FILE:

    COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml
    NVIDIA_GPU_COUNT=1
    

    Windows uses ; instead of : between compose files.

    Then docker compose up -d will include the NVIDIA override automatically.

  4. Configure through the admin UI

    Add libraries, users, metadata providers, and playback settings from the web interface.

Bind Mount Layout

The deploy-oriented compose files use host folder mappings rather than Docker-managed volumes.

By default, data is stored under /opt/silo:

  • /opt/silo/postgres
  • /opt/silo/redis
  • /opt/silo/transcode
  • /opt/silo/catalog-seeds

Media is mounted into the container at /mnt/media from the host path you set in MEDIA_ROOT.

Optional Profiles

The main compose file is integrated-first. These profiles exist for operators testing distributed mode or mirroring a split deployment shape. Most single-host installs should stay on the default integrated service, because it already includes proxying and transcoding.

Profile Command Description
default docker compose up -d Integrated server plus bundled PostgreSQL and Redis
proxy docker compose --profile proxy up -d Start a standalone proxy service for distributed-mode testing
transcode docker compose --profile transcode up -d Start a standalone transcode service for distributed-mode testing

You can enable both optional examples together:

docker compose --profile proxy --profile transcode up -d

If you are splitting workers across multiple hosts, use the separate remote worker example instead of trying to stretch the main compose file across machines.

Advanced Remote Node Example

For a dedicated remote transcode worker, use docker-compose.remote-transcode.yml. That file is intended for a separate worker host that connects back to an existing Silo deployment using shared PostgreSQL and Redis.

Deployment Notes

The default compose stack intentionally bundles PostgreSQL and Redis for ease of setup and assumes a fresh install without those services already available. If you already operate PostgreSQL and Redis, omit those examples from compose and point Silo at your existing infrastructure instead. For serious installs, PostgreSQL is better on a separate VM or a managed service so upgrades, tuning, and backups are isolated from the app host. Redis can stay local for many installs, but externalizing it is also reasonable if you already operate shared infrastructure.

Silo is externally stateful by default rather than fully stateless. Durable application state lives in PostgreSQL. Redis only stores coordination and cache-style data. Silo still writes transient transcode output locally under /tmp/silo-transcode. If you switch userdb.backend=sqlite, Silo also becomes locally stateful at /var/lib/silo/userdb.

Migrating an existing Continuum Docker install should be done with the preflight helper and cutover guide in docs/continuum-to-silo-docker-migration.md.

Build from Source

Prerequisites

  • Go 1.24+
  • Bun 1.0+
  • PostgreSQL 18+
  • FFmpeg (for transcoding support)

Quick Start

  1. Start PostgreSQL (skip if you already have one running)

    docker compose up -d postgres redis
    

    The main compose file still expects MEDIA_ROOT to be set even if you only want the bundled PostgreSQL and Redis services, so set that in .env first.

  2. Configure the database connection

    cp .env.example .env
    

    Edit .env and set DATABASE_URL to point to your PostgreSQL instance.

  3. Build and run

    make build
    ./silo
    

    The server starts at http://localhost:8080 by default. All other settings are configured through the admin UI.

Development

Local development remains intentionally separate from the deploy-oriented compose setup. Use docker-compose.yml and the existing source-build workflow for local development.

# Run the frontend dev server (hot reload, proxies API to :8090)
make dev-frontend

# Run the Go backend
make dev-backend

If you are developing Silo and silo-plugin-sdk together, keep using the local go.work workspace. That workspace is a developer convenience only. CI and release builds run with GOWORK=off, so any new SDK helper used here must be pushed and tagged in silo-plugin-sdk before this repo can merge or release the change.

See CONTRIBUTING.md for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.

Plugin authors should start with docs/architecture/plugin-development.md, which covers the RPC plugin package format, generated proto workflow, SDK import paths, route and asset exposure, and auth or user-config integration points.

Reporting Issues

If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.

Please include:

  • What you were trying to do
  • Exact steps you took
  • What you expected to happen
  • What actually happened
  • What exact action is slow or broken (save, scan, browse, import, playback, etc.)
  • Whether it happens every time or only sometimes
  • The library, media type, filter, setting, or value involved
  • Version, branch, commit, and deployment details if you know them
  • Screenshots, recordings, or log snippets if relevant

If you used Claude/Codex for debugging, put that under Technical notes at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.

Use this template:

Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:

Make Targets

Target Description
make build Build frontend + Go binary
make frontend Build frontend only
make dev-frontend Vite dev server with HMR
make dev-backend Run Go backend (integrated mode)
make dev-proxy Run a standalone proxy node
make dev-transcode Run a standalone transcode node
make migrate-create NAME=add_thing Create a timestamped Goose SQL migration
make migrate-validate Validate Goose migration files without touching a database
make migrate-status Show Goose migration status using Silo's bootstrapping runner
make migrate-up Apply pending Goose migrations using Silo's bootstrapping runner
make clean Remove build artifacts

Database Migrations

PostgreSQL schema migrations are managed by Goose. Migration SQL files live in migrations/sql/ and use Goose annotations. Converted legacy migrations keep their original numeric versions so existing schema_versions rows can bootstrap cleanly into Goose without replaying old SQL. New migrations should be created with timestamped filenames:

make migrate-create NAME=add_thing
make migrate-validate

Do not run goose fix; timestamped migrations are the repository policy because they avoid version collisions across parallel PRs. The existing 001-style files are historical compatibility records, not the naming pattern for new work. Runtime migrations are applied by the integrated/API server only. Proxy and transcode modes never mutate schema. For existing installs, use make migrate-status and make migrate-up rather than invoking the Goose CLI directly; those targets copy legacy schema_versions rows into public.goose_db_version under the migration lock before reading or applying migrations. Set ENV_FILE=path/to/.env when the database URL should be read from a non-default env file.

Running Tests

# Go tests (uses testcontainers — Docker must be running)
go test ./...

# Frontend tests
cd web && bun test

Linting

# Go
golangci-lint run

# Frontend
cd web && bun run lint
cd web && bun run format:check

License

Silo is licensed under AGPL-3.0-or-later. See LICENSE.

Configuration

Silo requires only a DATABASE_URL when running from source or against external infrastructure. In the default Docker Compose path, the stack wires the database and Redis URLs for you. All other settings — libraries, metadata providers, transcoding, users — are managed through the admin UI after first launch.

Server Modes

Mode Description
integrated Full server: API + frontend + scanner + transcode (default)
api API server only, no local transcoding
proxy Stream proxy node that connects to the shared deployment database and Redis
transcode HLS transcode worker node that connects to the shared deployment database and Redis

PostgreSQL Auto-Tuning

The default Docker Compose stack does not require a checked-in postgresql.conf. It enables Silo's pgtune-style OLTP tuning by default:

POSTGRES_TUNE: auto

When enabled, Silo connects with DATABASE_URL and applies recommendations with ALTER SYSTEM, which writes to PostgreSQL's postgresql.auto.conf inside the database data directory. Reloadable settings are applied immediately with pg_reload_conf(). Settings that PostgreSQL marks as restart-only are written too, and Silo logs the setting names so you can restart PostgreSQL once:

docker compose restart postgres

The default Compose database user has the required PostgreSQL permissions. If you use an external PostgreSQL server, make sure the configured DATABASE_URL user can run ALTER SYSTEM, or set POSTGRES_TUNE=off and manage PostgreSQL yourself.

For POSTGRES_TUNE_MEMORY=auto, Silo uses the first trustworthy memory source: a finite Docker cgroup limit, the read-only /host/proc/meminfo mount supplied by the bundled Compose file, then /proc/meminfo with container safety guards. Auto-detected memory is treated as a PostgreSQL budget, defaulting to 75% of detected RAM so Silo, Redis, plugins, transcodes, and the OS retain headroom. POSTGRES_TUNE_DB_SIZE=auto queries pg_database_size(current_database()) and classifies the workload by comparing the database size to that memory budget.

Optional tuning overrides:

Variable Default Description
POSTGRES_TUNE_PROFILE oltp Tuning profile. Only oltp is currently supported.
POSTGRES_TUNE_MEMORY auto Server/container RAM, such as 8GB or 32GB; explicit values are used as-is.
POSTGRES_TUNE_MEMORY_BUDGET_PERCENT 75 Percent of auto-detected RAM used for PostgreSQL recommendations.
POSTGRES_TUNE_CPUS auto CPU count used for worker recommendations.
POSTGRES_TUNE_STORAGE ssd One of hdd, ssd, san, or nvme.
POSTGRES_TUNE_DB_SIZE auto Use less_ram when the database comfortably fits in RAM, mid_ram, or greater_ram for very large databases.
POSTGRES_TUNE_CONNECTIONS 100 PostgreSQL max_connections; automatically raised if Silo's app pool is configured higher.
POSTGRES_SHM_SIZE 8gb Docker /dev/shm size for the bundled PostgreSQL container.

Advanced operators can still supply their own PostgreSQL configuration or override these env vars. Set POSTGRES_TUNE=off when you do not want Silo to change PostgreSQL server settings. Settings already written with ALTER SYSTEM remain in postgresql.auto.conf; reset those PostgreSQL parameters if you later move fully to a custom postgresql.conf.

Project Structure

cmd/silo/       Entry point
internal/
  api/               HTTP router, handlers, middleware
  auth/              JWT authentication and sessions
  catalog/           Media item, episode, season repositories
  config/            YAML + env var configuration
  jellycompat/       Jellyfin/Emby protocol compatibility
  metadata/          Plugin-driven metadata matching and enrichment
  playback/          Direct play, remux, transcode session management
  scanner/           Media file discovery and FFProbe
  worker/            Background jobs (scan, match, reconcile)
web/                 React + TypeScript frontend (Vite, Tailwind, shadcn/ui)
migrations/sql/      Goose-managed PostgreSQL schema migrations
S
Description
Self-hosted media streaming server with a Go backend, React web UI, Docker deployment, transcoding, and Jellyfin-compatible APIs.
Readme
340 MiB
Languages
Go 72.6%
TypeScript 26.3%
PLpgSQL 0.4%
CSS 0.3%
Python 0.2%
Other 0.1%