Files
silo-server/cmd/silo/main.go
T
42602b7896 feat(policy): access groups + embedded OPA policy engine with decision audit log (#282)
* docs(policy): add OPA policy engine design spec and implementation plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* build(deps): add OPA v1.18.2 SDK for the policy engine

Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for
the upcoming internal/policy subsystem) and the transitive upgrades go
mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5).
Full build verified.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): add OPA engine core, vendor scope policy, and parity suite

New internal/policy package (dead code — nothing wires into request paths
yet): prepared-query Engine with 25ms eval timeout and fail-closed decode,
typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown
for future admin-authored Rego, and vendor scope.rego reproducing
access.Resolver.Resolve (library intersection, disabled-library handling,
quality/rating ceilings) with a narrowing-only silo_custom.scope.override
extension hook.

Parity proven by 1368 dual-execution subtests against the real
access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and
quality/rating variation; rank tables are test-pinned to internal/access.
Rego unit tests run via opa/v1/tester inside go test. Bench:
~106µs/op per scope decision incl. input marshaling.

Also restores the OPA requirement to go.mod (the earlier deps commit ran
go mod tidy before any import existed, so tidy dropped it).

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed,
corrected (quality.allowed raw-file-rank divergence), and verified here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): add policy document store, foundation schema, and compile-check

policy_foundation migration: policy_documents (one enabled doc per domain
via partial unique index — two enabled docs would define override twice
and conflict at eval), immutable policy_document_versions, single-row
policy_generation counter, and the partitioned policy_decisions log table
(daily range partitions, no FK, denial partial index).

PolicyStore: transactional version numbering (FOR UPDATE), activation
that verifies compiled_ok and bumps the generation in the same tx,
enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for
documents with an active version. CompileCheck sandboxes admin Rego:
locked capabilities (no http.send/net.*/opa.runtime), enforced
silo_custom.<domain> package path, vendor+stub layering, 2s budget,
structured row/col errors. Engine gains NewEngineWithCustom /
NewEngineFromStore with WARN-and-skip for invalid custom rows.

DB-backed tests verified against a migrated Postgres (concurrent version
numbering, atomic generation bumps, activation guards).

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and
verified here (domain constants extracted).

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): add policy System lifecycle with hot reload and cross-node invalidation

policy.System owns one long-lived Engine and reloads it in place when
policy documents change: EventPolicyChanged on the existing ChannelAdmin
bus (new cache event constant) plus a 60s generation-poll fallback for
Redis-less deployments, with a generation-consistent snapshot read.
Vendor compile failure is startup-fatal; store/custom failures degrade
to vendor-only and the poll loop heals them; runtime reload failures
keep the last known-good engine. NotifyChanged gives the future admin
handlers synchronous local reload + cross-node publish.

Wiring: constructed in integrated/api modes only, PolicySystem field on
api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting
(hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a
full server boot smoke and DB-backed convergence tests (event + poll
paths, degraded boot, last-known-good).

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and
verified here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): add async decision logging with sampling, retention, and query repo

DecisionLogger batch-inserts each node's policy decisions straight to
the partitioned policy_decisions table via a non-blocking buffered
channel (drop-and-count on overflow — logging never adds latency to or
fails a decision). Scope decisions sample 1-in-N (default 50, setting
policy.decision_log_scope_sample_rate); denials and eval errors always
log; input/result JSON samples only at policy.decision_log_verbosity=
verbose. Cursor-paginated DecisionRepository backs the upcoming admin
log viewer. Retention via partman (daily partitions) and a
PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days
(default 14). PDP emits entries per evaluation; the System owns the
logger lifecycle and settings hot-reload.

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and
verified here (removed an unused, unsynchronized PDP setter).

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(api): add admin policy management API and capability endpoint

/api/v1/policy/capability (authenticated feature detection) plus the
acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document
CRUD with the one-enabled-per-domain conflict mapped to 409, immutable
version creation (compile-checked; failed versions persist as audit
history with structured row/col errors and can never activate),
activate/rollback with synchronous reload + cross-node invalidation via
System.NotifyChanged, stateless validate, throwaway-bundle simulate
(never touches the live engine, never logs decisions), and
cursor-paginated decision-log queries. Routes mount only when the
policy system is wired, keeping proxy/transcode modes untouched.

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and
verified here (seeded the FK'd test user; replaced an unchecked
fmt.Sscanf with strconv).

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log

New Policy admin page (System nav group): documents list with
one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor
(hand-rolled StreamLanguage mode) with server compile issues rendered as
inline lint diagnostics, explicit Save-version vs Activate flow with
confirm, read-only vendor module viewer, simulate panel with seeded
example inputs, version history with rollback, and a cursor-paginated
decision-log browser. Capability-gated via /policy/capability. Adds the
three decision-log settings to Log Retention. First code-editor
dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in
the design spec.

Implementation drafted by Codex (GPT-5.5) via codex exec; verified here
(lint, format:check, tsc --noEmit, vitest policy suites).

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): make OPA authoritative for viewer scope resolution

policy.ViewerResolver implements the ViewerResolver interface backed by
PDP.ResolveViewerScope and replaces access.Resolver at all five
construction sites: router viewer middleware, notifications scopes, the
reconciler, jellycompat's scope filter, and the ABS resolver (which now
accepts a pre-built resolver, preserving its PIN-at-login semantics).
PIN/profile-token verification and disabled-library loading are
extracted into shared exported helpers used by both implementations, so
the legacy resolver stays compiled as the parity reference with
identical behavior. The adapter lives in internal/policy (which already
depends on internal/access transitively) — direct typed PDP calls, no
new import cycle. Sites without a policy system (proxy modes, bare test
routers) keep the legacy resolver until the cleanup phase.

Verified: full test suite green (jellycompat TestBeginWebOperation* and
one playback GPU test are pre-existing failures, confirmed identical on
main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/
nil-vs-empty/fail-closed tests, and a full server boot smoke.

Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass
reflection-based adapter was rejected and reworked into the typed
in-policy adapter; reviewed line-by-line and verified here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): make OPA authoritative for acting-admin and permission gates

vendor/permission.rego reproduces the acting-admin rule (admin role +
primary-profile-or-none), HasEffectivePermission semantics for
marker_edit, and the metadata-curation rule including the subtle
admin-past-refused-bypass case that requires the explicitly ASSIGNED
permission. Policy-backed middleware in policy_gates.go keeps all Go-side
lookups (declared-profile primary check, item->library resolution, the
404-on-unknown-item path) and preserves the legacy status/body taxonomy
exactly — proven by dual-execution middleware tests that run every
scenario through both implementations and assert byte-equal responses.
Permission decisions always log (allowed flag populated); simulate and
the capability endpoint gain the permission domain automatically via the
domain registry. Router swaps behind single constructor choice points
with the legacy gates retained for policy-less wiring.

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and
verified here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(policy): make OPA authoritative for download and playback admission decisions

vendor/action.rego decides download eligibility (downloads enabled +
user allowed), download-transcode eligibility (transcode enabled + user
allowed + artifacts available), and playback admission (stream/transcode
counts vs limits, zero = unlimited), with a tightening-only
silo_custom.action override that can also clamp a quality ceiling (never
widen — merged via quality.min). Go keeps everything stateful: config
loading, preset-ladder enumeration, and live session counting.

Downloads consult an optional ActionDecider (nil = legacy logic) mapped
back to the existing sentinel errors and capability response. Playback
gains a minimal AdmissionDecider hook at the exact point of the legacy
limit comparison: counts snapshot under the session mutex, PDP evaluated
OUTSIDE the lock, then revalidated under lock before insert (retry on
count drift) — no admission ever decided on stale counts and no eval
under the mutex. Deny reasons map to the legacy ErrTooManyStreams /
ErrTooManyTranscodes sentinels, pinned by tests.

Parity: combination tables driven against the real PresetsFor /
ensureTranscodeAllowed / SessionLimits math; full suite green (known
pre-existing jellycompat flakes only).

Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed
(locking design verified line-by-line) and verified here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer

The production build (tsc -b) rejects assigning CodeMirror's
string | void next() result to string | undefined; tsc --noEmit did not
catch it. Restructured the string-literal loop.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): clearer error when a decision is undefined for partial input

Vendor policies index required input fields directly, so a hand-written
simulate payload missing fields yields an undefined decision. Surface
that as 'decision X is undefined for this input (missing required input
fields?)' instead of 'empty result' — found while exercising the
simulate API against a live server.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(web): set changeOrigin automatically when the API proxy target is remote

Remote dev backends sit behind vhost-routing proxies that reject a
localhost Host header; local targets keep the existing pass-through
behavior. Enables pointing the Vite dev server at a hosted backend via
VITE_API_PROXY_TARGET in web/.env.local.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(web): redesign the policy workspace around the decision pipeline

The first-pass UI was structurally generic: a five-column document table
squeezed beside the editor, three equal-weight action buttons with
hidden preconditions, raw version IDs, and jargon copy — nothing taught
the model. The page now teaches it:

- A pipeline strip states the mental model up front: Silo decides the
  baseline -> your overrides narrow it -> every decision is logged. Tabs
  renamed to Overrides / Baseline / Decision Log (ids stay stable for
  bookmarked URLs).
- The document table becomes one card per domain (Library visibility /
  Admin & permissions / Downloads & playback) with plain-language
  descriptions, example rules, status pills (Live vN / Draft / Disabled),
  inline creation, and the enable kill-switch in place.
- Selecting an override drills into a full-width editor with a visible
  lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual
  primary action per step; the unedited live source shows no actions
  until edited. Version comments appear only at the save step.
- Simulate is reframed as 'Test before going live' with a human verdict
  chip (Allowed / Denied — reason / ceiling summary) above the raw JSON;
  internal generation counters no longer surface.
- History uses 'Make live' with plain go-live copy; authors read
  'User N'; the baseline tab explains that upgrades never touch
  overrides.

Hand-written redesign (no Codex); verified via vitest, tsc, eslint,
prettier, and a production build.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(web): present the policy baseline as readable rules, not raw Rego

The Baseline tab dumped five Rego modules into read-only editors. It now
leads with what the rules actually do: one card per domain with
plain-language statements of the shipped behavior and a note on what an
override may change, plus content-rating and playback-quality tier
ladders parsed live from the lib module sources (so the tiers shown are
the ones the server enforces, not a hardcoded copy). The Rego source
stays one click away behind a per-module accordion and remains the
stated source of truth; unrecognized modules fall back to source-only.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(policy): add access-groups design addendum

Groups with permission toggles become the everyday admin surface; the
Rego editor is demoted behind policy.editor_enabled (default off).
Restriction-only composition: group grants are an upper bound, per-user
settings tighten further — same rule as the existing account/profile
merge, one layer up.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(access): add access groups — group defaults with restriction-only composition

New access_groups table + users.access_group_id (one group per user, NULL
= today's behavior). Group grants are an upper bound composed with the
user's own settings by strictest-wins rules — library intersection,
MinQuality, AND'd booleans, strictest positive stream/transcode limits,
permission-mask intersection, and a requests toggle gating CreateRequest.
The merge happens in Go (access.ApplyGroupPolicy /
EffectivePolicyForUser) before policy inputs are built, so vendor Rego,
the parity suites, and the decision log are untouched; every enforcement
surface (viewer scope in both resolvers, permission gates, downloads,
playback admission, requests) consumes the effective policy and fails
closed on provider errors. Changing a group's quality ceiling bumps its
members' access_policy_revision, mirroring the per-user rule.

Additive admin API: /admin/access-groups CRUD with member counts;
PUT /admin/users/{id} + user DTOs gain access_group_id.

Also demotes the Rego editor: policy.editor_enabled (default off,
hot-reloaded) drives the capability endpoint's editor_available and
403-gates editor endpoints while the engine and decision logging keep
running.

Design: docs/superpowers/specs/2026-07-02-access-groups-design.md.
Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed
(composition core + fail-closed call-site audit) and verified here.
DB-backed group-store tests pending local Postgres recovery.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(web): add Access Groups admin page and gate the policy editor

New /admin/access-groups: a card grid summarizing each group (member
count + key restrictions), drilling into an editor that reuses the same
LibraryAccessSelector and quality presets as the user editor, with
toggles for downloads/transcoded-downloads/requests, concurrent-stream
and transcode limits, and a permissions mask (all-assignable by default,
narrowable to specific permissions). Delete warns how many members fall
back to the built-in defaults. Copy states the composition rule up front:
a group grants the most a member can do; their own restrictions still
apply on top.

The user editor gains a Group picker and read-only row; the Policy nav
entry is now hidden unless the capability reports the editor enabled.
Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex
(GPT-5.5); the Groups page hand-built. Verified: 25 tests across the
touched suites, tsc, eslint, prettier, and a production build.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(access): seed a Default Group and auto-assign newly created users

Adds access_groups.is_default with a partial unique index (one default
at most — the profiles is_primary pattern) and seeds a permissive
'Default Group' whose ceiling is a no-op, so assignment never changes
anyone's effective access until an admin edits it. The seed is guarded
against pre-existing defaults and name collisions; the Down migration
only removes the row if it is still untouched.

Assignment happens at the single INSERT INTO users choke point
(UserRepository.Create): when no explicit group is given, access_group_id
is filled by a scalar subquery on the default flag — NULL when no default
exists. Every creation path (setup, signup, invites, OAuth, admin create)
is covered by construction. Setting a new default via the API atomically
clears the previous one in the same transaction.

Deleting or unsetting the default is legal: new users then start with no
group, which is pre-feature behavior.

Implementation drafted by Codex (GPT-5.5); migration guards and the
choke-point subquery reviewed line-by-line here.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(web): surface the default access group

Cards show a Default badge; the group editor gains a 'Default for new
users' toggle (with copy noting existing users are never moved); the
delete dialog warns when removing the default that new accounts will
start with no group.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(access): ship the Default Group with house-rule ceilings

Seed values per product decision: 5 concurrent streams, 5 transcodes,
transcoded downloads off, and a permission mask of marker_edit only
(metadata curation excluded). Plain downloads and requests stay on. The
Down guard matches the new values so it still only removes an untouched
seed row. Only newly created users are affected; existing users are
never assigned.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(access): retire per-user defaults — the Default Group is the sole default policy

Removes both legacy 'user defaults' mechanisms now that the seeded
Default Group owns new-user policy:

- users.max_streams / max_transcodes column defaults drop from 6/2 to 0
  (= unrestricted at the user layer), so group ceilings apply to new
  signups/invites/OAuth users instead of fighting stale per-user
  numbers. Existing rows keep their stored values — nobody is silently
  uncapped on upgrade.
- The dead defaults.max_playback_quality / defaults.max_profiles
  settings validation goes away with its only writer (the User Defaults
  dialog, removed on the web side).

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(web): replace the User Defaults dialog with group-governed creation

The Users page's 'User Defaults' dialog (defaults.* server settings)
duplicated what access groups now do properly, and its values were only
ever form prefill — no backend path applied them. The button now links
to Access Groups, and the create-user form seeds unrestricted user-layer
values (0 streams/transcodes, any quality, downloads allowed) so the
member's group governs; per-user fields remain for tightening individual
users.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(access): migrate existing non-admin users into the Default Group

Existing users join the seeded Default Group on upgrade so one policy
source governs the whole instance. Their per-user limits still holding
the retired 6/2 column defaults are normalized to 0 in the same
statement so the group's ceilings actually apply; deliberately
customized values are preserved. Admin accounts stay ungrouped —
scope/action decisions are role-blind, so grouping an admin would cap
the server owner on upgrade.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(access): keep admins out of the Default Group and treat group moves as policy changes

New-user creation now mirrors the migration's admin exclusion: the
default access group is only auto-assigned to non-admin roles, so a
fresh server owner no longer inherits the starter group's transcode
denial and stream caps.

Changing a user's access group now bumps access_policy_revision (the
group carries permissions, quality, and limits, exactly like the
per-user fields that already bump it) and triggers admin session
revocation when the group actually changes.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): enforce marker_edit through the PDP on marker write routes

The Rego permission policy owned marker_edit but no Go caller ever
consulted it: PUT/DELETE /markers went through a handler-local check
that short-circuited admins and read only the user's own permissions,
so group permission masks and custom policy overrides were ignored.

Marker writes are now gated by router middleware like the other
permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the
group-merged effective permissions (plus the legacy variant for
proxy/test wiring without a policy system). The handler-local check and
its user loader are gone.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): assert device/quality policy facts and honor the quality ceiling

The download_transcode action check hard-coded an empty device ID and
never asserted the requested quality, and no caller consumed
ActionDecision.QualityCeiling — custom download policies keyed on those
inputs were silently ineffective.

Resolve now threads the request's device ID and requested quality into
the action input, and a returned quality ceiling downscales the
prepared transcode target (the ceiling applies to what is served,
matching the serve-time rule in serveDownloadBytes). FileQuality and
the content-rating pair stay intentionally empty for downloads —
documented on downloadActionInput: those ceilings are enforced against
the served artifact by the scope-derived access filter, and asserting
the source's quality would wrongly deny capped transcodes.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(access): align the default-group seed assertions with the migration

The DB test still asserted the earlier no-op seed (transcode allowed,
unlimited streams/transcodes, null permissions); the shipped migration
seeds transcode denied, 5/5 limits, and marker_edit-only permissions,
so the test failed on any database with the migration applied.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): lock the Rego sandbox by builtin purity and bound compile work

Exclude every nondeterministic builtin from the admin sandbox instead of
denylisting names, so OPA upgrades cannot silently expose impure builtins
while pure helpers like net.cidr_contains stay usable. Apply the same
capabilities to the runtime engine, cap concurrent compile checks, and
reject oversized sources before they reach the uncancelable compiler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): require literal booleans in vendor override and input checks

Bare object.get truthiness treated any non-false value as satisfied, so a
malformed override 'allowed' value could fail to tighten a base grant and
hand-crafted simulate input could flip flag predicates. Compare against
literal true so anything else denies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): surface decision log cleanup failures to the task manager

CleanupDecisionLogsOnce now returns the first error alongside the deleted
count so a broken partition manager or DB outage marks the scheduled task
failed instead of reporting 100% success while policy_decisions grows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): log admission decider errors before failing closed

A policy-evaluation failure was silently mapped to the too-many-streams
denial, making an engine outage indistinguishable from a real limit hit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(access): nil-guard the downloads user and restore the ABS legacy resolver

effectiveDownloadUser dereferenced policy state before its nil-user check,
and the ABS handler lost viewer-scoped filtering entirely when the policy
system was unavailable because no legacy access.NewResolver fallback was
wired like the other resolver paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(web): address admin policy review feedback

- invalidate the version query by version_number, the key usePolicyVersion
  actually caches under
- keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke)
- make version history rows keyboard-selectable like the document list
- clamp download_transcode_allowed when downloads are disabled so groups
  cannot save a contradictory record

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api): cap policy endpoint request bodies at 1 MiB

The policy write endpoints (create document/version, set enabled,
validate, simulate) decoded JSON bodies without a size limit, so an
oversized payload buffered fully in memory before CompileCheck's
256 KiB source cap could reject it. Route all five through a shared
decodePolicyRequest helper that wraps the body in http.MaxBytesReader
and returns 413 with the repo's standard too_large error shape.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN

* fix(access): forbid deleting or demoting the default access group

Deleting the default group (or unsetting its is_default flag) left the
server with no default: new non-admin users were then created ungrouped
with max_streams/max_transcodes of 0 — unlimited — because the legacy
per-user column defaults were retired in favor of the group's ceilings.

The store now rejects both operations with ErrDefaultGroupRequired
(mapped to 409); promoting another group remains the supported way to
move the default, and atomically clears the previous one. The admin UI
disables the delete button and the default toggle on the default group
and explains the promote-another-group flow.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN

* fix(web): keep unsaved policy drafts when a newer version activates elsewhere

The editor state was keyed on the active version's id/sha, so a
background refetch after another admin (or another tab) activated a
version remounted the editor and silently discarded the dirty draft.

PolicyEditorPanel now pins the seed it is editing against and only
adopts an incoming seed when nothing can be lost: the editor is clean,
the draft already equals the incoming source (the same-admin activate
flow), or the selection moved to a different document. Otherwise the
pinned editor stays mounted and an inline notice offers an explicit
"Load live version" action.

Part of #272

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN

* fix(policy): fail reloads on invalid custom sources and surface degraded/apply state

A stored custom source that stops compiling used to be silently skipped on
reload: the bundle widened to vendor-only for that domain while the generation
reported fully applied. Reload is now strict — a bad enabled source fails the
reload and the last known-good engine keeps serving. Boot keeps its vendor
fallback for availability, but skips are recorded on the engine and exposed
(with store-outage reasons) through System.DegradedState and additive
degraded fields on GET /policy/capability. Activate/SetEnabled re-run
CompileCheck instead of trusting the stored compiled_ok flag.

Mutation endpoints also no longer conflate persistence with live apply:
activation/enable responses carry additive applied/failed_step/
loaded_generation fields and return 202 when the store change persisted but
the local reload failed.

Addresses review findings C1, C2, and the degraded-signal gap (6.1).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): type deny reasons across the contract and enforce profile_verified

Deny handling used to branch on exact free-text reason strings in three Go
consumers, and playback reported ANY unrecognized reason — including custom
override free text and engine failures — as a stream-limit error. Decisions
now carry a stable reason_code (custom overrides always get custom_denial);
downloads, the metadata-curation gate, and playback admission switch on codes,
with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for
non-limit denials. Rego tests pin every vendor code.

The scope contract's tighten-only profile_verified output was also emitted but
never consumed; a policy revocation now surfaces as ErrProfileUnverified (403
profile_unverified) instead of silently proceeding.

Addresses review findings 6.2 and C4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(catalog): close the dual-library disabled-scope bypass in direct item authorization

EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated
library access with allow/deny predicates over a single joined
media_item_libraries row, so an item linked to BOTH a passing library and a
disabled one satisfied the disabled check via the passing row — a direct-ID
bypass of disabled-library scope on the detail, media-file, playback, and
download paths. All library access predicates now share one helper
(libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries,
the semantics GetByIDsWithAccess already used, including the orphan-item
membership guard for disabled-only scopes. SQL-shape tests pin every builder
and a DB-gated regression test covers the dual-library item end to end.

Addresses review finding C3 (plus the same shape in
buildFilterAccessibleContentIDsSQL, which the review did not flag).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): serialize quota check and row creation under a per-user advisory lock

The concurrent-download quota was check-then-insert with nothing serializing
the pair: parallel creates could all observe free quota before any row
existed, bypassing the cap and stacking artifact encode jobs. All four
check->insert spans (ephemeral original, artifact-backed, series batch,
managed batch) now run inside Repository.WithUserQuotaLock — a
pg_advisory_xact_lock keyed by user, so the serialization holds across nodes.
The artifact path keeps the limiter-before-Ensure ordering (a rejected request
must not leave an encode job behind) by holding the lock across Ensure.
Managed-entry replacement stays quota-exempt and lock-free. A DB-gated
barrier test races 8 creates against a cap of 1.

Addresses review finding C5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): assert served quality at create time for original and remux downloads

Direct-original and remux downloads serve the source resolution unchanged, but
create-time policy checks left file_quality empty — an over-ceiling source
registered a row serveDownloadBytes could never satisfy. Resolve now runs a
final download action check with FileQuality populated on those two paths
(capped transcodes keep the ceiling-on-artifact behavior), a custom override
ceiling below the served resolution denies, and quality_ceiling_exceeded maps
to ErrQualityUnavailable. The ActionInput contract now documents exactly when
file_quality and the rating facts are supplied so custom policy authors are
not misled.

Addresses review finding C6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(policy): guard activation against slow overrides and make eval timeouts observable

A custom scope override that exceeds the 25ms eval budget compiled fine,
activated fine, and then converted to 500s on every authenticated request —
server-wide lockout authored in the admin editor. Activation and enable now
run GuardEvalCost: the candidate source is evaluated on a throwaway engine
against a canned representative input under the live budget, and a source
that cannot complete is rejected 422 with ErrPolicySlowEval before it goes
live. Runtime timeouts keep failing closed but now carry a distinct
ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed
as eval_timeouts on GET /policy/capability so intermittent near-budget
policies are attributable.

Addresses review finding C7.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style: gofmt remediation files

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 17:38:19 -04:00

3111 lines
113 KiB
Go

package main
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"flag"
"fmt"
"io"
"io/fs"
"log"
"log/slog"
"net/http"
"os"
"os/signal"
"path/filepath"
"runtime/debug"
"sort"
"strconv"
"strings"
"sync/atomic"
"syscall"
"time"
"github.com/go-chi/chi/v5"
chimiddleware "github.com/go-chi/chi/v5/middleware"
"github.com/google/uuid"
"github.com/hashicorp/go-hclog"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/prometheus/client_golang/prometheus/promhttp"
pluginv1 "github.com/Silo-Server/silo-plugin-sdk/pkg/pluginproto/silo/plugin/v1"
sdkcapability "github.com/Silo-Server/silo-plugin-sdk/pkg/pluginsdk/capability"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/activitylog"
"github.com/Silo-Server/silo-server/internal/adminjob"
"github.com/Silo-Server/silo-server/internal/api"
"github.com/Silo-Server/silo-server/internal/api/handlers"
"github.com/Silo-Server/silo-server/internal/audiobooks"
"github.com/Silo-Server/silo-server/internal/audiobooks/podcastfeed"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/autoscan"
"github.com/Silo-Server/silo-server/internal/branding"
"github.com/Silo-Server/silo-server/internal/cache"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/catalogseed"
"github.com/Silo-Server/silo-server/internal/chapterthumbs"
"github.com/Silo-Server/silo-server/internal/clientip"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/database"
"github.com/Silo-Server/silo-server/internal/downloads"
"github.com/Silo-Server/silo-server/internal/ebooks"
evt "github.com/Silo-Server/silo-server/internal/events"
"github.com/Silo-Server/silo-server/internal/historyimport"
"github.com/Silo-Server/silo-server/internal/imagecache"
"github.com/Silo-Server/silo-server/internal/intromarkers"
"github.com/Silo-Server/silo-server/internal/jellycompat"
"github.com/Silo-Server/silo-server/internal/libraryingest"
"github.com/Silo-Server/silo-server/internal/literaryworks"
"github.com/Silo-Server/silo-server/internal/logfilter"
"github.com/Silo-Server/silo-server/internal/logstream"
"github.com/Silo-Server/silo-server/internal/mail"
"github.com/Silo-Server/silo-server/internal/manga"
"github.com/Silo-Server/silo-server/internal/markers"
"github.com/Silo-Server/silo-server/internal/mdblist"
"github.com/Silo-Server/silo-server/internal/metadata"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/nodeconfig"
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/noderecipe"
"github.com/Silo-Server/silo-server/internal/nodesessions"
"github.com/Silo-Server/silo-server/internal/notifications"
"github.com/Silo-Server/silo-server/internal/opslog"
"github.com/Silo-Server/silo-server/internal/partman"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/pluginhost"
"github.com/Silo-Server/silo-server/internal/plugins"
"github.com/Silo-Server/silo-server/internal/policy"
"github.com/Silo-Server/silo-server/internal/proxy"
"github.com/Silo-Server/silo-server/internal/ratelimit"
"github.com/Silo-Server/silo-server/internal/recommendations"
mediarequests "github.com/Silo-Server/silo-server/internal/requests"
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/scanner"
"github.com/Silo-Server/silo-server/internal/scanqueue"
"github.com/Silo-Server/silo-server/internal/secret"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/server"
"github.com/Silo-Server/silo-server/internal/subtitles"
"github.com/Silo-Server/silo-server/internal/taskmanager"
taskrepository "github.com/Silo-Server/silo-server/internal/taskmanager/repository"
"github.com/Silo-Server/silo-server/internal/taskmanager/tasks"
"github.com/Silo-Server/silo-server/internal/taskmanager/triggers"
"github.com/Silo-Server/silo-server/internal/transcodenode"
"github.com/Silo-Server/silo-server/internal/usercollections"
"github.com/Silo-Server/silo-server/internal/userdb"
"github.com/Silo-Server/silo-server/internal/userstore"
"github.com/Silo-Server/silo-server/internal/userstore/pgstore"
"github.com/Silo-Server/silo-server/internal/watchlist"
"github.com/Silo-Server/silo-server/internal/watchstate"
"github.com/Silo-Server/silo-server/internal/watchsync"
watchmdblist "github.com/Silo-Server/silo-server/internal/watchsync/providers/mdblist"
"github.com/Silo-Server/silo-server/internal/watchsync/providers/simkl"
"github.com/Silo-Server/silo-server/internal/watchsync/providers/trakt"
"github.com/Silo-Server/silo-server/internal/worker"
"github.com/Silo-Server/silo-server/migrations"
siloweb "github.com/Silo-Server/silo-server/web"
)
// resolveNodeIdentity returns a stable node identifier used by the
// heartbeat writer, reconciler, and shutdown cleanup. Resolution order:
// SILO_NODE_NAME > NODE_NAME > os.Hostname().
func resolveNodeIdentity() string {
if v := os.Getenv("SILO_NODE_NAME"); v != "" {
return v
}
if v := os.Getenv("NODE_NAME"); v != "" {
return v
}
h, _ := os.Hostname()
return h
}
func resolvePluginCacheDir() string {
if v := strings.TrimSpace(os.Getenv("SILO_PLUGIN_CACHE_DIR")); v != "" {
return v
}
return filepath.Join(os.TempDir(), "silo-plugins")
}
func buildBaseHandler(format string, level slog.Leveler) slog.Handler {
opts := &slog.HandlerOptions{Level: level}
if strings.EqualFold(format, "json") {
return slog.NewJSONHandler(os.Stderr, opts)
}
return slog.NewTextHandler(os.Stderr, opts)
}
func parseLogLevel(level string) slog.Level {
switch strings.ToLower(level) {
case "debug":
return slog.LevelDebug
case "warn", "warning":
return slog.LevelWarn
case "error":
return slog.LevelError
default:
return slog.LevelInfo
}
}
func mustGetSetting(store interface {
Get(context.Context, string) (string, error)
}, ctx context.Context, key, fallback string) string {
value, err := store.Get(ctx, key)
if err != nil || strings.TrimSpace(value) == "" {
return fallback
}
return value
}
func configureOperationalLogging(
ctx context.Context,
pool *pgxpool.Pool,
settingsRepo catalog.SettingsStore,
redisCfg config.RedisConfig,
logStreamHub *logstream.Hub,
filteredHandler slog.Handler,
nodeID string,
) (opslog.Writer, *opslog.Repo, *partman.Manager) {
if err := opslog.SeedDefaults(ctx, settingsRepo); err != nil {
log.Fatalf("seed opslog defaults: %v", err)
}
opsPM := partman.NewManager(pool, "operational_logs", partman.Daily, 3)
if err := opsPM.EnsureFuturePartitions(ctx); err != nil {
// Non-fatal: a partition hiccup must not crash-loop the server (see the
// operational_logs partition incident). Writes fall back to the default
// partition and the periodic cleanup retries EnsureFuturePartitions.
slog.Warn("ensure operational log partitions; continuing in degraded mode", "error", err)
}
var operationalWriter opslog.Writer
operationalConsumer := opslog.NewConsumer(pool, nil, logStreamHub)
if redisCfg.URL != "" {
redisClient, redisErr := cache.NewRedisClient(redisCfg)
if redisErr == nil && redisClient != nil {
operationalWriter = opslog.NewRedisWriter(redisClient)
operationalConsumer = opslog.NewConsumer(pool, redisClient, logStreamHub)
go operationalConsumer.RunRedis(ctx)
}
}
if operationalWriter == nil {
memWriter := opslog.NewMemoryWriter(10000)
operationalWriter = memWriter
go operationalConsumer.RunMemory(ctx, memWriter.Chan())
}
opsCaptureLevel := slog.LevelInfo
switch strings.ToLower(strings.TrimSpace(mustGetSetting(settingsRepo, ctx, "opslog.capture_level", "info"))) {
case "debug":
opsCaptureLevel = slog.LevelDebug
case "warn", "warning":
opsCaptureLevel = slog.LevelWarn
case "error":
opsCaptureLevel = slog.LevelError
}
slog.SetDefault(slog.New(opslog.NewHandler(filteredHandler, operationalWriter, opsCaptureLevel, nodeID)))
return operationalWriter, opslog.NewRepo(pool), opsPM
}
func maybeApplyPostgresTuning(ctx context.Context, pool *pgxpool.Pool, appMaxConnections int, mode string) {
switch strings.ToLower(strings.TrimSpace(mode)) {
case "", "integrated", "api":
default:
return
}
opts, err := database.LoadPostgresTuneOptionsFromEnv(appMaxConnections)
if err != nil {
slog.Warn("postgres auto-tuning disabled", "error", err)
return
}
if !opts.Enabled {
return
}
tuneCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
result, err := database.ApplyPostgresTuning(tuneCtx, pool, opts)
for _, failure := range result.Failures {
slog.Warn("postgres auto-tuning setting failed",
"name", failure.Name,
"value", failure.Value,
"error", failure.Err,
)
}
if err != nil {
slog.Warn("postgres auto-tuning failed",
"error", err,
"applied", result.Applied,
"failures", len(result.Failures),
)
return
}
slog.Info("postgres auto-tuning applied",
"profile", opts.Profile,
"postgres_major", result.PostgresMajorVersion,
"settings", result.Applied,
"resets", len(result.Reset),
"failures", len(result.Failures),
"memory_budget_bytes", opts.MemoryBudgetBytes,
"detected_memory_bytes", opts.DetectedMemoryBytes,
"memory_source", opts.MemorySource,
"memory_budget_percent", opts.MemoryBudgetPercent,
"cpus", opts.CPUs,
"connections", opts.Connections,
"storage", opts.Storage,
"db_size", result.DBSize,
"database_size_bytes", result.DatabaseSizeBytes,
)
if len(result.RestartRequired) > 0 {
slog.Warn("postgres restart required to finish applying auto-tuned settings",
"settings", strings.Join(result.RestartRequired, ","),
)
}
if len(result.Reset) > 0 {
slog.Info("postgres auto-tuning reset stale settings",
"settings", strings.Join(result.Reset, ","),
)
}
}
// runCredentialBackfills sweeps any plaintext server-owned credential to
// ciphertext on the primary (migration-running) node. All passes are
// best-effort: a failed row leaves the prior plaintext (no new exposure) and
// still reads via the read-path pass-through, so a backfill error must never
// block boot. The sensitive-settings pass runs first so the arr
// resolve-then-encrypt pass sees consistent referenced settings.
func runCredentialBackfills(ctx context.Context, pool *pgxpool.Pool, cipher *secret.Cipher, settings *catalog.EncryptedSettingsRepo) {
settingsN, err := settings.BackfillSensitiveSettings(ctx)
if err != nil {
slog.Error("secret backfill: sensitive settings", "error", err)
}
columnsN, err := secret.BackfillColumns(ctx, pool, cipher, secret.ColumnBackfillTargets())
if err != nil {
slog.Error("secret backfill: credential columns", "error", err)
}
historyServersN, err := historyimport.NewRepository(pool, cipher).BackfillSessionServerSecrets(ctx)
if err != nil {
slog.Error("secret backfill: history import session server credentials", "error", err)
}
// The arr resolver is the encrypting settings decorator: it decrypts a
// sensitive target (e.g. requests.radarr.api_key) or passes through a
// plaintext custom key, exactly replicating the deleted resolveAPIKey.
arrN, err := secret.BackfillReferencedColumns(ctx, pool, cipher, settings.Get, secret.ArrKeyBackfillTargets())
if err != nil {
slog.Error("secret backfill: arr api keys", "error", err)
}
if total := settingsN + columnsN + historyServersN + arrN; total > 0 {
slog.Info("secret backfill: encrypted plaintext credentials at rest",
"settings", settingsN, "columns", columnsN, "history_session_servers", historyServersN, "arr_keys", arrN, "total", total)
}
}
func runCompatWebCommand(ctx context.Context, args []string) error {
if len(args) == 0 {
return fmt.Errorf("usage: silo compat-web {status|install|update|remove}")
}
command := args[0]
flags := flag.NewFlagSet("compat-web "+command, flag.ContinueOnError)
flags.SetOutput(io.Discard)
root := flags.String("dir", config.DefaultJellyfinWebInstallDir, "Jellyfin Web component install root")
version := flags.String("version", config.DefaultJellyfinWebVersion, "Jellyfin Web version without leading v")
source := flags.String("source", jellycompat.DefaultWebSourceURL, "upstream jellyfin-web git repository")
if err := flags.Parse(args[1:]); err != nil {
return err
}
switch command {
case "status":
status := jellycompat.WebComponentStatusForConfig(&config.Config{
JellyfinCompat: config.JellyfinCompatConfig{
Enabled: false,
WebVersion: *version,
WebInstallDir: *root,
WebDir: filepath.Join(*root, "current"),
},
}, map[string]string{
"jellyfin_compat.web_source_url": *source,
})
return json.NewEncoder(os.Stdout).Encode(status)
case "install", "update":
status, err := jellycompat.InstallWebComponent(ctx, jellycompat.WebComponentInstallOptions{
InstallRoot: *root,
SourceURL: *source,
Version: *version,
})
_ = json.NewEncoder(os.Stdout).Encode(status)
return err
case "remove":
return jellycompat.RemoveWebComponent(*root)
default:
return fmt.Errorf("unknown compat-web command %q", command)
}
}
func main() {
if len(os.Args) > 1 && os.Args[1] == "compat-web" {
if err := runCompatWebCommand(context.Background(), os.Args[2:]); err != nil {
log.Fatalf("compat-web: %v", err)
}
return
}
envFile := flag.String("env", ".env", "path to .env bootstrap file")
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
migrateStatus := flag.Bool("migrate-status", false, "show database migration status and exit")
flag.Parse()
ctx := context.Background()
// Step 1: Bootstrap from .env
bc, err := config.LoadBootstrap(*envFile)
if err != nil {
log.Fatalf("bootstrap: %v", err)
}
// Construct the at-rest credential cipher from SECRET_KEY immediately after
// bootstrap, before any settings repo is built. It is threaded explicitly as
// a dependency into every repo that stores a server-owned secret — never a
// package-level global.
dataCipher, err := secret.New(bc.SecretKey)
if err != nil {
log.Fatalf("secret cipher: %v", err)
}
// Step 2: Connect to PostgreSQL (bootstrap pool with default max connections)
bootstrapDBCfg := config.DatabaseConfig{URL: bc.DatabaseURL, MaxConnections: 20}
pool, err := database.NewPool(ctx, bootstrapDBCfg)
if err != nil {
log.Fatalf("database pool: %v", err)
}
defer pool.Close()
slog.Info("connected to PostgreSQL")
if *migrateStatus {
migCtx, migCancel := database.MigrationContext(ctx)
statuses, statusErr := database.MigrationStatuses(migCtx, pool, migrations.FS, "sql")
migCancel()
if statusErr != nil {
log.Fatalf("failed to read migration status: %v", statusErr)
}
fmt.Printf("%-8s %8s %-25s %s\n", "STATE", "VERSION", "APPLIED_AT", "MIGRATION")
for _, status := range statuses {
appliedAt := "-"
if !status.AppliedAt.IsZero() {
appliedAt = status.AppliedAt.UTC().Format(time.RFC3339)
}
source := status.Source
if source != "" {
source = filepath.Base(source)
} else {
source = "-"
}
fmt.Printf("%-8s %8d %-25s %s\n", status.State, status.Version, appliedAt, source)
}
return
}
if *migrateOnly {
migCtx, migCancel := database.MigrationContext(ctx)
migErr := database.RunMigrations(migCtx, pool, migrations.FS, "sql")
migCancel()
if migErr != nil {
log.Fatalf("failed to run migrations: %v", migErr)
}
slog.Info("database migrations applied")
return
}
// Run migrations only for integrated/api modes. Proxy and transcode nodes
// should never alter the schema — they may scale independently and would
// race or apply migrations before the primary node is deliberately upgraded.
// The same gate decides whether this node runs the credential-encryption
// backfills: only the primary (migration-running) node sweeps plaintext to
// ciphertext; secondary nodes read whatever the primary encrypted.
isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == ""
if isPrimaryNode {
migCtx, migCancel := database.MigrationContext(ctx)
if migErr := database.RunMigrations(migCtx, pool, migrations.FS, "sql"); migErr != nil {
migCancel()
log.Fatalf("failed to run migrations: %v", migErr)
}
migCancel()
slog.Info("database migrations applied")
}
// Step 3: Load settings from DB. settingsRepo is the encrypting decorator so
// every consumer (config.LoadFromDB, admin, ABS, watchers) transparently sees
// plaintext while sensitive keys rest as ciphertext. The settings backfill
// (run after migrations, before this GetAll) is wired further below.
settingsRepo := catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
if isPrimaryNode {
runCredentialBackfills(ctx, pool, dataCipher, settingsRepo)
}
settings, err := settingsRepo.GetAll(ctx)
if err != nil {
log.Fatalf("loading settings: %v", err)
}
// Step 4: YAML import (one-time)
yamlPath := "silo.yaml"
if _, yamlErr := os.Stat(yamlPath); yamlErr == nil {
if settings["_yaml_imported"] == "" {
yamlSettings, importErr := config.YAMLToSettingsMap(yamlPath)
if importErr != nil {
log.Printf("WARN: could not import YAML config: %v", importErr)
} else {
for k, v := range yamlSettings {
if err := settingsRepo.Set(ctx, k, v); err != nil {
log.Printf("WARN: failed to import setting %s: %v", k, err)
}
}
if err := settingsRepo.Set(ctx, "_yaml_imported", "true"); err != nil {
slog.Warn("failed to set yaml import flag", "error", err)
}
log.Println("Imported config from silo.yaml — this file is no longer used")
settings, _ = settingsRepo.GetAll(ctx)
}
}
}
// Step 5: Auto-generate secrets
if settings["auth.jwt_secret"] == "" {
secret := make([]byte, 32)
if _, err := rand.Read(secret); err != nil {
log.Fatalf("generating jwt secret: %v", err)
}
encoded := base64.StdEncoding.EncodeToString(secret)
if err := settingsRepo.Set(ctx, "auth.jwt_secret", encoded); err != nil {
slog.Warn("failed to persist generated JWT secret", "error", err)
}
settings["auth.jwt_secret"] = encoded
}
if settings["jellyfin_compat.server_id"] == "" {
serverID := uuid.NewSHA1(uuid.NameSpaceURL, []byte("https://silo.local/jellycompat")).String()
if err := settingsRepo.Set(ctx, "jellyfin_compat.server_id", serverID); err != nil {
slog.Warn("failed to persist generated server ID", "error", err)
}
settings["jellyfin_compat.server_id"] = serverID
}
// Step 6: Build config from DB
cfg, err := config.LoadFromDB(settings)
if err != nil {
log.Fatalf("building config: %v", err)
}
// Step 7: Apply bootstrap overrides
cfg.Server.Listen = bc.Listen
cfg.Server.Mode = bc.Mode
cfg.Database.URL = bc.DatabaseURL
cfg.JellyfinCompat.Listen = bc.JFListen
if bc.RedisURL != "" {
cfg.Redis.URL = bc.RedisURL
}
// Step 8: Recreate pool if max_connections differs from bootstrap default
if cfg.Database.MaxConnections != bootstrapDBCfg.MaxConnections {
pool.Close()
pool, err = database.NewPool(ctx, cfg.Database)
if err != nil {
log.Fatalf("recreating pool with configured max_connections: %v", err)
}
}
// Re-wrap with the encrypting decorator so the recreated pool's settings repo
// still encrypts/decrypts — no raw settings repo may escape into later wiring.
settingsRepo = catalog.NewEncryptedSettingsRepo(catalog.NewServerSettingsRepo(pool), dataCipher)
nodeID := resolveNodeIdentity()
catalogSearchStartupSettings, err := catalog.CatalogSearchSettingsFromMap(settings)
if err != nil {
slog.Warn("catalog search: failed to load settings for startup wiring; using postgres", "err", err)
catalogSearchStartupSettings = catalog.DefaultCatalogSearchSettings()
}
activeCatalogSearchProvider := catalog.ActiveCatalogSearchProvider(catalogSearchStartupSettings)
// Step 9: Validate
if err := cfg.Validate(); err != nil {
log.Fatalf("config validation: %v", err)
}
// Step 10: Configure log level. The level var and quiet filter are
// shared with the operational-logging handler chain and hot-reloaded by
// the config watcher in integrated mode.
logLevelVar := new(slog.LevelVar)
logLevelVar.Set(parseLogLevel(cfg.Server.LogLevel))
baseHandler := buildBaseHandler(cfg.Server.LogFormat, logLevelVar)
quietFilter := logfilter.New(baseHandler, cfg.Server.LogQuiet)
slog.SetDefault(slog.New(quietFilter))
mode := cfg.Server.Mode
maybeApplyPostgresTuning(ctx, pool, cfg.Database.MaxConnections, mode)
slog.Info("silo starting", "mode", mode, "listen", cfg.Server.Listen, "log_level", cfg.Server.LogLevel, "node_id", nodeID)
appCtx, appCancel := context.WithCancel(ctx)
defer appCancel()
restartReqCh := make(chan struct{}, 1)
var restartRequested atomic.Bool
eventBus := cache.NewEventBus(cfg.Redis.URL)
logStreamHub := logstream.NewHub(nodeID, eventBus)
if err := logStreamHub.Start(appCtx); err != nil {
log.Fatalf("log stream hub start: %v", err)
}
realtimeHub := notifications.NewHub(nodeID, eventBus)
if err := realtimeHub.Start(appCtx); err != nil {
log.Fatalf("realtime hub start: %v", err)
}
eventsHub := realtimeHub.EventsHub()
scanRegistry := evt.NewScanRegistry()
operationalWriter, opsRepo, opsPM := configureOperationalLogging(appCtx, pool, settingsRepo, cfg.Redis, logStreamHub, quietFilter, nodeID)
defer func() {
if err := eventBus.Close(); err != nil {
slog.Warn("event bus close error", "error", err)
}
}()
// Proxy and transcode modes run with DB + Redis for hot-reload.
if mode == "proxy" || mode == "transcode" {
redisClient, err := cache.NewRedisClient(cfg.Redis)
if err != nil || redisClient == nil {
slog.Error("redis is required for "+mode+" mode", "error", err)
os.Exit(1)
}
bootstrap := nodeconfig.BootstrapOverrides{
Listen: cfg.Server.Listen,
Mode: cfg.Server.Mode,
DatabaseURL: cfg.Database.URL,
JFListen: cfg.JellyfinCompat.Listen,
RedisURL: bc.RedisURL,
}
watcher := nodeconfig.NewWatcher(pool, dataCipher, eventBus, bootstrap)
if err := watcher.Start(appCtx); err != nil {
slog.Error("config watcher start failed", "error", err)
os.Exit(1)
}
nodeURL := os.Getenv("NODE_URL")
nodeName := os.Getenv("NODE_NAME")
if nodeURL == "" {
nodeURL = "http://localhost" + cfg.Server.Listen
slog.Warn("NODE_URL not set, using listen address — session keys may collide across nodes")
}
if nodeName == "" {
nodeName = mode
}
tracker := nodesessions.NewTracker(redisClient, nodeURL, nodeName, mode)
tracker.StartRefresh(appCtx)
defer func() {
cleanupCtx, cleanupCancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cleanupCancel()
tracker.Cleanup(cleanupCtx)
}()
var handler http.Handler
if mode == "proxy" {
srv := proxy.NewServer(watcher, tracker)
handler = srv.Handler()
} else {
srv := transcodenode.NewServer(watcher, tracker)
srv.SetFFmpegLogSink(playback.NewSlogFFmpegLogSink(slog.Default(), nodeID))
// Read jellycompat reconstruction recipes central wrote at transcode
// start, so this node can rebuild a Jellyfin transcode after its own
// restart (the node hop token is recipe-less). Shares the offload Redis.
srv.SetRecipeStore(noderecipe.NewStore(redisClient, 0))
handler = srv.Handler()
}
_ = operationalWriter
_ = opsRepo
startStandaloneServer(cfg.Server.Listen, handler)
return
}
// Hot-reload config watcher for integrated/api mode. Reloads on
// EventSettingsChanged (Redis) with a 60s poll fallback, so settings
// changes apply without restart even on Redis-less deployments. The
// watcher's config supersedes the startup snapshot from here on.
configWatcher := nodeconfig.NewWatcher(pool, dataCipher, eventBus, nodeconfig.BootstrapOverrides{
Listen: bc.Listen,
Mode: bc.Mode,
DatabaseURL: bc.DatabaseURL,
JFListen: bc.JFListen,
RedisURL: bc.RedisURL,
})
if err := configWatcher.Start(appCtx); err != nil {
log.Fatalf("config watcher start: %v", err)
}
cfg = configWatcher.Config()
// Apply server.log_level / server.log_quiet changes live. Both feed the
// shared level var and quiet filter inside the default logger chain.
configWatcher.OnChange(func(_, updated *config.Config) {
logLevelVar.Set(parseLogLevel(updated.Server.LogLevel))
quietFilter.SetQuiet(updated.Server.LogQuiet)
})
// Determine which components to initialize based on mode.
needsS3 := mode == "integrated" || mode == "api"
needsScanner := mode == "integrated" || mode == "api"
needsUserDB := mode == "integrated" || mode == "api"
needsWorkers := mode == "integrated" || mode == "api"
bootstrapSensitiveConfigured := map[string]bool{}
bootstrapSensitiveValues := map[string]string{}
if bc.RedisURL != "" {
bootstrapSensitiveConfigured["redis.url"] = true
bootstrapSensitiveValues["redis.url"] = bc.RedisURL
}
// Shared Redis client for components needing raw Redis beyond the event
// bus (websocket handshake tickets, session listing). Nil on Redis-less
// deployments; consumers fall back to in-process implementations.
apiRedisClient, apiRedisErr := cache.NewRedisClient(cfg.Redis)
if apiRedisErr != nil {
slog.Warn("redis client init failed; multi-node websocket tickets disabled", "error", apiRedisErr)
} else if apiRedisClient != nil {
defer func() { _ = apiRedisClient.Close() }()
}
// Assigned below once the trusted-proxy config is seeded; captured by the
// OnServerSettingUpdated closure, which only runs on admin requests after
// startup completes.
var ipResolver *clientip.Resolver
deps := api.Dependencies{
Config: cfg,
LiveConfig: configWatcher.Config,
OnConfigChange: configWatcher.OnChange,
BootstrapSensitiveConfigured: bootstrapSensitiveConfigured,
BootstrapSensitiveValues: bootstrapSensitiveValues,
AppContext: appCtx,
DB: pool,
SecretCipher: dataCipher,
EventBus: eventBus,
RedisClient: apiRedisClient,
LogStreamHub: logStreamHub,
RealtimeHub: realtimeHub,
EventsHub: eventsHub,
ScanRegistry: scanRegistry,
OpsLogRepo: opsRepo,
FFmpegLogSink: playback.NewSlogFFmpegLogSink(slog.Default(), nodeID),
PublicURL: os.Getenv("SILO_PUBLIC_URL"),
RequestServerRestart: func(context.Context) error {
if !restartRequested.CompareAndSwap(false, true) {
return handlers.ErrServerRestartAlreadyRequested
}
restartReqCh <- struct{}{}
return nil
},
OnServerSettingUpdated: func(_ context.Context, key, _ string) {
// Key-scoped reload for the client-IP trust boundary: unlike the
// whole-config watcher reload below, this cannot be blocked by an
// unrelated malformed setting failing config.LoadFromDB. Uses a
// fresh context — the setting is already persisted, so the reload
// must not be skipped because the admin request was canceled.
if key == clientip.SettingTrustedProxies && ipResolver != nil {
if cidrs, loadErr := clientip.LoadTrustedCIDRs(context.Background(), settingsRepo); loadErr != nil {
slog.Warn("clientip config reload failed", "error", loadErr)
} else {
ipResolver.UpdateTrustedCIDRs(cidrs)
}
}
// Nudge the hot-reload watcher so same-process settings changes
// apply immediately even without Redis (the event bus is a no-op
// then, leaving only the 60s poll).
configWatcher.RequestReload()
},
}
accessGroupStore := access.NewGroupStore(pool)
audiobooksService := audiobooks.New(&audiobooksSettingsAdapter{repo: settingsRepo})
absCompatEnabled, err := audiobooksService.ABSCompatEnabled(appCtx)
if err != nil {
slog.Warn("Audiobookshelf compatibility disabled; failed to read setting", "err", err)
absCompatEnabled = false
}
adminJobCancelRegistry := adminjob.NewCancelRegistry()
deps.AdminJobCancelRegistry = adminJobCancelRegistry
if needsWorkers && deps.DB != nil {
deps.IntroRepository = intromarkers.NewRepository(deps.DB)
deps.IntroAnalyzer = intromarkers.NewAnalyzer(
deps.IntroRepository,
intromarkers.DefaultConfig(cfg.Playback.FFmpegPath),
slog.Default(),
)
}
if deps.DB != nil {
markerRegistry := markers.NewRegistry(slog.Default())
markerProviderConfig := markers.NewProviderConfigStore(deps.DB)
if err := markerProviderConfig.Reload(appCtx); err != nil {
slog.Warn("load marker provider config failed; falling back to registration-order fetch",
"error", err)
} else {
markerRegistry.UseConfigStore(markerProviderConfig)
if deps.EventBus != nil {
if err := deps.EventBus.Subscribe(appCtx, cache.ChannelAdmin, func(event cache.Event) {
if event.Type != cache.EventMarkerProviderConfigChanged {
return
}
if err := markerProviderConfig.Reload(appCtx); err != nil {
slog.Warn("reload marker provider config failed", "provider", event.Payload, "error", err)
}
}); err != nil {
slog.Warn("subscribe marker provider config reload failed", "error", err)
}
}
}
deps.MarkerProviderConfig = markerProviderConfig
deps.MarkerRegistry = markerRegistry
markerResolver := markers.NewDBExternalIDResolver(deps.DB)
deps.MarkerResolver = markerResolver
markerContributionStore := markers.NewContributionStore(deps.DB)
deps.MarkerContributionStore = markerContributionStore
deps.MarkerContributionService = markers.NewContributionService(
markerRegistry, markerResolver, markerProviderConfig, markerContributionStore, slog.Default(),
)
}
var watchProviderService *watchsync.Service
if deps.DB != nil {
watchProviderRegistry := watchsync.NewRegistry()
if err := watchProviderRegistry.Register(trakt.NewProvider(nil, "")); err != nil {
log.Fatalf("register watch provider: %v", err)
}
if err := watchProviderRegistry.Register(simkl.NewProvider(nil, "")); err != nil {
log.Fatalf("register watch provider: %v", err)
}
if err := watchProviderRegistry.Register(watchmdblist.NewProvider(nil, "")); err != nil {
log.Fatalf("register watch provider: %v", err)
}
watchProviderService = watchsync.NewService(
watchsync.NewPostgresRepository(deps.DB, deps.SecretCipher),
watchProviderRegistry,
)
deps.WatchProviderService = watchProviderService
}
// Initialize node pools for integrated/api modes.
if mode == "integrated" || mode == "api" {
nodeRepo := nodepool.NewRepository(pool)
deps.NodeRepo = nodeRepo
proxyPool := nodepool.NewProxyPool()
transcodePool := nodepool.NewTranscodePool()
proxyNodes, _ := nodeRepo.ListEnabled(context.Background(), nodepool.NodeTypeProxy)
transcodeNodes, _ := nodeRepo.ListEnabled(context.Background(), nodepool.NodeTypeTranscode)
proxyPool.SetNodes(proxyNodes)
transcodePool.SetNodes(transcodeNodes)
deps.ProxyPool = proxyPool
deps.TranscodePool = transcodePool
deps.NodePlanner = nodepool.NewPlanner(proxyPool, transcodePool)
healthChecker := nodepool.NewHealthChecker(proxyPool, transcodePool, nodeRepo)
healthChecker.Start(appCtx)
slog.Info("node pools initialized", "proxy_nodes", len(proxyNodes), "transcode_nodes", len(transcodeNodes))
// Subscribe to node pool change events for multi-instance reload.
_ = eventBus.Subscribe(appCtx, cache.ChannelAdmin, func(event cache.Event) {
if event.Type == cache.EventNodePoolChanged {
pNodes, pErr := nodeRepo.ListEnabled(context.Background(), nodepool.NodeTypeProxy)
tNodes, tErr := nodeRepo.ListEnabled(context.Background(), nodepool.NodeTypeTranscode)
if pErr != nil || tErr != nil {
slog.Warn("node pool reload from event failed, keeping current pools",
"proxy_err", pErr, "transcode_err", tErr)
return
}
proxyPool.SetNodes(pNodes)
transcodePool.SetNodes(tNodes)
slog.Info("node pools reloaded from event", "proxy", len(pNodes), "transcode", len(tNodes))
}
})
}
// Step 3: Create S3 clients (if needed).
if needsS3 {
configureS3Clients(cfg, &deps)
}
var literaryWorkService *literaryworks.Service
if deps.DB != nil {
literaryWorkService = literaryworks.NewService(literaryworks.NewRepository(deps.DB))
}
// Step 4: Create scanner (if needed).
if needsScanner && deps.DB != nil {
folderRepo := catalog.NewFolderRepository(deps.DB)
fileRepo := scanner.NewFileRepository(deps.DB)
deps.FolderRepo = folderRepo
deps.FileRepo = fileRepo
ffprobePath := scanner.FFprobePathFromFFmpeg(cfg.Playback.FFmpegPath)
s := scanner.NewScanner(fileRepo, ffprobePath, deps.S3Public, cfg.Scanner.Workers, cfg.Scanner.EmptyTrashAfterScan)
s.SetSearchIndexProvider(activeCatalogSearchProvider)
configWatcher.OnChange(func(_, updated *config.Config) {
s.SetWorkers(updated.Scanner.Workers)
})
s.SetLiteraryWorkLinker(literaryWorkService)
deps.Scanner = s
deps.ProbeEnsurer = scanner.NewPlaybackProbeEnsurer(fileRepo, ffprobePath, 10*time.Second)
slog.Info("scanner initialized")
}
var chapterThumbService *chapterthumbs.Service
if deps.FileRepo != nil && deps.FolderRepo != nil && deps.S3Public != nil {
chapterThumbService = chapterthumbs.NewService(
deps.FileRepo,
deps.FolderRepo,
deps.ProbeEnsurer,
settingsRepo,
deps.S3Public,
nil,
deps.TranscodePool,
cfg.Playback.FFmpegPath,
cfg.Playback.HWAccel,
cfg.Playback.HWDevice,
cfg.Playback.ChapterThumbnailWorkers,
)
if chapterThumbService != nil {
chapterThumbService.Start(appCtx)
deps.ChapterThumbnailQueuer = chapterThumbService
}
}
var pluginHost *pluginhost.Host
var pluginService *plugins.Service
var pluginInstallationStore *plugins.InstallationStore
var pluginRuntimeConfigStore *plugins.RuntimeConfigStore
var pluginHTTPProxy *plugins.HTTPProxy
pluginAutoUpdateDone := make(chan struct{})
var pluginAutoUpdater *plugins.AutoUpdateService
if deps.DB != nil {
pluginCacheDir := resolvePluginCacheDir()
repositoryStore := plugins.NewRepositoryStore(deps.DB)
installationStore := plugins.NewInstallationStore(deps.DB)
runtimeConfigStore := plugins.NewRuntimeConfigStore(deps.DB)
catalogService := plugins.NewCatalogService(repositoryStore, plugins.CatalogServiceOptions{
SiloAPIVersion: plugins.DefaultSiloAPIVersion,
})
installer := plugins.NewInstaller(installationStore, plugins.InstallerOptions{
BaseDir: pluginCacheDir,
})
libDataSource := pluginhost.LibraryDataSourceFunc(
func(ctx context.Context, _ string) ([]pluginhost.LibraryRecord, error) {
// TODO: scope by userID when the requests plugin needs it (Plan B).
// For now, all callers see admin-scope.
if deps.FolderRepo == nil {
return nil, nil
}
folders, err := deps.FolderRepo.List(ctx)
if err != nil {
return nil, err
}
out := make([]pluginhost.LibraryRecord, 0, len(folders))
for _, f := range folders {
out = append(out, pluginhost.LibraryRecord{
ID: strconv.Itoa(f.ID),
Name: f.Name,
MediaType: mapFolderTypeToMediaType(f.Type),
})
}
return out, nil
},
)
presenceItemRepo := catalog.NewItemRepository(deps.DB)
catalogPresence := pluginhost.NewCatalogPresence(
func(ctx context.Context, mediaType string, tmdbIDs []string) ([]pluginhost.LibraryPresenceRecord, error) {
rows, err := presenceItemRepo.LookupTMDBIDs(ctx, mediaType, tmdbIDs)
if err != nil {
return nil, err
}
out := make([]pluginhost.LibraryPresenceRecord, 0, len(rows))
for _, r := range rows {
out = append(out, pluginhost.LibraryPresenceRecord{
ExternalID: r.TMDBID,
MediaID: r.MediaID,
LibraryID: r.LibraryID,
Title: r.Title,
})
}
return out, nil
},
)
pluginHost = pluginhost.NewHost(pluginhost.Config{
EventPublisher: eventsHub,
LibraryLister: pluginhost.NewLibraryLister(libDataSource),
CatalogPresence: catalogPresence,
InstalledPlugins: pluginhost.InstalledPluginListerFunc(
func(ctx context.Context) ([]pluginhost.InstalledPluginRecord, error) {
installations, err := installationStore.List(ctx)
if err != nil {
return nil, err
}
out := make([]pluginhost.InstalledPluginRecord, 0, len(installations))
for _, installation := range installations {
capabilities, err := installationStore.ListCapabilities(ctx, installation.ID)
if err != nil {
return nil, err
}
descriptors := make([]*pluginv1.CapabilityDescriptor, 0, len(capabilities))
for _, capability := range capabilities {
descriptor, err := plugins.DecodeCapability(capability)
if err != nil {
return nil, err
}
descriptors = append(descriptors, descriptor)
}
out = append(out, pluginhost.InstalledPluginRecord{
InstallationID: installation.ID,
PluginID: installation.PluginID,
Version: installation.Version,
Enabled: installation.Enabled,
Capabilities: descriptors,
})
}
return out, nil
},
),
GlobalConfigSetter: pluginhost.GlobalConfigSetterFunc(
func(ctx context.Context, installationID int, key string, value map[string]any) error {
return runtimeConfigStore.PutGlobalConfig(ctx, installationID, key, value)
},
),
Logger: hclog.New(&hclog.LoggerOptions{
Name: "plugin-host",
Level: hclog.Info,
Output: os.Stderr,
}),
})
pluginService = plugins.NewService(
repositoryStore,
installationStore,
runtimeConfigStore,
catalogService,
installer,
plugins.NewHostAdapter(pluginHost),
)
if deps.MarkerRegistry != nil && deps.MarkerProviderConfig != nil {
markerPluginResolver := markers.NewPluginResolverAdapter(pluginService)
pluginService.AddLifecycleHook(func(ctx context.Context) {
if err := reloadMarkerPluginProviders(
ctx,
deps.MarkerRegistry,
deps.MarkerProviderConfig,
installationStore,
runtimeConfigStore,
settingsRepo,
markerPluginResolver,
); err != nil {
slog.Warn("reload marker plugin providers failed", "error", err)
}
})
}
if err := pluginService.PreloadEnabled(appCtx); err != nil {
log.Fatalf("preload enabled plugins: %v", err)
}
slog.Info("plugin cache initialized", "base_dir", pluginCacheDir)
pluginAutoUpdater = plugins.NewAutoUpdateService(
repositoryStore,
installationStore,
catalogService,
installer,
pluginHost,
slog.Default(),
// Auto-updates rewrite installation rows (new version-specific
// InstallPath/Version) and delete the old install dir without going
// through pluginService. Wire OnLifecycleChange so the service's
// installation cache is invalidated and later plugin RPCs re-read
// the fresh row instead of a stale one.
pluginService.OnLifecycleChange,
)
go func() {
defer close(pluginAutoUpdateDone)
if err := pluginAutoUpdater.Run(appCtx); err != nil {
slog.Error("plugin auto-update failed", "error", err)
}
}()
pluginInstallationStore = installationStore
pluginRuntimeConfigStore = runtimeConfigStore
pluginHTTPProxy = plugins.NewHTTPProxyWithTypedResolver(pluginService, pluginInstallationStore)
if deps.DB != nil {
pluginHTTPProxy = pluginHTTPProxy.WithUserThemeLookup(plugins.NewPgUserThemeLookup(deps.DB))
pluginHTTPProxy = pluginHTTPProxy.WithUserIdentityLookup(plugins.NewPgUserIdentityLookup(deps.DB))
}
deps.PluginService = pluginService
deps.PluginHTTPProxy = pluginHTTPProxy
defer func() {
if pluginHost == nil {
return
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := pluginHost.Shutdown(shutdownCtx); err != nil {
slog.Warn("failed to shut down plugin host", "error", err)
}
}()
} else {
close(pluginAutoUpdateDone)
}
if pluginService != nil && pluginInstallationStore != nil {
dispatcher := plugins.NewEventDispatcherWithTypedResolver(deps.EventBus, deps.EventsHub, pluginInstallationStore, pluginService, 4)
pluginService.SetEventDispatcher(dispatcher)
if err := dispatcher.Start(appCtx); err != nil {
log.Fatalf("plugin event dispatcher: %v", err)
}
defer dispatcher.Stop()
// Backfill the capability-subscriber index from the already-preloaded
// installations. PreloadEnabled ran earlier (before the dispatcher
// existed), so its rebuildDispatcherIndex was a no-op. Without this
// call, capability-scoped subscriptions never fire until the next
// lifecycle mutation.
pluginService.OnLifecycleChange(appCtx)
}
// backgroundInit collects non-critical startup work (catalog-size-dependent
// seeding, network-bound reconciliation) that must not block the HTTP
// listener. The steps run sequentially in a background goroutine once the
// server is ready to serve. Failures are logged, never fatal.
var backgroundInit []func(context.Context)
// Step 4b: Create metadata service and match worker (if needed).
var metadataService *metadata.MetadataService
var metadataImageCacheProcessor *metadata.ImageCacheProcessor
var personRefreshService *metadata.PersonRefreshService
var matchWorker *metadata.MatchWorker
var libraryIngestExecutor *libraryingest.Executor
var libraryScanQueue *scanqueue.Service
var itemRefreshExecutor *adminjob.ItemRefreshExecutor
var libraryRefreshExecutor *adminjob.LibraryRefreshExecutor
var itemRepo *catalog.ItemRepository
var skippedRootRepo *metadata.SkippedRootRepository
var movieQueueRepo *metadata.MovieMatchQueueRepository
var seriesQueueRepo *metadata.SeriesRootMatchQueueRepository
var matchQueueCoordinator *metadata.MatchQueueCoordinator
var rootClaimRepo *catalog.RootClaimRepository
var groupClaimRepo *catalog.GroupClaimRepository
var seasonRepo *catalog.SeasonRepository
var episodeRepo *catalog.EpisodeRepository
var audiobookEnricher *audiobooks.Enricher
var ebookEnricher *ebooks.Enricher
var mangaEnricher *manga.Enricher
if needsWorkers && deps.DB != nil && deps.FileRepo != nil {
chainRepo := metadata.NewChainRepository(deps.DB)
skippedRootRepo = metadata.NewSkippedRootRepository(deps.DB)
itemRepo = catalog.NewItemRepository(deps.DB).WithActiveSearchProvider(activeCatalogSearchProvider)
episodeRepo = catalog.NewEpisodeRepository(deps.DB)
seasonRepo = catalog.NewSeasonRepository(deps.DB)
personRepo := catalog.NewPersonRepository(deps.DB)
libraryRepo := catalog.NewLibraryItemRepository(deps.DB)
// Wait for plugin auto-update to finish before registering image resolvers.
<-pluginAutoUpdateDone
imageResolver := metadata.NewPluginImageResolver()
if pluginService != nil && pluginInstallationStore != nil {
reloadImageResolvers := func(ctx context.Context) {
if err := reloadPluginImageResolvers(ctx, pluginInstallationStore, imageResolver, pluginService); err != nil {
slog.Warn("failed to reload plugin image resolvers", "error", err)
}
}
pluginService.AddLifecycleHook(reloadImageResolvers)
reloadImageResolvers(appCtx)
}
if deps.S3Public != nil {
presignTTL := cfg.S3.MetadataPresignExpiry
if presignTTL <= 0 {
presignTTL = 4 * time.Hour
}
imageResolver.SetS3Presigner(deps.S3Public, deps.S3Public.EffectivePresignTTL(presignTTL))
}
deps.ImageResolver = imageResolver
deps.PluginImageResolver = imageResolver
staleIDRepo := metadata.NewStaleMediaIDRepository(deps.DB)
providerIDRepo := catalog.NewProviderIDRepository(deps.DB)
movieQueueRepo = metadata.NewMovieMatchQueueRepository(deps.DB, deps.FileRepo)
seriesQueueRepo = metadata.NewSeriesRootMatchQueueRepository(deps.DB)
deps.MovieMatchQueueRepo = movieQueueRepo
deps.SeriesRootMatchQueueRepo = seriesQueueRepo
matchQueueCoordinator = metadata.NewMatchQueueCoordinator(movieQueueRepo, seriesQueueRepo)
rootClaimRepo = catalog.NewRootClaimRepository(deps.DB)
groupClaimRepo = catalog.NewGroupClaimRepository(deps.DB)
pluginResolver := metadata.NewPluginResolverAdapter(pluginService)
// Serve the metadata chain's plugin-installation enabled-check from the
// plugins service's in-memory installation cache. Declared as the
// interface type and only assigned when pluginService is non-nil so a
// nil *plugins.Service is passed as a genuine nil interface (not a
// typed-nil), letting buildProviders fall back to the pool query.
var installationEnabledChecker metadata.InstallationEnabledChecker
if pluginService != nil {
installationEnabledChecker = pluginService
}
metadataService = metadata.NewMetadataService(
chainRepo, pluginResolver, installationEnabledChecker,
itemRepo, providerIDRepo, episodeRepo, seasonRepo, libraryRepo, deps.FolderRepo,
personRepo,
deps.FileRepo, skippedRootRepo, staleIDRepo, rootClaimRepo,
)
// Drop the resolved-chain cache whenever a plugin is installed, enabled,
// disabled, updated, or uninstalled. The installation-enabled check is
// served from the plugins service's in-memory cache (invalidated on the
// same events), but resolveChainCached would otherwise keep serving a
// stale provider chain for up to chainCacheTTL after a provider's
// availability changes.
if pluginService != nil {
pluginService.AddLifecycleHook(func(context.Context) {
metadataService.InvalidateChainCache()
})
}
personRefreshService = metadata.NewPersonRefreshService(deps.DB, pluginResolver, personRepo)
personRefreshService.SetImageResolver(imageResolver)
// Wire the audiobook enricher. It uses the same plugin resolver and chain
// repo as the movie/TV pipeline, but resolves providers at
// content_level='audiobook' and sweeps items directly rather than via a queue.
audiobookEnricher = audiobooks.NewEnricher(
deps.DB,
chainRepo,
pluginResolver,
itemRepo,
personRepo,
providerIDRepo,
)
ebookEnricher = ebooks.NewEnricher(
deps.DB,
chainRepo,
pluginResolver,
itemRepo,
personRepo,
providerIDRepo,
)
audiobookEnricher.SetLiteraryWorkLinker(literaryWorkService)
ebookEnricher.SetLiteraryWorkLinker(literaryWorkService)
mangaEnricher = manga.NewEnricher(
deps.DB,
chainRepo,
pluginResolver,
itemRepo,
personRepo,
providerIDRepo,
)
// Always wire the image resolver so plugin-prefixed URLs (e.g.
// metadb://) can be resolved to presigned HTTP URLs in API responses.
metadataService.SetImageResolver(imageResolver)
// Wire the image cacher whenever object storage is available so explicit
// admin image applies can succeed even if automatic metadata caching is off.
if deps.S3Public != nil {
imageCacher := imagecache.New(deps.S3Public)
metadataService.SetImageCacher(imageCacher)
imageCacheJobs := metadata.NewImageCacheJobRepository(deps.DB)
metadataService.SetImageCacheJobEnqueuer(imageCacheJobs)
metadataImageCacheProcessor = metadata.NewImageCacheProcessorWithTargets(
imageCacheJobs,
imageCacher,
imageResolver,
metadata.ImageCacheProcessorTargets{
Items: itemRepo,
Seasons: seasonRepo,
Episodes: episodeRepo,
ItemLocalizations: catalog.NewMediaItemLocalizationRepository(deps.DB),
SeasonLocalizations: catalog.NewSeasonLocalizationRepository(deps.DB),
People: personRepo,
},
)
metadataService.SetAutoCacheImages(cfg.Metadata.CacheImages)
metadataImageCacheProcessor.SetEnabled(cfg.Metadata.CacheImages)
configWatcher.OnChange(func(_, updated *config.Config) {
metadataService.SetAutoCacheImages(updated.Metadata.CacheImages)
metadataImageCacheProcessor.SetEnabled(updated.Metadata.CacheImages)
})
if deps.Scanner != nil {
deps.Scanner.SetImageCacher(imageCacher)
}
if cfg.Metadata.CacheImages {
personRefreshService.SetImageCacher(imageCacher)
personRefreshService.SetImageCacheJobEnqueuer(imageCacheJobs)
slog.Info("metadata image caching enabled")
}
if audiobookEnricher != nil {
audiobookEnricher.SetImageCacher(imageCacher)
audiobookEnricher.SetImageCacheJobEnqueuer(imageCacheJobs)
audiobookEnricher.SetFFmpegPath(scanner.FFmpegPathFromFFprobe(scanner.FFprobePathFromFFmpeg(cfg.Playback.FFmpegPath)))
}
if ebookEnricher != nil {
ebookEnricher.SetImageCacher(imageCacher)
ebookEnricher.SetImageCacheJobEnqueuer(imageCacheJobs)
}
if mangaEnricher != nil {
mangaEnricher.SetImageCacher(imageCacher)
mangaEnricher.SetImageCacheJobEnqueuer(imageCacheJobs)
}
}
matchWorker = metadata.NewMatchWorker(metadataService, deps.FileRepo, cfg.Matcher.Workers, cfg.Matcher.BatchSize, 30*time.Second)
mwForReload := matchWorker
configWatcher.OnChange(func(_, updated *config.Config) {
mwForReload.SetConcurrency(updated.Matcher.Workers, updated.Matcher.BatchSize)
})
matchWorker.SetRealtimeHub(deps.RealtimeHub)
if movieQueueRepo != nil {
matchWorker.SetMovieFileClaimer(movieQueueRepo)
}
if seriesQueueRepo != nil {
matchWorker.SetSeriesRootClaimer(seriesQueueRepo, cfg.Matcher.TVSeriesRootQueueEnabled())
backgroundInit = append(backgroundInit, func(ctx context.Context) {
if cleaned, err := seriesQueueRepo.CleanupLegacySeriesGroupQueue(ctx); err != nil {
slog.Warn("failed to clean legacy series group queue rows", "error", err)
} else if cleaned > 0 {
slog.Info("cleaned legacy series group queue rows", "count", cleaned)
}
})
}
if deps.FolderRepo != nil {
backgroundInit = append(backgroundInit, func(ctx context.Context) {
start := time.Now()
enabledFolders, err := deps.FolderRepo.GetEnabled(ctx)
if err != nil {
slog.Warn("failed to seed metadata queues", "error", err)
return
}
seedMovieQueue := func(folderID int) {
if movieQueueRepo == nil {
return
}
if err := movieQueueRepo.SyncForFolder(ctx, folderID); err != nil {
slog.Warn("failed to seed movie match queue", "folder_id", folderID, "error", err)
}
}
seedSeriesQueue := func(folderID int) {
if seriesQueueRepo == nil {
return
}
if err := seriesQueueRepo.SyncForFolder(ctx, folderID); err != nil {
slog.Warn("failed to seed series root queue", "folder_id", folderID, "error", err)
}
}
for _, folder := range enabledFolders {
if folder == nil {
continue
}
switch strings.ToLower(strings.TrimSpace(folder.Type)) {
case "movie", "movies":
seedMovieQueue(folder.ID)
case "series", "tv", "show", "tvshows":
seedSeriesQueue(folder.ID)
case "mixed":
seedSeriesQueue(folder.ID)
seedMovieQueue(folder.ID)
}
}
slog.Info("deferred init: metadata match queues seeded", "folders", len(enabledFolders), "duration", time.Since(start))
})
}
deps.SkippedRootRepo = skippedRootRepo
deps.StaleIDRepo = staleIDRepo
deps.PersonRepo = personRepo
deps.PersonRefreshQueue = worker.NewPersonRefreshWorker(
personRefreshService,
worker.DefaultPersonRefreshWorkerConfig(),
)
deps.PersonRefresher = personRefreshService
deps.Refresher = metadataService
deps.MetadataService = metadataService
slog.Info("metadata service initialized and running")
}
if deps.Scanner != nil {
if matchQueueCoordinator != nil {
deps.Scanner.SetMetadataQueueProducer(matchQueueCoordinator)
}
if movieQueueRepo != nil {
deps.Scanner.SetMovieQueueSyncer(movieQueueRepo)
}
if seriesQueueRepo != nil {
deps.Scanner.SetSeriesQueueSyncer(seriesQueueRepo)
}
}
if deps.Scanner != nil && matchWorker != nil && deps.FolderRepo != nil && skippedRootRepo != nil {
libraryIngestExecutor = libraryingest.NewExecutor(
deps.Scanner,
matchWorker,
deps.FolderRepo,
skippedRootRepo,
deps.EventBus,
deps.RealtimeHub,
)
deps.LibraryIngester = libraryIngestExecutor
if deps.DB != nil {
libraryScanQueue = scanqueue.NewService(
scanqueue.NewRepository(deps.DB),
deps.FolderRepo,
libraryIngestExecutor,
deps.EventsHub,
appCtx,
cfg.Scanner.MaxConcurrentLibraries,
cfg.Scanner.MaxConcurrentScoped,
)
// Started below, after the notification system has attached its
// availability detector to the executor: a scan resumed by the
// workers before that wiring would complete without recording
// episode availability, silently losing release notifications.
deps.LibraryScanQueue = libraryScanQueue
}
if deps.DB != nil && deps.FileRepo != nil && metadataService != nil {
itemRefreshResolver := adminjob.NewItemRefreshResolver(
itemRepo,
seasonRepo,
episodeRepo,
deps.FolderRepo,
deps.FileRepo,
)
libraryRefreshExecutor = adminjob.NewLibraryRefreshExecutor(
adminjob.NewPGLibraryRefreshItemLister(deps.DB),
deps.FolderRepo,
itemRefreshResolver,
libraryIngestExecutor,
metadataService,
deps.EventBus,
deps.RealtimeHub,
)
}
if metadataService != nil && deps.FileRepo != nil {
itemRefreshExecutor = adminjob.NewItemRefreshExecutor(
deps.FolderRepo,
deps.FileRepo,
rootClaimRepo,
groupClaimRepo,
skippedRootRepo,
seasonRepo,
episodeRepo,
libraryIngestExecutor,
metadataService,
deps.EventBus,
deps.RealtimeHub,
)
}
}
// Ensure PersonRepo is available for the router's DetailService.
if deps.DB != nil && deps.PersonRepo == nil {
deps.PersonRepo = catalog.NewPersonRepository(deps.DB)
}
// Step 5: Create user store provider (if needed).
var userStoreProvider userstore.UserStoreProvider
if needsUserDB {
switch cfg.UserDB.Backend {
case "sqlite":
poolConfig := userdb.PoolConfig{
MaxOpen: cfg.UserDB.PoolMaxOpen,
IdleTimeout: cfg.UserDB.IdleTimeout,
DataDir: "/var/lib/silo/userdb",
}
pool := userdb.NewUserDBPool(poolConfig)
userStoreProvider = userdb.NewSQLiteProvider(pool)
slog.Info("user store initialized", "backend", "sqlite", "max_open", poolConfig.MaxOpen)
default: // "postgres"
userStoreProvider = pgstore.NewPostgresProvider(deps.DB)
slog.Info("user store initialized", "backend", "postgres")
}
defer userStoreProvider.Close()
}
var policySystem *policy.System
if mode == "integrated" || mode == "api" {
policyDecisionLogger := policy.NewDecisionLogger(
deps.DB,
nodeID,
policy.WithDecisionLogLogger(slog.Default()),
)
policyDecisionLogger.SetVerbosity(cfg.Policy.DecisionLogVerbosity)
policyDecisionLogger.SetScopeSampleRate(cfg.Policy.DecisionLogScopeSampleRate)
policySystem = policy.NewSystem(
policy.NewPolicyStore(deps.DB),
deps.EventBus,
slog.Default(),
policy.WithSystemEvalTimeout(time.Duration(cfg.Policy.EvalTimeoutMS)*time.Millisecond),
policy.WithSystemDecisionLogger(policyDecisionLogger),
)
if err := policySystem.Start(appCtx); err != nil {
log.Fatalf("policy system start: %v", err)
}
deps.PolicySystem = policySystem
configWatcher.OnChange(func(_, updated *config.Config) {
policySystem.SetEvalTimeout(time.Duration(updated.Policy.EvalTimeoutMS) * time.Millisecond)
if logger := policySystem.DecisionLogger(); logger != nil {
logger.SetVerbosity(updated.Policy.DecisionLogVerbosity)
logger.SetScopeSampleRate(updated.Policy.DecisionLogScopeSampleRate)
}
})
defer policySystem.Stop()
}
// User-facing release notifications. The system reads user state through
// the raw store provider; the provider handed to everything downstream is
// wrapped so every favorites/watchlist/progress mutation (REST handlers,
// jellycompat, imports, playback) feeds the interest index.
var notificationSystem *notifications.System
if deps.DB != nil && userStoreProvider != nil {
userRepo := auth.NewUserRepository(deps.DB)
profileTokens := access.NewProfileTokenService(cfg.Auth.JWTSecret, 0)
var notificationScopes notifications.ScopeResolver
if policySystem != nil {
notificationScopes = policy.NewViewerResolver(userRepo, userStoreProvider, profileTokens, policySystem.PDP(), accessGroupStore)
} else {
// Legacy resolver: proxy/test wiring without a policy system. Production integrated/api modes always take the policy path. Removed with the legacy cleanup phase.
notificationScopes = access.NewResolver(userRepo, userStoreProvider, profileTokens, accessGroupStore)
}
notificationSystem = notifications.NewSystem(
deps.DB,
settingsRepo,
userStoreProvider,
notificationScopes,
userRepo,
deps.EventsHub,
deps.RedisClient,
deps.SecretCipher,
mail.NewSMTPSender(settingsRepo),
)
userStoreProvider = notifications.WrapUserStoreProvider(userStoreProvider, notificationSystem)
deps.Notifications = notificationSystem
if libraryIngestExecutor != nil {
libraryIngestExecutor.SetAvailabilityDetector(notificationSystem.Detector)
}
if needsWorkers {
notificationSystem.Start(appCtx)
defer notificationSystem.Wait()
}
}
// Start the scan queue only now that the availability detector (when
// notifications are enabled) is attached to the ingest executor, so scans
// resumed at startup cannot complete before the detector exists.
if libraryScanQueue != nil {
libraryScanQueue.Start()
defer libraryScanQueue.Stop()
}
if userStoreProvider != nil && pluginService != nil {
deps.PluginUserConfig = plugins.NewUserConfigStore(userStoreProvider, pluginService)
}
// Step 6: Create playback session manager and wire into dependencies.
sessionMgr := playback.NewSessionManager(6, 2) // defaults from plan: max_streams=6, max_transcodes=2
if deps.DB != nil {
userRepo := auth.NewUserRepository(deps.DB)
sessionMgr.SetLimitProvider(func(ctx context.Context, userID int) (playback.SessionLimits, error) {
user, err := userRepo.GetByID(ctx, userID)
if err != nil {
return playback.SessionLimits{}, err
}
effective, err := access.EffectivePolicyForUser(ctx, user, accessGroupStore)
if err != nil {
return playback.SessionLimits{}, err
}
return playback.SessionLimits{
MaxStreams: effective.MaxStreams,
MaxTranscodes: effective.MaxTranscodes,
}, nil
})
if policySystem != nil {
sessionMgr.SetAdmissionDecider(policy.NewPlaybackAdmissionDecider(policySystem.PDP()))
}
}
if userStoreProvider != nil {
deps.UserStoreProvider = userStoreProvider
}
if watchProviderService != nil {
historyRepo := historyimport.NewRepository(deps.DB, deps.SecretCipher)
historyIdentity := watchstate.NewStableIdentityResolver(itemRepo, episodeRepo, catalog.NewProviderIDRepository(deps.DB))
watchProviderService.
WithMatcher(historyimport.NewMatcher(historyRepo)).
WithWatchState(watchstate.NewService(userStoreProvider).WithStableIdentityResolver(historyIdentity)).
WithUserStoreProvider(userStoreProvider)
backgroundInit = append(backgroundInit, func(ctx context.Context) {
if err := watchProviderService.SweepOpenScrobbles(ctx); err != nil {
slog.Warn("failed to sweep open watch provider scrobbles", "error", err)
}
})
}
// Auto-remove fully-watched items from the watchlist (standalone behavior,
// default-on per profile), propagating removals to connected providers.
if itemRepo != nil && episodeRepo != nil && userStoreProvider != nil {
maintainer := watchlist.NewMaintainer(userStoreProvider, itemRepo, episodeRepo)
if watchProviderService != nil {
maintainer.WithListEventDispatcher(watchProviderService)
}
deps.WatchCompletionObserver = maintainer
}
deps.SessionMgr = sessionMgr
deps.PlaybackRealtimeHub = playback.NewRealtimeHub()
if chapterThumbService != nil && deps.S3Public != nil {
chapterThumbService.SetNotifier(
playback.NewChapterThumbnailNotifier(sessionMgr, deps.PlaybackRealtimeHub, deps.S3Public, 0),
)
}
// Build the reconciler early enough that playback handlers can trigger
// immediate session syncs after start/stop events.
nodeIdentity := resolveNodeIdentity()
var reconciler *worker.Reconciler
var heartbeatWriter *worker.HeartbeatWriter
if needsWorkers && deps.DB != nil {
sessionProvider := func() []worker.SessionSync {
sessions := sessionMgr.AllSessions()
syncs := make([]worker.SessionSync, len(sessions))
for i, s := range sessions {
syncs[i] = buildLiveSessionSync(s, nodeIdentity)
}
return syncs
}
reconciler = worker.NewReconciler(deps.DB, nodeIdentity, sessionProvider)
reconciler.EventBus = deps.EventBus
reconciler.EventsHub = deps.EventsHub
reconciler.PreSync = func() {
// Retire sessions that have not shown real playback activity
// recently enough to count as live. This keeps the in-memory
// limiter, transcode teardown, and synced admin view aligned.
if expired := sessionMgr.CleanStale(); len(expired) > 0 {
slog.Info("expired idle sessions", "count", len(expired))
}
}
deps.SessionSyncer = reconciler
nodeURL := fmt.Sprintf("http://%s%s", nodeIdentity, cfg.Server.Listen)
heartbeatWriter = worker.NewHeartbeatWriter(deps.DB, nodeIdentity, mode, nodeURL)
}
if deps.DB != nil {
adminStatsProvider, statsErr := handlers.NewAdminStatsProvider(appCtx, deps.DB, deps.EventBus)
if statsErr != nil {
log.Fatalf("failed to create admin stats provider: %v", statsErr)
}
defer adminStatsProvider.Close()
deps.AdminStatsProvider = adminStatsProvider
}
// Wire recommendations engine, worker, and ratings repo if enabled.
var recEngine *recommendations.Engine
var recWorker *recommendations.Worker
if cfg.Recommendations.Enabled && deps.DB != nil {
deps.RatingsRepo = catalog.NewRatingsRepo(deps.DB)
recEngine = recommendations.NewEngine(
deps.DB,
deps.RatingsRepo,
catalog.NewItemRepository(deps.DB),
catalog.NewPersonRepository(deps.DB),
userStoreProvider,
cfg.Recommendations,
)
deps.Recommender = recEngine
deps.CatalogSearchVectorizer = recEngine
var err error
recWorker, err = recommendations.NewWorker(
recEngine,
cfg.Recommendations.EmbeddingsCron,
cfg.Recommendations.TasteProfilesCron,
cfg.Recommendations.CowatchCron,
cfg.Recommendations.RecommendationsCron,
cfg.Recommendations.EmbeddingsJobTimeout,
)
if err != nil {
slog.Error("failed to create recommendation worker", "error", err)
} else {
deps.RecWorker = recWorker
}
}
// Client IP resolver with trusted proxy config.
if err := clientip.SeedDefaults(ctx, settingsRepo); err != nil {
log.Fatalf("seed clientip defaults: %v", err)
}
trustedCIDRs, err := clientip.LoadTrustedCIDRs(ctx, settingsRepo)
if err != nil {
log.Fatalf("load trusted CIDRs: %v", err)
}
ipResolver = clientip.NewResolver(trustedCIDRs)
deps.ClientIPResolver = ipResolver
// Hot-reload trusted proxies on settings changes via two complementary
// paths. The direct event-bus subscription re-reads only the clientip key,
// so a malformed unrelated setting (which fails the whole-config reload)
// cannot leave stale trust CIDRs on Redis-backed multi-instance deploys.
_ = eventBus.Subscribe(appCtx, cache.ChannelAdmin, func(event cache.Event) {
if event.Type != cache.EventSettingsChanged {
return
}
cidrs, loadErr := clientip.LoadTrustedCIDRs(context.Background(), settingsRepo)
if loadErr != nil {
slog.Warn("clientip config reload failed", "error", loadErr)
return
}
ipResolver.UpdateTrustedCIDRs(cidrs)
})
// The config watcher covers the Redis-less poll/RequestReload path, so
// admin UI edits apply without a restart on single-node deployments too.
configWatcher.OnChange(func(old, updated *config.Config) {
if old != nil && old.ClientIP.TrustedProxies == updated.ClientIP.TrustedProxies {
return
}
raw := updated.ClientIP.TrustedProxies
if raw == "" {
raw = clientip.DefaultTrustedProxies
}
cidrs, parseErr := clientip.ParseCIDRs(raw)
if parseErr != nil {
slog.Warn("clientip config reload failed", "error", parseErr)
return
}
ipResolver.UpdateTrustedCIDRs(cidrs)
})
// Step 6b: Create rate limiter.
if cfg.RateLimit.Enabled && deps.DB != nil {
var perKeyLimiter, globalLimiter ratelimit.RateLimiter
isMemory := true
if cfg.RateLimit.Backend == "redis" {
redisClient, redisErr := cache.NewRedisClient(cfg.Redis)
if redisErr != nil {
log.Fatalf("failed to create Redis client for rate limiting: %v", redisErr)
}
if redisClient != nil {
perKeyLimiter = ratelimit.NewRedisLimiter(redisClient)
globalLimiter = ratelimit.NewRedisLimiter(redisClient)
isMemory = false
defer redisClient.Close()
}
}
if isMemory {
perKeyLimiter = ratelimit.NewMemoryLimiter()
globalLimiter = ratelimit.NewMemoryLimiter()
}
defer perKeyLimiter.Close()
defer globalLimiter.Close()
rateLimitMW := ratelimit.NewMiddleware(perKeyLimiter, globalLimiter, settingsRepo, isMemory)
if err := rateLimitMW.Init(context.Background()); err != nil {
log.Fatalf("failed to init rate limiter: %v", err)
}
// Subscribe for multi-instance reload (only fires if EventBus is Redis-backed)
_ = eventBus.Subscribe(appCtx, cache.ChannelAdmin, func(event cache.Event) {
if event.Type == cache.EventSettingsChanged {
if reloadErr := rateLimitMW.Reload(context.Background()); reloadErr != nil {
slog.Warn("rate limit config reload from event failed", "error", reloadErr)
}
}
})
deps.RateLimitMW = rateLimitMW
}
// Activity log writer + consumer.
if err := activitylog.SeedDefaults(ctx, settingsRepo); err != nil {
log.Fatalf("seed activitylog defaults: %v", err)
}
// Seed default page sections for home and existing libraries.
sectionRepo := sections.NewRepository(pool)
var folders []*models.MediaFolder
if deps.FolderRepo != nil {
var listErr error
folders, listErr = deps.FolderRepo.List(ctx)
if listErr != nil {
log.Fatalf("list libraries for section defaults: %v", listErr)
}
}
if err := sectionRepo.SeedDefaults(ctx, "home", nil, sections.DefaultHomeSections(folders)); err != nil {
log.Fatalf("seed home section defaults: %v", err)
}
if deps.FolderRepo != nil {
for _, f := range folders {
id := f.ID
if seedErr := sectionRepo.SeedDefaults(ctx, "library", &id, sections.DefaultLibrarySectionsForType(&id, f.Type)); seedErr != nil {
slog.Warn("seed library section defaults", "library_id", id, "error", seedErr)
}
}
}
activityPM := partman.NewManager(pool, "activity_log", partman.Weekly, 2)
if err := activityPM.EnsureFuturePartitions(appCtx); err != nil {
// Non-fatal: see the operational_logs partition incident. Writes fall
// back to the default partition and periodic cleanup retries.
slog.Warn("ensure activity log partitions; continuing in degraded mode", "error", err)
}
policyPM := partman.NewManager(pool, "policy_decisions", partman.Daily, 3)
if err := policyPM.EnsureFuturePartitions(appCtx); err != nil {
// Non-fatal: decision logs fall back to the default partition and
// periodic cleanup retries partition creation.
slog.Warn("ensure policy decision log partitions; continuing in degraded mode", "error", err)
}
var activityWriter activitylog.Writer
activityConsumer := activitylog.NewConsumer(pool, nil, logStreamHub)
if cfg.Redis.URL != "" {
actRedisClient, actRedisErr := cache.NewRedisClient(cfg.Redis)
if actRedisErr == nil && actRedisClient != nil {
activityWriter = activitylog.NewRedisWriter(actRedisClient)
activityConsumer = activitylog.NewConsumer(pool, actRedisClient, logStreamHub)
go activityConsumer.RunRedis(appCtx)
defer actRedisClient.Close()
}
}
if activityWriter == nil {
memWriter := activitylog.NewMemoryWriter(10000)
activityWriter = memWriter
go activityConsumer.RunMemory(appCtx, memWriter.Chan())
}
deps.ActivityLogWriter = activityWriter
deps.ActivityLogRepo = activitylog.NewRepo(pool)
deps.NodeID = nodeID
// Create refresh worker early so the task manager can use it for FindCandidates.
var refreshWorker *worker.RefreshWorker
var personRefreshWorker *worker.PersonRefreshWorker
if needsWorkers && deps.DB != nil {
refreshWorker = worker.NewRefreshWorker(deps.DB)
if deps.PersonRefreshQueue != nil {
personRefreshWorker, _ = deps.PersonRefreshQueue.(*worker.PersonRefreshWorker)
}
}
// Construct collection service for both the router and the collection sync scheduler.
var collectionSyncScheduler *catalog.CollectionSyncScheduler
var userCollectionScheduler *usercollections.Scheduler
var trendingRefresher *sections.TrendingRefresher
if needsWorkers && deps.DB != nil {
collectionRepo := catalog.NewLibraryCollectionRepository(deps.DB)
collItemRepo := catalog.NewItemRepository(deps.DB)
libraryItemRepo := catalog.NewLibraryItemRepository(deps.DB)
collectionService := catalog.NewLibraryCollectionService(collectionRepo, collItemRepo, libraryItemRepo, nil)
collectionService.TMDBCollections = api.NewTMDBCollectionFetcher(cfg.TMDBAPIKey)
deps.CollectionService = collectionService
collectionSyncScheduler = catalog.NewCollectionSyncScheduler(collectionRepo, collectionService, slog.Default())
// The trending refresher reuses the section repo (to find used source/
// window combos), a snapshot repo, an item repo (external-ID matching),
// and the TMDB fetcher. The Trakt fetcher needs settingsRepo and is
// propagated onto deps.TrendingRefresher later in router.go.
trendingRefresher = sections.NewTrendingRefresher(
sectionRepo,
sections.NewTrendingSnapshotRepository(pool),
catalog.NewItemRepository(deps.DB),
collectionService.TMDBCollections,
collectionService.TraktCollections,
)
deps.TrendingRefresher = trendingRefresher
if deps.UserStoreProvider != nil {
userSync := usercollections.NewService(deps.UserStoreProvider, collItemRepo, libraryItemRepo, nil, slog.Default())
userSync.TMDBCollections = collectionService.TMDBCollections
// Trakt fetchers are wired in router.go (they need settingsRepo);
// router.go propagates them onto userSync once configured.
userCollectionScheduler = usercollections.NewScheduler(deps.DB, userSync, slog.Default())
deps.UserCollectionSync = userSync
deps.UserCollectionScheduler = userCollectionScheduler
deps.MDBListClient = mdblist.NewClient(cfg.MDBListAPIKey, nil)
mdblistForReload := deps.MDBListClient
configWatcher.OnChange(func(_, updated *config.Config) {
mdblistForReload.SetAPIKey(updated.MDBListAPIKey)
})
}
}
// Wire up task manager for admin task API.
if needsWorkers && deps.DB != nil {
triggerRepo := taskrepository.NewPgTriggerRepository(deps.DB)
historyRepo := taskrepository.NewPgExecutionRepository(deps.DB)
taskMgr := taskmanager.New(triggerRepo, historyRepo, triggers.New, slog.Default())
if deps.EventsHub != nil {
taskMgr.AddObserver(evt.NewTaskObserver(deps.EventsHub))
}
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil {
taskMgr.Register(tasks.NewScanLibrariesTask(deps.FolderRepo, deps.LibraryScanQueue, deps.EventBus))
}
taskMgr.Register(tasks.NewCleanupOrphanedMediaItemsTask(catalog.NewOrphanedProvisionalCleaner(deps.DB)))
catalogSearchIndexer := catalog.NewCatalogSearchIndexer(deps.DB, settingsRepo)
taskMgr.Register(tasks.NewSyncCatalogSearchIndexTask(catalogSearchIndexer))
taskMgr.Register(tasks.NewRebuildCatalogSearchIndexTask(catalogSearchIndexer))
if deps.IntroAnalyzer != nil {
taskMgr.Register(tasks.NewDetectIntroMarkersTask(deps.IntroAnalyzer, settingsRepo))
}
if deps.MarkerContributionService != nil && deps.MarkerProviderConfig != nil && deps.MarkerContributionStore != nil && deps.FileRepo != nil {
taskMgr.Register(tasks.NewContributeMarkersTask(
deps.MarkerContributionService, deps.MarkerProviderConfig, deps.MarkerContributionStore, deps.FileRepo,
))
}
if chapterBackfiller, ok := deps.ChapterThumbnailQueuer.(*chapterthumbs.Service); ok {
taskMgr.Register(tasks.NewChapterThumbnailBackfillTask(chapterBackfiller, 25))
}
taskMgr.Register(tasks.NewActivityLogCleanupTask(deps.DB, settingsRepo, activityPM))
taskMgr.Register(tasks.NewOperationalLogCleanupTask(deps.DB, settingsRepo, opsPM))
taskMgr.Register(tasks.NewPolicyDecisionLogCleanupTask(deps.DB, settingsRepo, policyPM))
if deps.FileRepo != nil {
// Download prepare-to-file pipeline (Phase 3): a durable, leased encode
// queue hosted on the task manager. Built here (before Start) and shared
// with the API via deps so the download service can enqueue jobs.
artifactMgr := downloads.NewArtifactManager(
downloads.NewArtifactRepository(deps.DB),
downloads.NewRepository(deps.DB),
deps.FileRepo,
downloads.NewPlaybackPreparer(),
deps.NodeID,
func() *config.Config {
if deps.LiveConfig != nil {
if c := deps.LiveConfig(); c != nil {
return c
}
}
return deps.Config
},
func(ctx context.Context, d *downloads.Download) {
if deps.EventsHub == nil {
return
}
_ = deps.EventsHub.PublishJSON(ctx, evt.ChannelUserState, "download", map[string]any{
"download_id": d.ID,
"status": d.Status,
"media_item_id": d.ContentID,
"format": d.Format,
}, evt.PublishOptions{UserID: d.UserID, ProfileID: d.ProfileID})
},
)
encodeTask := tasks.NewEncodeDownloadArtifactsTask(artifactMgr)
artifactMgr.SetKick(func() { _ = taskMgr.RunTask(appCtx, encodeTask.Key()) })
taskMgr.Register(encodeTask)
deps.ArtifactManager = artifactMgr
}
if notificationSystem != nil {
taskMgr.Register(tasks.NewSeedContentAvailabilityTask(notificationSystem))
taskMgr.Register(tasks.NewRebuildReleaseInterestTask(notificationSystem))
taskMgr.Register(tasks.NewNotificationsRetentionTask(notificationSystem))
}
if matchWorker != nil {
taskMgr.Register(tasks.NewMatchMediaTask(matchWorker))
}
if refreshWorker != nil && metadataService != nil {
taskMgr.Register(tasks.NewRefreshMetadataTask(refreshWorker, metadataService))
}
if metadataImageCacheProcessor != nil {
taskMgr.Register(tasks.NewCacheMetadataImagesTask(metadataImageCacheProcessor))
}
if pluginAutoUpdater != nil {
taskMgr.Register(tasks.NewCheckPluginUpdatesTask(pluginAutoUpdater))
}
if collectionSyncScheduler != nil {
taskMgr.Register(tasks.NewSyncCollectionsTask(collectionSyncScheduler))
}
if trendingRefresher != nil {
taskMgr.Register(tasks.NewRefreshTrendingDiscoverTask(trendingRefresher))
}
if userCollectionScheduler != nil {
taskMgr.Register(tasks.NewSyncUserCollectionsTask(userCollectionScheduler))
}
if watchProviderService != nil {
taskMgr.Register(tasks.NewSyncWatchProvidersTask(watchProviderService))
}
requestReconcileSvc := mediarequests.NewService(
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
nil,
mediarequests.NewCatalogPresence(
catalog.NewItemRepository(deps.DB),
catalog.NewProviderIDRepository(deps.DB),
),
)
requestReconcileSvc.SetRequesterIdentityResolver(plugins.RequesterIdentityFromLookup(plugins.NewPgUserIdentityLookup(deps.DB)))
api.AttachRequestRouter(requestReconcileSvc, pluginService)
requestReconcileSvc.SetGroupPolicyProvider(accessGroupStore)
if userStoreProvider != nil {
userRepo := auth.NewUserRepository(deps.DB)
profileTokens := access.NewProfileTokenService(cfg.Auth.JWTSecret, 0)
var reconcileResolver scopeResolver
if policySystem != nil {
reconcileResolver = policy.NewViewerResolver(userRepo, userStoreProvider, profileTokens, policySystem.PDP(), accessGroupStore)
} else {
// Legacy resolver: proxy/test wiring without a policy system. Production integrated/api modes always take the policy path. Removed with the legacy cleanup phase.
reconcileResolver = access.NewResolver(userRepo, userStoreProvider, profileTokens, accessGroupStore)
}
requestReconcileSvc.SetEntitlementResolver(scopeEntitlementResolver{resolver: reconcileResolver})
}
if notificationSystem != nil {
requestReconcileSvc.SetFulfillmentNotifier(notifications.NewRequestFulfillmentNotifier(notificationSystem))
}
taskMgr.Register(tasks.NewReconcileRequestsTask(requestReconcileSvc, 100))
if deps.FolderRepo != nil && deps.LibraryScanQueue != nil && pluginService != nil && pluginInstallationStore != nil {
autoscanRepo := autoscan.NewRepository(deps.DB, deps.SecretCipher)
if err := autoscanRepo.MarkInterruptedEvents(appCtx); err != nil {
slog.Warn("autoscan: failed to mark interrupted polls", "err", err)
}
autoscanSvc := api.BuildAutoscanService(
autoscanRepo,
pluginService,
pluginInstallationStore,
mediarequests.NewRepository(deps.DB, deps.SecretCipher),
deps.FolderRepo,
deps.LibraryScanQueue,
deps.RedisClient,
)
// The poll task's default interval seeds the schedule from the stored
// settings (DefaultPollIntervalSeconds); per-cycle gating still runs
// off the live settings inside PollOnce. Seed in MILLISECONDS as
// seconds*1000 — the SAME computation HandleUpdateSettings uses to
// reschedule — so startup and reschedule agree for sub-minute and
// non-60-multiple intervals (the old seconds/60 minutes path diverged).
var intervalMs int64 = 10 * 60 * 1000
if settings, serr := autoscanRepo.GetSettings(appCtx); serr == nil && settings.DefaultPollIntervalSeconds > 0 {
intervalMs = int64(settings.DefaultPollIntervalSeconds) * 1000
}
taskMgr.Register(tasks.NewAutoscanPollTask(autoscanSvc, intervalMs))
}
reconcileProviderIDRepo := catalog.NewProviderIDRepository(deps.DB)
reconcileEpisodeRepo := catalog.NewEpisodeRepository(deps.DB)
historyResolver := watchstate.NewStableIdentityResolver(nil, reconcileEpisodeRepo, reconcileProviderIDRepo)
historyReconciler := watchstate.NewHistoryReconciler(deps.DB, historyResolver)
taskMgr.Register(tasks.NewRepairProviderIDIntegrityTask(metadata.NewProviderIDIntegrityRepairer(deps.DB), historyReconciler))
taskMgr.Register(tasks.NewReconcileWatchHistoryTask(historyReconciler))
taskMgr.Register(tasks.NewSyncPodcastFeedsTask(podcastfeed.New(), podcastfeed.NewDBStore(deps.DB)))
if audiobookEnricher != nil {
taskMgr.Register(tasks.NewSyncAudiobookMetadataTask(audiobookEnricher))
}
if ebookEnricher != nil {
taskMgr.Register(tasks.NewSyncEbookMetadataTask(ebookEnricher))
}
if mangaEnricher != nil {
taskMgr.Register(tasks.NewSyncMangaMetadataTask(mangaEnricher))
}
if pluginInstallationStore != nil && pluginRuntimeConfigStore != nil && pluginService != nil {
pluginTasks, err := plugins.NewTaskRegistryWithTypedResolver(pluginInstallationStore, pluginRuntimeConfigStore, pluginService).Tasks(appCtx)
if err != nil {
log.Fatalf("plugin task registry: %v", err)
}
for _, pluginTask := range pluginTasks {
taskMgr.Register(pluginTask)
}
}
taskMgr.Start(appCtx)
defer taskMgr.Stop()
deps.TaskManager = taskMgr
slog.Info("task manager started")
}
// Build the ABS-compatible REST + Socket.io handler when a DB pool is
// available. Routes are mounted at the root level by NewRouter (not under
// /api/v1/) so ABS clients resolve /login, /api/*, /abs/api/*, and
// /abs/socket.io/* without path prefix hacks.
if absCompatEnabled && deps.DB != nil {
absUserRepo := auth.NewUserRepository(deps.DB)
absSessionRepo := auth.NewSessionRepository(deps.DB)
absJWTService := auth.NewJWTService(
cfg.Auth.JWTSecret,
cfg.Auth.AccessTokenExpiry,
cfg.Auth.RefreshTokenExpiry,
)
configWatcher.OnChange(func(_, updated *config.Config) {
absJWTService.SetExpiries(updated.Auth.AccessTokenExpiry, updated.Auth.RefreshTokenExpiry)
})
absAuthSvc := auth.NewService(
auth.NewLocalProvider(absUserRepo, absSessionRepo),
absJWTService,
absSessionRepo,
absUserRepo,
nil, // invite codes: not needed for ABS compat
nil, // settings: not needed here
nil, // user store: not needed here
)
absItemRepo := catalog.NewItemRepository(deps.DB)
absEpisodeRepo := catalog.NewEpisodeRepository(deps.DB)
absSeasonRepo := catalog.NewSeasonRepository(deps.DB)
absPersonRepo := catalog.NewPersonRepository(deps.DB)
var absFileFetcher catalog.FileVersionFetcher
if deps.FileRepo != nil {
absFileFetcher = deps.FileRepo
}
absDetailSvc := catalog.NewDetailService(absItemRepo, absEpisodeRepo, absSeasonRepo, absPersonRepo, absFileFetcher)
if deps.ImageResolver != nil {
absDetailSvc.SetImageResolver(deps.ImageResolver)
}
var absScopeResolver scopeResolver
if policySystem != nil {
absScopeResolver = policy.NewViewerResolver(absUserRepo, userStoreProvider, nil, policySystem.PDP(), accessGroupStore)
} else {
absScopeResolver = access.NewResolver(absUserRepo, userStoreProvider, nil, accessGroupStore)
}
absHDeps := audiobooks.ABSHandlerDeps{
Pool: deps.DB,
Items: absItemRepo,
Files: deps.FileRepo,
Settings: settingsRepo,
Auth: &audiobooks.SiloCredValidator{
Auth: absAuthSvc,
Pool: deps.DB,
},
AccessResolver: audiobooks.NewABSAccessResolver(absUserRepo, userStoreProvider, absScopeResolver, accessGroupStore),
Recs: recommendations.NewRepo(deps.DB),
Detail: absDetailSvc,
SessionMgr: sessionMgr,
SessionSyncer: deps.SessionSyncer,
}
absH := audiobooksService.BuildABSHandler(absHDeps)
deps.ABSHandler = absH
}
_ = audiobooksService
if deps.DB != nil && pluginInstallationStore != nil && pluginRuntimeConfigStore != nil && deps.PluginService != nil {
userRepo := auth.NewUserRepository(deps.DB)
sessionRepo := auth.NewSessionRepository(deps.DB)
authBindings, err := pluginRuntimeConfigStore.ListAuthBindings(appCtx)
if err != nil {
log.Fatalf("list plugin auth bindings: %v", err)
}
for _, binding := range authBindings {
if binding == nil || !binding.Enabled {
continue
}
installation, err := pluginInstallationStore.GetByID(appCtx, binding.InstallationID)
if err != nil {
log.Fatalf("load plugin auth installation %d: %v", binding.InstallationID, err)
}
if !installation.Enabled {
continue
}
displayName := binding.CapabilityID
mode := "credentials"
iconURL := ""
capabilities, err := pluginInstallationStore.ListCapabilities(appCtx, binding.InstallationID)
if err == nil {
for _, capability := range capabilities {
if capability != nil && capability.Type == "auth_provider.v1" && capability.ID == binding.CapabilityID {
if name, ok := capability.Metadata["display_name"].(string); ok && strings.TrimSpace(name) != "" {
displayName = name
}
// auth_modes ["oauth2"] flips the login button into
// an OAuth-style "Sign in with X" path. Mode is "oauth"
// when oauth2 is the only declared mode; "credentials"
// when password is supported alongside or alone.
if rawModes, ok := capability.Metadata["auth_modes"].([]any); ok {
hasPassword := false
hasOAuth := false
for _, m := range rawModes {
switch m {
case "password":
hasPassword = true
case "oauth2":
hasOAuth = true
}
}
if hasOAuth && !hasPassword {
mode = "oauth"
}
}
if url, ok := capability.Metadata["icon_url"].(string); ok {
iconURL = url
}
break
}
}
}
// Generic OIDC and similar multi-instance plugins ship one binary
// but install once per IdP. Their admin SPA writes display_name
// + icon_url_path to runtime config so each install renders its
// own brand on the login page. Manifest values are the fallback.
if runtimeConfigs, err := pluginRuntimeConfigStore.ListGlobalConfigs(appCtx, binding.InstallationID); err == nil {
for _, rc := range runtimeConfigs {
switch rc.Key {
case "display_name":
if v, ok := rc.Value["value"].(string); ok && strings.TrimSpace(v) != "" {
displayName = v
}
case "icon_url_path":
if v, ok := rc.Value["value"].(string); ok && strings.TrimSpace(v) != "" {
iconURL = fmt.Sprintf("/api/v1/plugins/%d/assets/%s", binding.InstallationID, strings.TrimLeft(v, "/"))
}
}
}
}
deps.AuthProviders = append(deps.AuthProviders, auth.RegisteredProvider{
Info: auth.LoginProviderInfo{
ID: fmt.Sprintf("plugin:%d:%s", binding.InstallationID, binding.CapabilityID),
DisplayName: displayName,
Mode: mode,
Default: binding.DefaultLogin,
IconURL: iconURL,
InstallationID: binding.InstallationID,
},
Provider: auth.NewPluginProvider(
auth.PluginProviderConfig{
InstallationID: binding.InstallationID,
CapabilityID: binding.CapabilityID,
DisplayName: displayName,
AutoProvision: binding.AutoProvision,
},
sessionRepo,
userRepo,
deps.DB,
deps.PluginService,
),
})
}
}
// Step 7: Build HTTP router with all dependencies.
// compatServer is populated after the compat server is constructed below;
// the closure captures the pointer so revocation calls reach the live instance.
var compatServer *jellycompat.Server
deps.OnUserSessionsRevoked = func(ctx context.Context, userID int) {
if compatServer != nil {
compatServer.SessionStore().DeleteByUserID(userID)
}
}
distFS, fsErr := fs.Sub(siloweb.DistFS, "dist")
if fsErr != nil {
log.Fatalf("failed to create frontend FS: %v", fsErr)
}
deps.FrontendFS = distFS
server.WebDistFS = distFS
// White-label branding: one service shared by the API (public read + admin
// upload) and the frontend handler (index.html title, favicon, manifest).
// S3 is optional — pass a nil AssetStore (not the typed-nil *s3client.Client)
// when it isn't configured so text branding still works without it.
if settingsRepo != nil {
var brandingStore branding.AssetStore
if deps.S3Public != nil {
brandingStore = deps.S3Public
}
brandingSvc := branding.NewService(settingsRepo, brandingStore)
deps.BrandingService = brandingSvc
server.Branding = brandingSvc
}
router := api.NewRouter(deps)
// Step 8: Expose Prometheus metrics endpoint (not behind auth).
metricsMux := http.NewServeMux()
metricsMux.Handle("/metrics", promhttp.Handler())
metricsMux.Handle("/api/", router)
// ABS-compat is NOT mounted on the main listener — see the "ABS compat
// listener" block below. It binds its own port so the discovery probes
// (/ping, /healthcheck, /status, /init, /login, /socket.io) own the URL
// space without collision with silo's SPA fallback. Mirrors how the
// Jellyfin compat server is set up at :8096.
metricsMux.Handle("/", server.FrontendHandler())
// Step 9: Start background workers (if needed).
var sessionCleaner *worker.SessionCleaner
var adminJobRunner *adminjob.Runner
if needsWorkers && deps.DB != nil {
if reconciler == nil {
log.Fatal("reconciler must be initialized before starting workers")
}
reconciler.Start()
defer reconciler.Stop()
if heartbeatWriter != nil {
heartbeatWriter.Start()
defer heartbeatWriter.Stop()
}
// RefreshWorker is kept as a RefreshCandidateFinder for the task manager's
// RefreshMetadataTask but no longer runs its own background loop.
// Scanning is handled exclusively by the task manager's ScanLibrariesTask.
if personRefreshWorker != nil {
personRefreshWorker.Start()
defer personRefreshWorker.Stop()
}
sessionCleaner = worker.NewSessionCleaner(deps.DB, cfg.UserDB.StaleGraceSeconds)
sessionCleaner.EventBus = deps.EventBus
sessionCleaner.EventsHub = deps.EventsHub
sessionCleaner.Start()
defer sessionCleaner.Stop()
var templateBundleApplyExecutor interface {
ExecuteTemplateBundleApply(context.Context, adminjob.TemplateBundleApplyRequest, func(int, int, string)) (any, error)
}
if deps.CollectionService != nil {
collectionRepo := catalog.NewLibraryCollectionRepository(deps.DB)
itemRepo := catalog.NewItemRepository(deps.DB)
collectionHandler := handlers.NewLibraryCollectionHandler(
collectionRepo,
deps.CollectionService,
itemRepo,
4*time.Hour,
nil,
deps.S3Public,
)
collectionHandler.FrontendFS = deps.FrontendFS
collectionHandler.SectionRepo = sectionRepo
collectionHandler.FolderRepo = deps.FolderRepo
if collectionHandler.FolderRepo == nil {
collectionHandler.FolderRepo = catalog.NewFolderRepository(deps.DB)
}
templateBundleApplyExecutor = collectionHandler
}
adminJobRunner = adminjob.NewRunner(
adminjob.NewRepository(deps.DB),
catalogseed.NewService(deps.DB, catalog.NewPersonRepository(deps.DB), recommendations.NewRepo(deps.DB)),
deps.S3Private,
itemRefreshExecutor,
libraryRefreshExecutor,
adminjob.NewLibraryDeleteExecutor(deps.FolderRepo, sectionRepo),
adminjob.NewImageCacheCleanupExecutor(deps.S3Public),
templateBundleApplyExecutor,
deps.RealtimeHub,
)
adminJobRunner.SetCancelRegistry(adminJobCancelRegistry)
adminJobRunner.Start()
defer adminJobRunner.Stop()
// Start recommendation worker if enabled (reuse worker created above).
if recWorker != nil {
recWorker.Start()
defer recWorker.Stop()
// Check if this is first run (no embeddings yet).
embCount, _ := recommendations.NewRepo(deps.DB).EmbeddingCount(appCtx)
if embCount == 0 {
slog.Info("first run detected, triggering initial embedding")
recWorker.RunEmbeddingsNow()
}
}
slog.Info("background workers started")
}
// Step 10: Create and start the HTTP server.
srv := &http.Server{
Addr: cfg.Server.Listen,
Handler: metricsMux,
ReadTimeout: 30 * time.Second,
WriteTimeout: 120 * time.Second,
IdleTimeout: 120 * time.Second,
}
var compatSrv *http.Server
if (mode == "integrated" || mode == "api") && cfg.JellyfinCompat.Enabled && cfg.JellyfinCompat.Listen != "" {
compatDeps := jellycompat.Dependencies{
Config: cfg,
LiveConfig: configWatcher.Config,
DB: deps.DB,
SecretCipher: dataCipher,
ClientIPResolver: ipResolver,
NodePlanner: deps.NodePlanner,
JWTSecret: cfg.Auth.JWTSecret,
RecWorker: recWorker,
FrontendFS: deps.FrontendFS,
// Hand remote-transcode recipes to the shared recipe store so a dedicated
// transcode node that restarts can rebuild a jellycompat session.
RecipeNodeStore: noderecipe.NewStore(apiRedisClient, 0),
SessionSyncer: deps.SessionSyncer,
}
// Wire direct dependencies when DB is available.
if deps.DB != nil {
browseRepo := catalog.NewBrowseRepository(deps.DB)
itemRepo := catalog.NewItemRepository(deps.DB)
seasonRepo := catalog.NewSeasonRepository(deps.DB)
episodeRepo := catalog.NewEpisodeRepository(deps.DB)
providerIDRepo := catalog.NewProviderIDRepository(deps.DB)
personRepo := catalog.NewPersonRepository(deps.DB)
folderRepo := deps.FolderRepo
var fileFetcher catalog.FileVersionFetcher
if deps.FileRepo != nil {
fileFetcher = deps.FileRepo
}
detailSvc := catalog.NewDetailService(itemRepo, episodeRepo, seasonRepo, personRepo, fileFetcher)
detailSvc.SetFolderRepository(folderRepo)
detailSvc.SetGroupClaimRepository(catalog.NewGroupClaimRepository(deps.DB))
detailSvc.SetProbeEnsurer(deps.ProbeEnsurer)
detailSvc.SetChapterThumbnailQueuer(deps.ChapterThumbnailQueuer)
if deps.ImageResolver != nil {
detailSvc.SetImageResolver(deps.ImageResolver)
}
compatDeps.BrowseRepo = browseRepo
compatDeps.ItemRepo = itemRepo
compatDeps.SeasonRepo = seasonRepo
compatDeps.EpisodeRepo = episodeRepo
compatDeps.ProviderIDRepo = providerIDRepo
compatDeps.DetailSvc = detailSvc
compatDeps.FolderRepo = folderRepo
compatDeps.SessionMgr = sessionMgr
compatDeps.UserStoreProvider = userStoreProvider
compatDeps.WatchCompletionObserver = deps.WatchCompletionObserver
compatDeps.SettingsRepo = settingsRepo
compatDeps.PersonRepo = personRepo
compatSearchService := catalog.NewCatalogSearchService(
appCtx,
settingsRepo,
itemRepo,
catalog.NewSearchIndexEventRepository(deps.DB),
deps.CatalogSearchVectorizer,
)
if compatSearchService != nil {
compatSearchService.StartCoverageRefresh(appCtx)
compatDeps.CatalogSearchProvider = compatSearchService.Provider()
// Latch the resolved provider for the package-level enqueue
// helpers (idempotent with the API router's latch; this also
// covers modes that wire jellycompat without the router).
activeSearchProvider := catalog.SearchProviderPostgres
if _, ok := compatSearchService.Provider().(*catalog.MeilisearchSearchProvider); ok {
activeSearchProvider = catalog.SearchProviderMeilisearch
}
catalog.SetActiveSearchIndexProvider(activeSearchProvider)
}
if deps.S3Public != nil {
compatDeps.PosterPresigner = deps.S3Public
compatDeps.S3Client = deps.S3Public
compatDeps.S3Bucket = deps.S3Public.Bucket()
}
if deps.FileRepo != nil {
compatDeps.FileResolver = deps.FileRepo
}
compatDeps.SubtitleRepo = subtitles.NewPgRepository(deps.DB, deps.SecretCipher)
// Construct auth service for jellycompat login.
userRepo := auth.NewUserRepository(deps.DB)
compatDeps.APIKeyValidator = auth.NewAPIKeyRepository(deps.DB)
compatDeps.APIKeyUserLoader = userRepo
compatDeps.ScanQueue = deps.LibraryScanQueue
sessionRepo := auth.NewSessionRepository(deps.DB)
jwtService := auth.NewJWTService(
cfg.Auth.JWTSecret,
cfg.Auth.AccessTokenExpiry,
cfg.Auth.RefreshTokenExpiry,
)
configWatcher.OnChange(func(_, updated *config.Config) {
jwtService.SetExpiries(updated.Auth.AccessTokenExpiry, updated.Auth.RefreshTokenExpiry)
})
provider := auth.NewLocalProvider(userRepo, sessionRepo)
compatDeps.AuthService = auth.NewService(provider, jwtService, sessionRepo, userRepo, nil, nil, nil)
// Access filter resolver for viewer-scoped library access.
// Backed by the shared access.Resolver so account-level library
// restrictions (users.library_ids), profile restrictions,
// user-disabled libraries, and rating/quality ceilings apply to
// the compat API exactly as they do to the native API.
if userStoreProvider != nil {
var compatScopeResolver jellycompat.ScopeResolver
if policySystem != nil {
compatScopeResolver = policy.NewViewerResolver(
userRepo,
userStoreProvider,
nil, // profile tokens unused: compat login already verifies PINs
policySystem.PDP(),
accessGroupStore,
)
} else {
// Legacy resolver: proxy/test wiring without a policy system. Production integrated/api modes always take the policy path. Removed with the legacy cleanup phase.
compatScopeResolver = access.NewResolver(
userRepo,
userStoreProvider,
nil, // profile tokens unused: compat login already verifies PINs
accessGroupStore,
)
}
compatDeps.AccessFilterFn = jellycompat.NewScopeAccessFilter(compatScopeResolver)
}
}
compat := jellycompat.NewServerWithDependencies(compatDeps)
compatServer = compat
compat.StartBackgroundTasks(context.Background())
compatSrv = compat.HTTPServer()
compatSrv.ReadTimeout = 30 * time.Second
compatSrv.WriteTimeout = 0
compatSrv.IdleTimeout = 120 * time.Second
}
// ABS-compat listener — dedicated http.Server bound to its own port
// (default :13378) that hosts the Audiobookshelf-compatible API.
// Mirrors the Jellyfin compat layout above. The ABS handler mounts
// onto a fresh chi router here so /ping, /healthcheck, /status, /login,
// /socket.io, etc. own the URL space at the root — no SPA fallback,
// no collision with silo's /api/v1.
var absSrv *http.Server
if (mode == "integrated" || mode == "api") && deps.ABSHandler != nil && cfg.AudiobookshelfCompat.Listen != "" {
absRouter := chi.NewRouter()
absRouter.Use(chimiddleware.Recoverer)
absRouter.Use(chimiddleware.Compress(5))
deps.ABSHandler.Mount(absRouter)
absSrv = &http.Server{
Addr: cfg.AudiobookshelfCompat.Listen,
Handler: absRouter,
ReadHeaderTimeout: 10 * time.Second,
ReadTimeout: 60 * time.Second,
WriteTimeout: 0,
IdleTimeout: 120 * time.Second,
}
}
// Run non-critical startup work in the background so it doesn't delay the
// HTTP listener from accepting connections. Steps run sequentially and stop
// early if the app context is cancelled (shutdown).
if len(backgroundInit) > 0 {
go func() {
start := time.Now()
for _, step := range backgroundInit {
if appCtx.Err() != nil {
return
}
func() {
defer func() {
if p := recover(); p != nil {
slog.Error("deferred startup init step panicked; continuing",
"panic", p, "stack", string(debug.Stack()))
}
}()
step(appCtx)
}()
}
slog.Info("deferred startup init completed", "steps", len(backgroundInit), "duration", time.Since(start))
}()
}
errCh := make(chan error, 3)
go func() {
slog.Info("HTTP server listening", "addr", cfg.Server.Listen)
if listenErr := srv.ListenAndServe(); listenErr != nil && listenErr != http.ErrServerClosed {
errCh <- fmt.Errorf("HTTP server error: %w", listenErr)
}
}()
if compatSrv != nil {
go func() {
slog.Info("Jellyfin compat server listening", "addr", compatSrv.Addr)
if listenErr := compatSrv.ListenAndServe(); listenErr != nil && listenErr != http.ErrServerClosed {
errCh <- fmt.Errorf("jellyfin compat server error: %w", listenErr)
}
}()
}
if absSrv != nil {
go func() {
slog.Info("ABS compat server listening", "addr", absSrv.Addr)
if listenErr := absSrv.ListenAndServe(); listenErr != nil && listenErr != http.ErrServerClosed {
errCh <- fmt.Errorf("abs compat server error: %w", listenErr)
}
}()
}
// Step 11: Wait for termination signal.
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT)
defer signal.Stop(sigCh)
select {
case sig := <-sigCh:
appCancel()
slog.Info("received signal, shutting down", "signal", sig)
case <-restartReqCh:
appCancel()
slog.Info("server restart requested, shutting down")
case serverErr := <-errCh:
appCancel()
slog.Error("server error, shutting down", "error", serverErr)
}
// Step 12: Graceful shutdown sequence.
slog.Info("beginning graceful shutdown")
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
// 1. Stop accepting new requests.
if shutdownErr := srv.Shutdown(shutdownCtx); shutdownErr != nil {
slog.Error("HTTP shutdown error", "error", shutdownErr)
}
if compatSrv != nil {
if shutdownErr := compatSrv.Shutdown(shutdownCtx); shutdownErr != nil {
slog.Error("jellyfin compat shutdown error", "error", shutdownErr)
}
}
if absSrv != nil {
if shutdownErr := absSrv.Shutdown(shutdownCtx); shutdownErr != nil {
slog.Error("abs compat shutdown error", "error", shutdownErr)
}
}
// 2. Clean up stale sessions.
if sessionCleaner != nil {
cleaned, cleanErr := sessionCleaner.CleanStale(shutdownCtx)
if cleanErr != nil {
slog.Error("stale session cleanup error", "error", cleanErr)
} else if cleaned > 0 {
slog.Info("cleaned stale sessions", "count", cleaned)
}
}
// 2b. Remove this node's heartbeat and sessions from shared state.
if heartbeatWriter != nil {
if err := heartbeatWriter.CleanupSelf(shutdownCtx); err != nil {
slog.Error("heartbeat cleanup error", "error", err)
}
}
// 3. Close user store provider.
if userStoreProvider != nil {
if closeErr := userStoreProvider.Close(); closeErr != nil {
slog.Error("user store provider close error", "error", closeErr)
}
}
// 4. (match worker is now managed by the task manager — no separate cancel needed)
// Suppress unused variable warnings for workers used only in deferred calls.
_ = reconciler
_ = heartbeatWriter
_ = refreshWorker
_ = adminJobRunner
slog.Info("server stopped")
}
// startStandaloneServer runs a standalone HTTP server for proxy/transcode modes.
// It listens on the given address, handles graceful shutdown on SIGTERM/SIGINT.
func startStandaloneServer(addr string, handler http.Handler) {
srv := &http.Server{
Addr: addr,
Handler: handler,
ReadTimeout: 30 * time.Second,
WriteTimeout: 0, // no timeout for long streams
IdleTimeout: 120 * time.Second,
}
errCh := make(chan error, 1)
go func() {
slog.Info("HTTP server listening", "addr", addr)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
errCh <- fmt.Errorf("HTTP server error: %w", err)
}
}()
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT)
select {
case sig := <-sigCh:
slog.Info("received signal, shutting down", "signal", sig)
case serverErr := <-errCh:
slog.Error("server error, shutting down", "error", serverErr)
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := srv.Shutdown(shutdownCtx); err != nil {
slog.Error("HTTP shutdown error", "error", err)
}
slog.Info("server stopped")
}
// newS3ClientIfConfigured creates an S3 client only if the bucket name is
// configured. Returns nil if the bucket is empty (not configured).
func newS3ClientIfConfigured(cfg s3client.BucketConfig) *s3client.Client {
if cfg.Bucket == "" {
return nil
}
return s3client.NewClient(cfg)
}
func configureS3Clients(cfg *config.Config, deps *api.Dependencies) {
if s3Public := newS3ClientIfConfigured(s3client.BucketConfig{
Endpoint: cfg.S3.Public.Endpoint,
PublicEndpoint: cfg.S3.Public.ReadEndpoint,
Region: cfg.S3.Public.Region,
Bucket: cfg.S3.Public.Bucket,
KeyPrefix: cfg.S3.Public.KeyPrefix,
AccessKey: cfg.S3.Public.AccessKey,
SecretKey: cfg.S3.Public.SecretKey,
PathStyle: cfg.S3.Public.PathStyle,
URLAuth: cfg.S3.Public.URLAuth,
TokenSecret: cfg.S3.Public.TokenSecret,
TokenParam: cfg.S3.Public.TokenParam,
TokenTTL: cfg.S3.Public.TokenTTL,
}); s3Public != nil {
deps.S3Public = s3Public
slog.Info("S3 public assets client configured", "bucket", s3Public.Bucket())
// Allow browsers to fetch presigned client-facing assets directly from S3.
// Skip for public/token auth (e.g. Cloudflare R2) where CORS is managed externally.
if !s3Public.UsesExternalAuth() {
corsCtx, corsCancel := context.WithTimeout(context.Background(), 10*time.Second)
if corsErr := s3Public.SetBucketCORS(corsCtx, s3Public.Bucket(), []string{"*"}); corsErr != nil {
slog.Warn("failed to set CORS on public assets bucket", "error", corsErr)
}
corsCancel()
}
}
if s3Private := newS3ClientIfConfigured(s3client.BucketConfig{
Endpoint: cfg.S3.Private.Endpoint,
Region: cfg.S3.Private.Region,
Bucket: cfg.S3.Private.Bucket,
KeyPrefix: cfg.S3.Private.KeyPrefix,
AccessKey: cfg.S3.Private.AccessKey,
SecretKey: cfg.S3.Private.SecretKey,
PathStyle: cfg.S3.Private.PathStyle,
}); s3Private != nil {
deps.S3Private = s3Private
slog.Info("S3 private internal client configured", "bucket", s3Private.Bucket())
if !s3Private.UsesExternalAuth() {
corsCtx, corsCancel := context.WithTimeout(context.Background(), 10*time.Second)
if corsErr := s3Private.SetBucketCORS(corsCtx, s3Private.Bucket(), []string{"*"}); corsErr != nil {
slog.Warn("failed to set CORS on private assets bucket", "error", corsErr)
}
corsCancel()
}
}
if s3UserDB := newS3ClientIfConfigured(s3client.BucketConfig{
Endpoint: cfg.S3.UserDB.Endpoint,
Region: cfg.S3.UserDB.Region,
Bucket: cfg.S3.UserDB.Bucket,
KeyPrefix: cfg.S3.UserDB.KeyPrefix,
AccessKey: cfg.S3.UserDB.AccessKey,
SecretKey: cfg.S3.UserDB.SecretKey,
PathStyle: cfg.S3.UserDB.PathStyle,
}); s3UserDB != nil {
deps.S3UserDB = s3UserDB
slog.Info("S3 user-db client configured", "bucket", s3UserDB.Bucket())
}
}
type pluginImageResolverCapabilityStore interface {
ListEnabled(ctx context.Context) ([]*plugins.Installation, error)
ListCapabilities(ctx context.Context, installationID int) ([]*plugins.Capability, error)
}
func reloadPluginImageResolvers(
ctx context.Context,
store pluginImageResolverCapabilityStore,
resolver *metadata.PluginImageResolver,
service *plugins.Service,
) error {
if resolver == nil {
return nil
}
if store == nil || service == nil {
resolver.ReplaceSources(nil)
return nil
}
installations, err := store.ListEnabled(ctx)
if err != nil {
return fmt.Errorf("list enabled plugin installations: %w", err)
}
sort.Slice(installations, func(i, j int) bool {
if installations[i] == nil {
return false
}
if installations[j] == nil {
return true
}
return installations[i].ID < installations[j].ID
})
var registrations []metadata.PluginImageResolverSourceRegistration
for _, installation := range installations {
if installation == nil {
continue
}
capabilities, err := store.ListCapabilities(ctx, installation.ID)
if err != nil {
return fmt.Errorf("list image resolver capabilities for installation %d: %w", installation.ID, err)
}
sort.Slice(capabilities, func(i, j int) bool {
if capabilities[i] == nil {
return false
}
if capabilities[j] == nil {
return true
}
if capabilities[i].Type != capabilities[j].Type {
return capabilities[i].Type < capabilities[j].Type
}
return capabilities[i].ID < capabilities[j].ID
})
for _, capability := range capabilities {
if capability == nil {
continue
}
switch capability.Type {
case sdkcapability.ImageResolver:
schemes, priority := imageResolverCapabilityConfig(capability)
if len(schemes) == 0 {
slog.Warn("plugin image resolver capability has no valid schemes",
"installation_id", installation.ID,
"capability_id", capability.ID)
continue
}
for _, scheme := range schemes {
source := metadata.NewPluginClientSource(installation.ID, capability.ID, func(
ctx context.Context, installationID int, capabilityID string,
) (metadata.PluginMetadataClient, error) {
return service.ImageResolverClient(ctx, installationID, capabilityID)
})
registrations = append(registrations, metadata.PluginImageResolverSourceRegistration{
Scheme: scheme,
Source: source,
Kind: metadata.PluginImageResolverSourceExplicit,
Priority: priority,
InstallationID: installation.ID,
CapabilityID: capability.ID,
})
}
case sdkcapability.MetadataProvider:
scheme := strings.TrimSpace(capability.ID)
if !metadata.ValidImageResolverScheme(scheme) {
slog.Warn("skipping legacy metadata image resolver with invalid scheme",
"installation_id", installation.ID,
"capability_id", capability.ID)
continue
}
source := metadata.NewPluginClientSource(installation.ID, capability.ID, func(
ctx context.Context, installationID int, capabilityID string,
) (metadata.PluginMetadataClient, error) {
return service.MetadataProviderClient(ctx, installationID, capabilityID)
})
registrations = append(registrations, metadata.PluginImageResolverSourceRegistration{
Scheme: scheme,
Source: source,
Kind: metadata.PluginImageResolverSourceLegacy,
InstallationID: installation.ID,
CapabilityID: capability.ID,
})
}
}
}
resolver.ReplaceSources(registrations)
slog.Info("reloaded plugin image resolvers", "sources", len(registrations))
return nil
}
func imageResolverCapabilityConfig(capability *plugins.Capability) ([]string, int) {
if capability == nil {
return nil, 0
}
meta := capabilityMetadataFields(capability.Metadata)
return metadataStringList(meta["schemes"]), metadataInt(meta["priority"])
}
func capabilityMetadataFields(raw map[string]any) map[string]any {
if raw == nil {
return nil
}
if nested, ok := raw["metadata"]; ok {
switch typed := nested.(type) {
case map[string]any:
return typed
}
}
return raw
}
func metadataStringList(value any) []string {
var out []string
switch typed := value.(type) {
case []string:
for _, item := range typed {
if scheme := strings.TrimSpace(item); metadata.ValidImageResolverScheme(scheme) {
out = append(out, scheme)
}
}
case []any:
for _, item := range typed {
text, ok := item.(string)
if !ok {
continue
}
if scheme := strings.TrimSpace(text); metadata.ValidImageResolverScheme(scheme) {
out = append(out, scheme)
}
}
}
return out
}
func metadataInt(value any) int {
switch typed := value.(type) {
case int:
return typed
case int32:
return int(typed)
case int64:
return int(typed)
case float64:
return int(typed)
case json.Number:
n, _ := typed.Int64()
return int(n)
default:
return 0
}
}
type markerPluginCapabilityStore interface {
ListEnabled(ctx context.Context) ([]*plugins.Installation, error)
ListCapabilities(ctx context.Context, installationID int) ([]*plugins.Capability, error)
}
type markerPluginRuntimeConfigStore interface {
ListGlobalConfigs(ctx context.Context, installationID int) ([]*plugins.RuntimeConfig, error)
PutGlobalConfig(ctx context.Context, installationID int, key string, value map[string]any) error
}
type markerLegacySettingsStore interface {
Get(ctx context.Context, key string) (string, error)
}
func reloadMarkerPluginProviders(
ctx context.Context,
registry *markers.Registry,
configStore *markers.ProviderConfigStore,
store markerPluginCapabilityStore,
runtimeConfigs markerPluginRuntimeConfigStore,
legacySettings markerLegacySettingsStore,
resolver *markers.PluginResolverAdapter,
) error {
if registry == nil {
return nil
}
var providers []markers.Provider
if store == nil || resolver == nil {
return registry.SetProviders(providers)
}
installations, err := store.ListEnabled(ctx)
if err != nil {
return fmt.Errorf("list enabled plugin installations: %w", err)
}
sort.Slice(installations, func(i, j int) bool {
if installations[i] == nil {
return false
}
if installations[j] == nil {
return true
}
return installations[i].ID < installations[j].ID
})
nextPriority := 1000
for _, installation := range installations {
if installation == nil {
continue
}
capabilities, err := store.ListCapabilities(ctx, installation.ID)
if err != nil {
return fmt.Errorf("list marker provider capabilities for installation %d: %w", installation.ID, err)
}
sort.Slice(capabilities, func(i, j int) bool {
if capabilities[i] == nil {
return false
}
if capabilities[j] == nil {
return true
}
return capabilities[i].ID < capabilities[j].ID
})
for _, capability := range capabilities {
if capability == nil || capability.Type != sdkcapability.MarkerProvider {
continue
}
descriptor, err := plugins.DecodeCapability(capability)
if err != nil {
return fmt.Errorf("decode marker provider capability %d/%s: %w", installation.ID, capability.ID, err)
}
metadataMap := markerCapabilityMetadata(descriptor)
provider, err := markers.NewPluginProvider(markers.PluginProviderOptions{
InstallationID: installation.ID,
CapabilityID: capability.ID,
DisplayName: firstNonEmptyMarkerText(descriptor.GetDisplayName(), capability.ID),
PluginID: installation.PluginID,
RequiredExternalIDs: markers.PluginRequiredExternalIDsFromMetadata(metadataMap),
}, resolver)
if err != nil {
return err
}
providers = append(providers, provider)
priority := nextPriority
nextPriority++
if configuredPriority, ok := markers.PluginDefaultFetchPriorityFromMetadata(metadataMap); ok {
priority = configuredPriority
}
if configStore != nil {
defaultConfig := markers.ProviderConfig{
Provider: provider.ID(),
FetchEnabled: true,
FetchPriority: priority,
ContributeEnabled: false,
ContributeAutoLocal: false,
ContributeMinConfidence: 0.95,
}
if legacy, ok := legacyIntroDBProviderConfig(configStore, installation, capability, provider.ID()); ok {
defaultConfig = legacy
}
if err := configStore.Ensure(ctx, defaultConfig); err != nil {
return err
}
}
if err := copyLegacyIntroDBPluginConfig(ctx, runtimeConfigs, legacySettings, installation, capability); err != nil {
return err
}
}
}
return registry.SetProviders(providers)
}
func legacyIntroDBProviderConfig(
configStore *markers.ProviderConfigStore,
installation *plugins.Installation,
capability *plugins.Capability,
providerID string,
) (markers.ProviderConfig, bool) {
if configStore == nil ||
installation == nil ||
capability == nil ||
installation.PluginID != "silo.theintrodb" ||
capability.ID != "introdb" {
return markers.ProviderConfig{}, false
}
if _, exists := configStore.Get(providerID); exists {
return markers.ProviderConfig{}, false
}
legacy, ok := configStore.Get("introdb")
if !ok {
return markers.ProviderConfig{}, false
}
legacy.Provider = providerID
return legacy, true
}
func copyLegacyIntroDBPluginConfig(
ctx context.Context,
runtimeConfigs markerPluginRuntimeConfigStore,
legacySettings markerLegacySettingsStore,
installation *plugins.Installation,
capability *plugins.Capability,
) error {
if runtimeConfigs == nil ||
legacySettings == nil ||
installation == nil ||
capability == nil ||
installation.PluginID != "silo.theintrodb" ||
capability.ID != "introdb" {
return nil
}
configs, err := runtimeConfigs.ListGlobalConfigs(ctx, installation.ID)
if err != nil {
return fmt.Errorf("list TheIntroDB plugin config: %w", err)
}
for _, config := range configs {
if config != nil && config.Key == "account" {
return nil
}
}
apiKey, err := legacySettings.Get(ctx, "introdb.api_key")
if err != nil {
return fmt.Errorf("load legacy introdb.api_key: %w", err)
}
if strings.TrimSpace(apiKey) == "" {
return nil
}
if err := runtimeConfigs.PutGlobalConfig(ctx, installation.ID, "account", map[string]any{
"api_key": strings.TrimSpace(apiKey),
}); err != nil {
return fmt.Errorf("copy legacy introdb.api_key to plugin config: %w", err)
}
return nil
}
func markerCapabilityMetadata(descriptor *pluginv1.CapabilityDescriptor) map[string]any {
if descriptor == nil || descriptor.GetMetadata() == nil {
return nil
}
return descriptor.GetMetadata().AsMap()
}
func firstNonEmptyMarkerText(values ...string) string {
for _, value := range values {
if strings.TrimSpace(value) != "" {
return strings.TrimSpace(value)
}
}
return ""
}
// mapFolderTypeToMediaType maps silo's MediaFolder.Type values
// ("movies", "series", "mixed") to the SDK's MediaType values
// ("movie", "tv", "mixed"). Unknown values map to "mixed".
func mapFolderTypeToMediaType(t string) string {
switch t {
case "movies":
return "movie"
case "series":
return "tv"
default:
return "mixed"
}
}
type scopeResolver interface {
Resolve(ctx context.Context, input access.ResolveInput) (access.Scope, error)
}
type scopeEntitlementResolver struct {
resolver scopeResolver
}
func (r scopeEntitlementResolver) MaxPlaybackQuality(ctx context.Context, userID int, profileID string) (string, error) {
scope, err := r.resolver.Resolve(ctx, access.ResolveInput{
UserID: userID,
ProfileID: profileID,
SkipPINVerification: true,
})
if err != nil {
return "", err
}
return scope.MaxPlaybackQuality, nil
}
// audiobooksSettingsAdapter bridges catalog.ServerSettingsRepo (which
// exposes Get) to the audiobooks.SettingsReader interface (which
// requires GetString). The two signatures are identical modulo name.
type audiobooksSettingsAdapter struct {
repo catalog.SettingsStore
}
func (a *audiobooksSettingsAdapter) GetString(ctx context.Context, key string) (string, error) {
return a.repo.Get(ctx, key)
}