codex/bound-transcode-segments
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
f235524365 |
feat(diagnostics): chunked report upload fallback for proxy body caps (#494)
* feat(diagnostics): chunked report upload fallback for proxy body caps
Diagnostics bundles can be up to max_bundle_bytes (10 MiB default), but a
reverse proxy in front of Silo commonly caps request bodies at nginx's
default client_max_body_size of 1 MiB. Such a proxy answers the single-shot
multipart upload with its own 413 before Silo ever sees the request, so any
report over the cap could never be delivered.
Add a chunked upload fallback under /api/v1/diagnostics/reports/uploads:
- POST / {manifest, bundle_bytes} opens a session
- PUT /{id}/chunks/{index} streams one ≤768 KiB chunk (proxy-safe)
- POST /{id}/complete ingests the assembled bundle
- DELETE /{id} best-effort abandon
The assembled bundle goes through the exact same Ingest path as the
single-shot endpoint, so every content check (manifest contract, archive
sha/bytes/entries, quotas, profile attribution) applies identically.
Sessions reuse internal/uploads (the plugin chunked-upload spool manager)
plus a small owner map for per-user isolation; they spool to disk, expire
after 15 minutes, cap at one per user / 16 global, and complete shares the
existing per-user + global in-flight ingest limiter.
/diagnostics/status now advertises upload_chunk_bytes so clients can detect
support; older servers omit the field and clients treat that as
unsupported. The demo guard's diagnostics prefix gains PUT to cover the
chunk route.
Verified end to end against an OpenResty proxy with a 1m body cap: the
single-shot upload 413s, the same 1.6 MiB bundle uploads in three chunks
and lands as an accepted report; also exercised from the tvOS client's
fallback path in the simulator.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(diagnostics): harden chunked upload sessions per review
- Reserve the per-user slot and global cap atomically in init (a
reservation map counted with live sessions), so concurrent inits by one
account can no longer fan out past one session or transiently exceed the
cap. Creation failures roll the reservation back.
- Move chunk body I/O outside the uploads.Manager mutex: a slow client
streaming one chunk no longer serializes every other session's chunk
writes, completes, and cancels. A per-chunk in-flight flag rejects
duplicate concurrent writes to the same offset (ErrChunkBusy → 409), and
cancel/expiry defer spool-directory removal to the last finishing
writer.
- Chunk arrivals refresh the session expiry, making the TTL an idle
timeout instead of an absolute deadline so a slow-but-progressing upload
cannot expire mid-transfer.
- Extend the request read deadline on chunk PUTs and both deadlines on
complete, matching the single-shot handler's slow-uplink handling.
- Keep the session when complete's availability re-check fails
transiently (status load error → 500): only definitive
disabled/storage-unavailable answers discard the spool, so a retried
complete succeeds without re-uploading every chunk.
- Reclaim orphaned spool directories at startup (a restart previously
stranded the old process's partial uploads forever) and sweep expired
sessions on a timer instead of only from later init traffic.
- Document that session state is process-local and what that means for
multi-replica deployments.
Adds concurrency/race tests (go test -race) for atomic admission,
same-chunk write exclusion, expiry refresh, transient-status retry, and
startup reclaim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(diagnostics): count detached chunk writers and lift chunk PUT write deadline
Second review round:
- A canceled session whose slow chunk writer was still draining held a
connection and spool disk but vanished from every count, so a
cancel-and-reinit loop could stack unbounded live writers behind the
16-session cap. The uploads manager now parks such sessions in a
detached set (exposed as DetachedWriterSessions) until their last
writer returns, and diagnostics init counts them in its admission gate.
- Chunk PUTs now extend the write deadline as well as the read deadline:
on an uplink slow enough to eat the server's 120s WriteTimeout, the
stored chunk's JSON acknowledgement would otherwise be lost and the
client would retry an already-accepted chunk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Quick104
|
3c56ed606c | fix(admin): enforce settings contracts end to end | ||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
1f2125b920 |
feat(plugins): group Apps sidebar by plugin manifest category (#366)
* feat(plugins): group Apps sidebar by plugin manifest category Implements the plugin SDK's documented PluginManifest.category semantics (silo-plugin-sdk proto/silo/plugin/v1/common.proto): a slash-delimited path that groups plugins in the user-facing Apps section, e.g. "Books/Audiobooks" lands under Apps -> Books. The field existed in the manifest proto but silo-server never surfaced it. Server: the user plugin-settings list/detail responses now include an additive-only `category,omitempty` string sourced from the already-loaded manifest via GetCategory(); no new parsing paths. Web: PluginSettingsSummary gains `category?: string`, and AppSidebar groups Apps entries by the FIRST segment of the category path (one level of grouping for now; deeper segments intentionally ignored, documented against the SDK contract). When fewer than 2 distinct categories exist among the visible app links, today's flat list under the single "Apps" header is kept; with 2+ categories, per-category sub-headers render via the existing SidebarSectionHeader (labels hide in the collapsed sidebar the same way other section headers do). Uncategorized plugins fall under "Other", which always sorts last. Tests: Go unit tests for the summary converter (category passthrough and JSON omission when empty) and vitest coverage for the pure groupAppNavLinks helper plus grouped/flat/collapsed sidebar rendering. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(plugins): use generic category examples in comments and tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(web): simplify Apps sidebar link list rendering - fold the duplicated <ul> list markup in the grouped and flat Apps branches into a single renderAppNavList helper so the list styling cannot drift between the two render paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandGitHub
|
10e15798e0 | feat(plugins): add approved community catalog hub (#355) | ||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a26b2de454 |
fix(metadata): stop specialist providers hijacking new library chains (#294)
* fix(metadata): seed specialist providers off and scope chains to declared levels New library provider chains were seeded from every enabled metadata provider, ordered purely by each plugin's declared default_priority and enabled whenever that priority was > 0. Two consequences: - A specialist provider (e.g. silo.sportarr, which declares series/season/ episode) could out-rank the general providers and land at position 1, enabled, on every new TV series library. - Single-purpose providers that declare only their own level (audiobook / ebook / manga metadata) were still attached as disabled rows to series and movie libraries, cluttering the chain editor with providers that cannot serve that content. Introduce a `default_enabled` capability-metadata flag (defaults to true, so every existing plugin is unaffected). A provider sets it false to be seeded installed-but-disabled while keeping its declared priority, so a user can opt in per-library and it slots in where the manifest intends instead of jumping to the top. At the same time, seedDefaultChain and AppendProviderToAllChains now drop providers that do not declare a content level, reusing the same providerSupportsLevel rule as the chain-less fallback (issue #106). LookupSeedPlacement resolves support/priority/enabled with a single metadata fetch. buildSeededChainEntries is extracted as a pure, unit-tested helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): standardize metadata provider slug casing in library chain editor The library provider-chain editor showed the same provider differently depending on where the chain came from: a freshly defaulted chain used the capability display name ("TMDB"), while a chain loaded from the server used the capability id ("tmdb", which the API returns as provider_slug). So a provider read one way before saving and another after, and differed between library types depending on which levels already had a saved chain. Standardize on the capability id everywhere (matches the server's provider_slug and the mono/slug styling). Extract the provider mapping into a pure, unit-tested metadataProvidersFromInstallations helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): mirror server seeding rules in the library form's default chain The form builds its own default chain client-side, and any touch (including changing the library type on create, the normal path for a series library) marks it dirty and POSTs it after create — replacing the server-seeded chain. That chain still enabled every provider with a declared priority and listed unsupported providers as disabled rows, so the server-side fix evaporated on the UI create path. buildDefaultLevelChains now applies the same rules as buildSeededChainEntries: providers that don't declare the level are dropped, a declaring provider is enabled only if it doesn't opt out via default_enabled, and a legacy catch-all (no declared levels) is parked last, disabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web,api): serve default provider chains from the server Replace the form's client-side reimplementation of the seeding rules with a new additive endpoint, GET /api/v1/libraries/provider-defaults?library_type=X, which returns the exact chain seedDefaultChain would write for that type. The create form now renders those server-computed defaults, changing the library type just refetches them (no longer marking the chain dirty), and a create with an untouched chain lets the server-seeded chain stand instead of writing one back. Editing an existing library uses the same defaults to fill levels its saved chain doesn't cover. Types the server seeds no metadata levels for (e.g. podcasts) return an empty levels map rather than an error. This removes buildDefaultLevelChains / metadataProvidersFromInstallations and the default_priority/default_enabled manifest parsing from the frontend — one source of truth for default ordering and enablement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): show a loading state in the provider chain editor While the server chain (for an existing library) or the type's defaults are still in flight, the editor rendered empty provider lists for a moment. Show a spinner row instead; local edits always render immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
Quick
|
2045b7a0b2 | feat(plugins): add image resolver registry | ||
|
|
5afe56cfc0 |
feat(jellycompat): add runtime-managed Jellyfin Web compatibility (#77)
* feat(jellycompat): install web assets at runtime * fix(jellycompat): recover stale web operation locks * fix(jellycompat): harden web component management * feat(admin): refine compat settings and restart status * chore(dev): add hot-reload docker compose stack * fix(dev): include npm in hot-reload backend * feat(admin): refine Jellyfin compatibility settings * feat(settings): improve jellyfin proxy summary * feat(settings): improve jellyfin web controls * fix(settings): update jellyfin web removal status * fix(settings): enable jellyfin web after install * feat(jellycompat): auto-select web ui version * test(api): update rate limit handler setup * feat(jellycompat): refine web ui install onboarding * fix(jellycompat): address web ui install review issues * fix(onboarding): mirror jellyfin api runtime status * fix(admin): remove global restart banner * fix(settings): gate restart required tracking * fix(jellyfin): ignore live settings for restart status * fix(jellyfin): avoid restart for live compat settings * fix(subtitles): normalize AI language codes * fix(catalog): support partial title search tokens * feat(branding): add white-label customization * Add push relay engineering plan - Document relay API contracts, APNs/FCM behavior, auth, storage, and ops - Capture implementation plan, provider references, decisions, and README --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
2c714e4ef2 |
feat(requests): pluginize request fulfillment behind request_router.v1 (#104)
* docs: design spec for pluginizing requests fulfillment Pluginize the requests fulfillment backend behind an agnostic request_router.v1 capability (high seam: whole-request fulfiller). Host keeps lifecycle/quota/policy/quality-governance and a generic two-tier connection registry; plugins own routing+submission+status. First plugin extracts multi-instance Sonarr/Radarr; Seerr follows in a separate spec. Preserves autoscan reuse of arr connection rows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for requests pluginization Three-phase plan: (1) request_router.v1 SDK capability, (2) new silo-plugin-requests-arr plugin extracting multi-instance Sonarr/Radarr, (3) host refactor routing fulfillment through the plugin while keeping quality governance, target records, and autoscan connection reuse host-side. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db): generalize request_integrations into a two-tier connection registry Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): add generic connection fields to Integration + repo mapping * feat(pluginhost): typed RequestRouter capability client + resolver Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): plugin-backed RequestRouterProvider seam Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): route fulfillment through RequestRouterProvider; host keeps quality governance Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): base auto-approve gate on router connection model Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): wire plugin-backed request router at both service sites Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(requests): remove in-host Sonarr/Radarr fulfillment code * test(autoscan): lock request-integration reuse after connection generalization Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): plugin-driven request integration config form Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): echo router connection fields in integration response * fix(requests): retry dropped qualities, contain to one router installation, dedupe targets Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): harden plugin trust boundary (validate targets, contain bad connections, media-type routing) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): tighten auto-approve gate, restore default/4k validation, propagate config-encode error, drop itoa wrapper, test status/options translation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(requests): resolve integrations/settings/secrets once per reconcile cycle Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): dedupe config helpers, preserve zero profile id, stabilize installation default, drop redundant options write Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for schema-driven plugin config form Extends AdminFormDescriptor into a full form-description language (dynamic options, multi-select, conditional visibility, sections, validation) + a plugin Validate RPC, rendered by one reusable SchemaForm engine. Retires the bespoke arr connection form and integrationOptionsFromRouter so any request_router backend renders its config UI from manifest data with zero host changes. Addresses code-review finding #9. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for schema-driven plugin config form Six phases: SDK AdminFormDescriptor extensions + Validate RPC; reusable SchemaForm renderer (refactor PluginConfigForm onto it); host Validate plumbing + generic options + legacy-column derivation + retire integrationOptionsFromRouter; requests admin page swap to SchemaForm with per-plugin grouping; arr manifest enrichment + Validate impl; verification. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): extend plugin admin-form TS types (sections, conditions, validation, multi-select) * feat(web): schema-form pure utils (show_when, validation, value coercion) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): SchemaForm renderer (controls, sections, show_when, dynamic options, errors) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(web): render PluginConfigForm via the shared SchemaForm engine Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): RequestRouter Validate client + provider seam Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): plugin Validate on save, generic options, derive legacy columns from plugin_config Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): generic options response + 400 field_errors on plugin validation failure * feat(web): generic request-integration options type + surface validation field_errors Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): render request connections via SchemaForm; per-plugin grouping; retire bespoke arr form Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): silent connection-options probe with inline failure status (no toast spam) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): serialize admin_form sections/show_when/dynamic_options/validation to the client Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): drop show_when-hidden fields from buildSchemaValues payload Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): pass requester user id as int64 (no truncation) * refactor(requests): drop legacy arr columns; plugin_config is sole source of truth Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): backfill api key in plugin validate; centralize validation 400; drop duplicate host cross-field check; guard admin-form serializer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): refuse stored api key reuse when base_url changes (security hardening) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): SchemaForm regex-guard, default_value, type-driven coercion, validity callback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): connection-options latest-wins + narrowed deps + clear stale errors; auto-select; type-driven persist; reuse types Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for silo-plugin-requests-seerr (request_router.v1 backend) * docs: implementation plan for silo-plugin-requests-seerr * docs(spec): FindExistingRequest uses /api/v1/request (carries request id) * docs(spec): seerr hardening — id-recovery, 404 terminal, media-status, sort pin, single missing-tmdb message * docs: design spec for shared plugin-platform SDK helpers (code-review #10) * docs: plan for plugin-platform SDK helpers (#10) + spec fix (inline broker wiring, no import cycle) * docs: design spec for typed 4K quality-tier signal (code-review #9) * docs: implementation plan for typed 4K quality-tier signal (#9) * feat(requests): stamp is4k per quality (host owns the 4K-tier fact) * fix(requests): store capability sub-id, not the type, in request_integrations request_integrations.capability_id carried the capability TYPE ("request_router.v1") instead of the capability sub-id ("arr"/"seerr"). The host resolves a router plugin via requireCapability("request_router.v1", id), which keys on (type, id), so storing the type resolved to no capability: every save/options/fulfill 500'd ("Request operation failed" / "no fulfillment backend configured") in ~1ms, before the arr/Seerr API was ever contacted. The path was internally split-brained (the fulfillment filter matched the type while the dispatcher needed the sub-id), so it never worked end-to-end; the unit tests hid it behind a fake provider that skips requireCapability. Align capability_id with the scan_source/metadata convention (sub-id): - validateInstance: require a non-empty sub-id; drop the default-to-type and the "!= request_router.v1" reject. - resolveRouterConnections / integrationConfigured / unbound-guidance: match on a non-empty capability, not type equality. - repository: persist capability_id verbatim (never default to the type). - web AdminRequests: send the selected plugin's capability.id in both the options probe and the save payload (was a hardcoded type constant). - migration 20260608131649: backfill capability_id from each bound installation's request_router.v1 capability and drop the column's misleading default. Unbound legacy rows are left for admin re-save. Tests: validateInstance now requires the sub-id, and the selected sub-id must reach the plugin Validate RPC (fakeRouterProvider records it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): polish request connection cards (grouped toggles + option loading states) The schema-driven connection cards rendered each boolean as its own bordered, double-labeled box and showed dynamic SELECTs (root folder, quality profile, tags) as empty controls with a single "Loading options…" line while the host probed the service. - Toggles render as a cohesive settings list: consecutive switches collapse into one bordered, divided container; each row is toggle-first with the label + description hugging beside it (no stranded whitespace between a short label and its switch). Honors show_when, so conditional toggles still group correctly. - Dynamic SELECT/MULTI_SELECT fields show a per-field spinner + shimmer skeleton while options load, and only when there's nothing to show yet — a background re-probe never flashes over the operator's current value. - Sections get a softer surface and clearer titles; the card's enable switch is labeled Enabled/Disabled; the options-load failure is a proper inline alert with retry guidance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): treat "Any"/no-cap playback ceiling as 4K-allowed allowedQualities decided whether to also request 2160p with `CompareQuality(ceiling, PlaybackQuality4K) >= 0`. But an "Any" max playback quality resolves to an empty ceiling ("no cap"), and in qualityRank "" is the LOWEST rank (0) — so CompareQuality("", "2160p") returns -1 and 4K was dropped. A requester with unlimited playback quality only got a 1080p request, never the 4K one. Use access.QualityAllowed(PlaybackQuality4K, ceiling), which already encodes "empty ceiling == no cap == allows everything". Now: - "" / "Any" -> 1080p + 2160p - "2160p" -> 1080p + 2160p - "1080p" -> 1080p only - resolver error still fails safe to the HD ceiling. Tests: add an "any/no-cap ceiling adds 2160p" case; the unknown-quality, status-coercion, dedup, and per-quality-idempotency submit tests now pin an explicit HD ceiling (they relied on the old empty-default == HD-only behavior and were not about 4K entitlement). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for collapsible Library + anime gate/nesting (request card UI, Spec A) Spec A of two for the request connection card UX: Library section becomes collapsible/collapsed (auto-expanding on validation errors) and the anime override fields move into a single gated section below Library instead of popping out as a detached sibling card. Single-default enforcement is Spec B. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for collapsible Library + anime gate/nesting (Spec A) Task-by-task TDD plan: SchemaForm auto-expand-on-error + nested-field affordance (silo-server), arr manifest regroup (collapsible Library, anime gate section), then build/deploy/reinstall + manual verify. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): auto-expand collapsible schema sections that have validation errors SchemaFormSection now accepts a forceOpen prop; when any field in the section has a mergedError (client validation or server error), the section expands automatically so required-field setup can never be hidden behind a collapsed accordion. The operator's manual toggle is preserved via a nullable userOpen state that only takes effect when forceOpen is false. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): indent show_when-revealed schema fields to read as nested Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: design spec for schema-driven single-default exclusivity enforcement (Spec B) At most one connection per service_kind may be the HD default (is_default) or 4K default (is_default_4k). Generic exclusivity: a new AdminFormField exclusive_group_field declares the rule, the plugin Validate enforces it against host-supplied siblings (config only, no creds), and the admin UI auto-clears conflicts as you toggle. Host stays plugin-agnostic. Forward-only; no migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for single-default exclusivity enforcement (Spec B) Five TDD tasks across 3 repos: SDK proto (siblings + exclusive_group_field) + buf regen; arr Validate cross-sibling + manifest; host gathers siblings (config-only) into Validate; frontend generic mutual-exclusion helper; then re-vendor/rebuild/redeploy + plugininstall. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): pass sibling connections to plugin Validate for cross-connection rules Adds siblings []ResolvedRouterConnection to RequestRouterProvider.Validate so the plugin can enforce cross-connection invariants (e.g. one default per service_kind) without the host resolving sibling credentials. The new siblingConnections helper gathers other connections on the same installation, carrying only ID + PluginConfig. Vendor updated to the Task 1 SDK version that carries ValidateRequest.Siblings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): auto-clear mutually-exclusive defaults across request connection cards Adds generic applyExclusivity helper and wires it into updateCardConfig so turning on a field with exclusive_group_field proactively clears the same field on sibling cards sharing the same group value, matching server-side enforcement with a proactive UX. * docs: design spec for single-flighting plugin client launch (cold-start herd fix) Concurrent ensureClient calls for a cold installation each spawn a redundant plugin process (Host.Start releases its lock during launch). Wrap ensureClient in a per-installation singleflight.Group so concurrent first-use collapses to one launch. Host-only fix; surfaced while testing the request-router feature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for single-flighting plugin client launch TDD: concurrency tests (herd collapses to one launch, warm-cache reuse, distinct installations stay parallel, failed launch propagates) + the singleflight wrapper around ensureClient; then rebuild/redeploy + verify. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(plugins): single-flight ensureClient to prevent cold-start launch herd Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(requests): harden capability containment + dedupe eligibility; UI/migration cleanups Addresses /code-review high findings on the previously-unreviewed commits: - resolveRouterConnections contains fulfillment to the first chosen (installation, capability) and locks only after a connection's key resolves, so a plugin exposing >1 request_router capability never mixes connections and a skipped bad-key connection never pins the capability (+ test). - extract eligibleRouterConnection, shared by resolveRouterConnections and integrationConfigured so the auto-approval gate and fulfillment filter can't drift. - SchemaForm: shared FieldDescription helper (field/switch/section); key switch groups by position so a show_when reveal doesn't remount the group (focus loss). - migration backfill uses a deterministic correlated subquery instead of a join cross-product when an installation exposes multiple request_router capabilities. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for opt-in Seerr per-user requester mapping Per-connection requester_mode (admin default | mapped). In mapped mode the host pushes the requester email/username into the Fulfill descriptor and the seerr plugin resolves/creates the matching Seerr user by email with operator-chosen default permissions, attributing the request (and gating Seerr-side approval via the auto-approve permission). Spans SDK (descriptor fields), host (extend UserIdentityLookup with email + a requester resolver), and the seerr plugin (Seerr user API + mapping). Fallback to admin on any failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for Seerr per-user requester mapping Five TDD tasks across 3 repos: SDK descriptor fields (requester_email/username); host resolves identity (UserIdentityLookup+email, RequesterIdentityResolver, populate descriptor at both Fulfill sites); seerr config+user API (find/create by email, exported PermissionBits); seerr Fulfill mapping + admin_form; then re-vendor/rebuild/redeploy + plugininstall (installation 6). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: make Seerr unmapped-requester behavior a toggle (admin fallback | fail request) Per user feedback: require_mapped_user switch (default off = admin fallback, on = fail the request). Updates spec + plan Tasks 3/4 (config field, Fulfill honoring the toggle via a mapFailed signal, a new test, and the manifest switch). * feat(requests): resolve requester email/username into the Fulfill descriptor Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: design spec for simplified Seerr mapped-user permissions Reduce the 5 permission toggles to two (request_4k_all + auto_approve); 1080p always granted; remove manage_requests; 4K eligibility per-user from the request's qualities (host-decided, same as arr) with a blanket override toggle. Seerr-plugin-only; permission-only override (host still gates 4K requests). * docs: implementation plan for simplified Seerr mapped-user permissions Two tasks (seerr-plugin-only): replace the 5 perm toggles with request_4k_all + auto_approve (1080p always; 4K from request qualities via userPermissions; remove PermManageRequests/PermissionBits; manifest + json_schema), then rebuild + reinstall (installation 6). No host/SDK change. * docs: design spec for host rebase onto main + #95 credential-model adoption Per-commit rebase of our 68 request-router commits onto the force-pushed origin/main (drops 188 patch-equivalent). At the credential-path conflicts, adopt #95's inline secret.Cipher model: keep our plugin columns + #95's encrypt/decrypt in repository.go; drop our SecretResolver and read in.APIKeyRef directly in service.go; wire NewRepository(pool, dataCipher). #39-area conflicts take ours (our pluginization supersedes it). Security review + SECRET_KEY deploy note. * docs: implementation plan for host rebase + #95 credential adoption Four tasks: (1) guided per-commit rebase onto origin/main, take-ours on credential files so it builds; (2) TDD integration commit adopting #95's secret.Cipher (encrypt/decrypt in repository.go, drop SecretResolver, read APIKeyRef directly, wire NewRepository(pool, cipher)); (3) security review; (4) pin published SDK v0.6.0, push fork, open host PR with SECRET_KEY deploy note. * chore(rebase): restore scan-source service methods + temp requests-repo arity Post-rebase conflict fixups: take-ours on internal/plugins/service.go dropped origin's ScanSourceClientByPluginID (independent upstream capability) — restored. mediarequests.NewRepository temporarily 1-arg to match our pre-#95 repo; Task 2 restores the cipher arg when adopting #95's at-rest credential model. * feat(requests): adopt at-rest credential cipher (#95) for plugin api keys; drop SecretResolver * build: pin published silo-plugin-sdk v0.6.0 (drop local replace) * test(requests): guard at-rest cipher round-trip + empty-key auto-approval (code-review) Max-effort code review of the #95 credential integration. Fixes the actionable findings: - TestEncryptAPIKeyRoundTripAndAAD: pins encryptAPIKey<->DecryptIfEncrypted inversion, the id-bound apiKeyAAD == secret.RowAAD(...) match (so #95's backfill rows decrypt), the blank-key "" sentinel, and row-bound AAD — the security- critical invariants had no automated guard (no DB harness for scanIntegration). - TestCreateRequestAutoApprovalEmptyKeyTreatedAsUnconfigured: pins that a keyless connection reads as unconfigured (request stays pending, never submitted), so integrationConfigured and resolveRouterConnections can't drift. - Fix stale fulfillContext comment (referenced a resolved-API-key cache removed with SecretResolver). Assessed-not-changed (documented): decrypt-error-fails-closed and failed-backfill behaviors are origin/main #95 design we adopt; nil-cipher is unreachable in prod and matches the codebase-wide no-guard pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: drop stale machine-local SDK replace comment from go.mod The replace directive was already removed when v0.6.0 was pinned (3410df7); this leftover comment falsely claimed a local replace still existed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(web): prettier-format schema-form utils to 100-col width Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: drop internal superpowers specs/plans from PR These design specs and implementation plans are internal development artifacts; keep them out of the upstream PR diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(metadata): exclude providers from content levels they don't declare ResolveChain falls back to every enabled metadata provider when a library + content-level has no enabled chain entry. That fallback was media-type blind: a provider declaring default_priority only for an unrelated level (e.g. an audiobook provider declaring {"audiobook": N}) was kept in the list (merely sorted last) and invoked for video content levels. In production this made silo.audiobook-metadata hammer external audiobook APIs with anime/movie/series titles every scheduled enrichment pass (MatchWorker, 30s) for the season/episode levels that had no enabled chain entry. Disabling the chain entries did not help because the fallback never consults them; only disabling the installation removed it from the global set. Treat a non-empty default_priority map as the provider enumerating the content levels it supports: in resolveEnabledProvidersByPriority, exclude providers whose declared map omits the requested level instead of ranking them last. Providers that declare no default_priority make no claim and stay eligible everywhere (legacy behavior). Fixes #105 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(plugins): isolate singleflight launch from leader ctx cancellation The deduped ensureClient launch ran doEnsureClient under the leader caller's ctx, so if that caller's request was canceled/timed out mid-launch the shared plugin start was torn down and the error propagated to every waiter. Run the launch under context.WithoutCancel so a single caller cannot cancel work the other waiters depend on (values preserved for tracing/auth). (CodeRabbit) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): nil-guard request-router wiring RequestRouterClient dereferenced a.Svc unconditionally and AttachRequestRouter called SetRouterProvider even with nil deps, so a build without the plugin service would panic instead of degrading. Guard both: the adapter returns a controlled error and AttachRequestRouter no-ops, leaving fulfillment to fail with the existing "no backend configured" path. (CodeRabbit) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): correct value coercion + track capability sub-id in request form - schemaForm: Boolean("false") was true; parse string booleans explicitly. array:num now coerces decimals ("1.5"), array:int stays integer-only. - AdminRequests: track capability_id alongside installation_id (composite <Select> value) so a multi-capability installation resolves the exact backend; reset pluginConfig when the selected plugin changes so plugin A's keys never reach plugin B's options probe/save. (CodeRabbit) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): address request-router review findings * fix(requests): handle router review edge cases * fix(web): resolve schema form build casing * fix(requests): skip unconfigured 4k fulfillment targets --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandClaude Opus 4.8
|
6e5d096fdb |
feat(plugins): chunked plugin uploads with progress
Large plugin packages previously uploaded through a single multipart POST, which can exceed proxy/body-size limits and offered no progress feedback. Add a chunked upload path alongside the existing one. Server: - New generic internal/uploads session Manager assembles chunks into a pre-sized temp file via positioned writes, with TTL expiry, idempotent chunk retries, and size validation. - Four endpoints under /admin/plugins/uploads/chunked (create/put-chunk/complete/cancel); completion reuses the shared install path and sniffs the zip magic from disk instead of reading the whole archive into memory. Web: - Reusable uploadFileInChunks helper with adaptive chunk-size backoff on 413, plus a usePluginUpload hook and shared Progress component driving the upload progress bars on both admin plugin pages. - Files at or below the chunk threshold keep using the multipart path. - api() now respects a caller-supplied Content-Type so chunks can be sent as application/octet-stream. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Silo Server Migration
|
c085b12fd1 | Initial Silo migration |