codex/bound-transcode-segments
37
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
2dedf3ff26 |
feat(metadata): user-triggered trailer refresh with weekly per-item cooldown (#531)
* feat(metadata): user-triggered trailer refresh with weekly per-item cooldown
Adds POST /api/v1/items/{id}/trailers/refresh so any viewer with access to a
movie or series can ask the server to fetch its remote trailers, bounded by a
one-week per-item cooldown enforced server-side.
The cooldown lives in a new nullable media_items.trailers_refresh_requested_at
column rather than the refresh debt queue, whose last_attempt_at evaporates on
success (MarkTargetSuccess deletes the row when the reason mask clears). The
gate is a single UPDATE that writes NOW() only when the stored timestamp is
NULL or older than the window, so concurrent viewers cannot both win it; a
losing caller reads the stored timestamp back to compute next_allowed_at.
MetadataService.RequestTrailersRefresh resolves the per-library trailer_kinds
allow-list first: a non-nil empty map means every containing library disabled
remote videos, which answers "disabled" without consuming the cooldown slot
(a nil map is allow-all and must not short-circuit). On winning the gate it
reuses startOnDemandMetadataRefresh, whose scheduled mode merges fill-empty,
so this non-admin trigger cannot clobber unlocked admin edits while found
videos still persist.
The handler checks item access before calling the service, so an unauthorized
caller can never burn an item's slot, and rejects non movie/series types since
those detail responses never carry videos. cooldown and disabled are expected
client-rendered states and answer 200; 429 is reserved for the per-user
in-memory limiter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): release trailer-refresh slot on failed refresh; resolve episode ids to 400
Three review findings on the viewer-facing trailer fetch.
The weekly per-item slot was consumed unconditionally on winning the gate,
but the refresh it started ran detached and only logged on failure — nothing
ever put the slot back. A brief TMDb outage therefore answered 202 queued,
failed 30s later, and then answered cooldown for seven days over work that
never happened. The repository gains an equality-guarded release
(trailers_refresh_requested_at = NULL only while it still equals the
timestamp this request wrote, so a later claim is never clobbered), and
TryClaimTrailersRefresh now RETURNINGs the timestamp it stored so a winner
holds the key to its own slot. startOnDemandMetadataRefresh splits into a
claim step and runOnDemandMetadataRefresh, which takes an optional failure
hook; only the trailer path passes one, so the existing callers are
unchanged. A timeout counts as failure. A refresh that succeeds but finds
nothing still keeps the slot — that semantics was chosen deliberately.
The in-process dedup claim (shared with the item-detail view's stale nudge)
silently dropped the start while the slot had already been consumed, so the
caller was told queued for a refresh that never began. It is now taken
before the durable slot: a request landing while an equivalent refresh is
already in flight reports queued without consuming the slot, which is both
honest and retryable if that refresh fails.
Real episode and season content IDs answered 404 rather than the contracted
400, because neither is a media_items row and GetByID queries media_items
alone. The handler now falls through to the same season/episode lookups
HandleTranslateOnView uses, authorizing through the parent series, so a
genuine episode ID reports unsupported-type and only unknown content 404s.
The type-check test no longer fabricates a MediaItem{Type: "episode"} row
that production never writes; it covers the types that do exist as
media_items rows, with the episode and season paths tested through the
lookups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): address PR review on the trailer refresh action
Six review findings on the viewer-facing trailer fetch, all verified against
the current code before changing anything.
Durable claim no longer rides the request context. A cancellation landing
after Postgres commits the gate UPDATE but before pgx returns would consume
the item's weekly slot with no refresh started and nothing holding the
timestamp needed to release it. The claim now runs on
context.WithoutCancel with its own deadline, mirroring the release.
The cooldown gate retries once when the follow-up read finds the slot free.
Classification spans two statements, so a concurrent failure-release can
land between them; the old code reported that as a cooldown with no
next_allowed_at while the slot was in fact free. A NULL read now retries the
claim, and the doubly-lost case answers "queued" (an equivalent refresh is
running) rather than an undateable cooldown.
A failed item_videos write now releases the slot. mergeAndPersist logs and
continues when the write fails, so the refresh reported success and the
viewer was locked out for a week having stored nothing. A context-scoped
observer, installed only by this action, surfaces that failure to the
existing release hook.
Winning the gate also records durable refresh debt, so a restart that kills
the detached goroutine leaves work the refresh worker picks up instead of a
consumed slot and no fetch. Uses a new reason bit rather than the generic
failure reason: nothing is wrong with the item, so it must not sit in the
failure band ahead of real debt or count as a failure in operator metrics.
Any library lookup failure now degrades the video-kind scope to unknown. An
item in two libraries where one resolved with trailers off and the other
could not be read reported "disabled" — a guess made on behalf of a library
that might be the one enabling trailers. A library that is genuinely gone is
still skipped.
Adds GET /api/v1/items/trailers/capability, following the existing
per-subsystem probe convention. The action route is registered conditionally,
so "this build has the feature" is not the same question as "this deployment
serves it", and a 404 on the POST is indistinguishable from a missing item.
The probe is registered unconditionally and answers refresh:false when
unwired.
Not changed: content-ID canonicalization mid-refresh stranding the cooldown
on the old row. The re-anchor path is manual-refresh only and this action
runs in scheduled mode, so only local-skeleton promotion can fire, and the
rename carries the timestamp and the debt row to the new id along with
everything else.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style(web): format overlays schema after merging main
The line came in over-length from main's card_overlays merge and the Web
CI format check runs prettier across all of src, not just changed files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): address second review round — recovery-debt lease, locked-videos preflight, shared limiter
The restart mitigation added in the first round reintroduced two of the
problems it was closing, and the reviewer was right to push again.
Lease the recovery debt behind the fast path. The row was enqueued due
now, so refresh_metadata could claim it while the detached goroutine was
still running the same refresh — RefreshScheduledTarget does not consult
the in-process claim, so both would fetch the item at once. It is now due
5 minutes out, comfortably past the 2-minute on-demand timeout, and the
goroutine settles the row on success so it fires only when the fast path
really did not finish. Settling clears just the trailers-requested bit,
keeping any real debt the item still carries.
Release the cooldown after a failed recovery. A recovery runs in a worker
that never saw the claim, so a failure left the viewer blocked for the
week having stored nothing. RefreshScheduledTarget now adopts the claim
when the debt row carries the trailers-requested reason, reading the
stored timestamp so the release stays equality-guarded, and hands the
slot back on the same failures the fast path's hook covers — including a
videos write that failed and was only logged.
Preflight the videos lock. locked_fields containing FieldVideos makes
mergeAndPersist skip the item_videos write, so the refresh "succeeded"
and kept the cooldown while never being able to save trailers. It now
answers disabled before consuming the claim; reusing that status rather
than adding one is deliberate, since clients treat an unknown status as a
dead end and "trailers cannot be fetched for this item" is what disabled
already means to a viewer.
Use the shared limiter. A private MemoryLimiter gave every instance an
independent per-user allowance on Redis deployments, and the per-item
cooldown cannot compensate — it bounds one item, while this budget bounds
how many distinct items a user can start refreshes for. The action now
takes the middleware's configured limiter, with namespaced keys, and
falls back to a private one only when rate limiting is off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
99d205676f |
fix(metadata): prevent stale cross-provider IDs (#480)
* fix(metadata): prevent stale cross-provider IDs * fix(metadata): address stale ID review findings * fix(migrations): build the stale-ID primary key concurrently ALTER TABLE ... ADD PRIMARY KEY builds the index under ACCESS EXCLUSIVE, blocking reads and writes on stale_media_ids for the whole build. Create the wider unique index with CREATE UNIQUE INDEX CONCURRENTLY and attach it with ADD CONSTRAINT ... PRIMARY KEY USING INDEX instead; all three key columns are already NOT NULL, so the attach is metadata-only. Same treatment on the rollback path, plus the repo's INVALID-remnant cleanup so a failed concurrent build is not silently accepted by IF NOT EXISTS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
383973ec22 |
feat(metadata): improve match accuracy and localized titles (#461)
* feat(metadata): improve match accuracy and localized titles * fix(metadata): address matching review findings * test(catalog): align empty alias snapshot scope --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
1664c60425 |
fix(metadata): publish artwork revisions atomically (#399)
* fix(metadata): publish artwork revisions atomically * fix(metadata): harden artwork revision cleanup * fix(metadata): address artwork revision review findings - restore image applies for all media_items types and reject unsupported target/image combinations with 400 before uploading; episodes coerce to stills and the web dialog no longer offers image tabs episodes can't use - add WHEN clauses to displacement triggers and hoist to_jsonb so bulk catalog upserts that assign unchanged artwork columns skip the trigger - make artworkkey the single variant-ladder owner: imagecache derives its widths from it and triggers store image_type instead of hardcoded variant arrays, expanded by the collector at deletion time - sweep dormant registry rows periodically so references lost through untriggered surfaces degrade to slow cleanup instead of leaking - park just-published revisions dormant, keep dormant rows dormant on re-cache, and batch the GC reference pre-check per run - heal rows re-referencing a just-deleted revision via reconciler-style resets after the deletion commits - share a per-URL image-loaded hook across DetailHero, ItemCard, SectionItemCard, GlobalSearch, and CollectionPosterCard - deduplicate Cache/CacheBytes finalization and drop unused VariantPaths plumbing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): cast reused timestamp parameter in revision upsert Postgres cannot deduce one type for $3 used both as a plain value and inside a CASE arm; the dev deploy surfaced it as SQLSTATE 42P08 on every publication. Cast both uses and cover the arm/park/track upserts with database-backed tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): address artwork revision review comments - keep a durable heal path: deletion marks deleted_at instead of removing the registry row, so a failed post-delete heal retries with backoff and broken references never park; trackers clear the marker on re-upload - never treat bare existence as an immutable-content match; backends without content verification rewrite the object - exercise revisioned cover keys in scanner/enrichment fakes, compare the tracked manifest exactly, and honor cancellation in the blocking test deleter Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0694787504 |
fix(overlays): show_status persistence + card overlay layout fixes (#335)
* fix(metadata): persist series show_status from provider metadata
Plugin-reported series status (proto status field 31) was mapped into
MetadataResult.ShowStatus but dropped by both metadataResultToItem and
itemToMetadataResult, so media_items.show_status stayed empty for every
movie and series - only the manga enrichment path ever wrote it. This
left the Show Status card overlay permanently blank for series.
- carry ShowStatus through both converters; series values normalize to
a canonical lowercase domain (returning/ended/cancelled/in_production/
upcoming) so TMDB "Returning Series"/"Canceled" and TVDB
"Continuing"/"Upcoming" converge on one spelling
- pass non-series values through verbatim so the manga status domain
("Ongoing", ...) can never be mangled by a generic refresh round-trip
- round-tripping the existing item's status also stops refreshes from
wiping a previously persisted value via show_status = EXCLUDED.show_status
- extend the web overlay formatter with continuing/upcoming/planned
TMDB/TVDB plugins need follow-up changes to actually emit the status
field; prepped separately in their repos.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(web): overlay badge layout, ordering, and wordmark rendering
Fixes from a full card-overlay audit (every issue verified by DOM
geometry measurement before/after):
- render each card edge as one flex row holding both corner stacks so
opposing badges share the width (min-w-0 + truncate) instead of
overlapping on narrow cards; long labels ellipsize instead of
wrapping over the opposite corner
- honor prefs.order via orderedOverlaysForPosition — the renderer
previously ignored the stored order entirely
- cap corners at 3 badges so maxed-out configs can't collide with the
opposite vertical corner
- lift bottom-right badges above the card menu button, which is always
visible on touch devices and occluded them
- suppress the text label when a wordmark icon (HDR10/ATMOS/AV1/HDR)
already spells it — pill/vibrant presets rendered "HDR10 HDR10" —
and widen the wordmark viewBoxes, which clipped their own text;
drop the never-used iconOnly flag the wordmark rule supersedes
- standalone resolution badge now uses prettyResolution ("4K", not
"2160P"), matching the combined badge
- manga cards skip generic overlays (their status/count chips own both
top corners) and the two chips now share a row and truncate instead
of overlapping each other
- useOverlayPrefs returns null while loading so cards no longer flash
default badges before the user's config or admin kill switch arrives
- settings rows for Resolution/HDR now say why they're hidden while
the combined badge is enabled
- add a CardOverlays test suite covering every registered overlay,
ordering, suppression, wordmarks, the corner cap, and menu clearance
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
e140bd9424 |
feat(metadata,scanner): trailers and extras for movies and series (#322)
* feat(metadata,scanner): trailers and extras for movies and series Remote provider videos (TMDB trailers/teasers/featurettes/...) are fetched through the unified match/refresh pipeline into the new item_videos table, filtered per-library via media_folders.trailer_kinds, merged across providers with site/provider dedup, and lockable via FieldVideos. The movie scanner stops discarding supplemental directories (Trailers/, Featurettes/, Behind The Scenes/, ...) and classifies them — plus Jellyfin-style filename suffixes (-trailer, -behindthescenes, ...) and series-root supplemental dirs — into the new media_extras entity backed by ordinary media_files rows (extra_id ownership, content_id/episode_id NULL so existing version/matching queries stay structurally blind to extras). Series Extras/SxxExx season-0 mapping is unchanged. Extras are playable watch targets via a GetWatchDetail fallback tier (episodes precedent), with contentid.ForLocal minting stable ids. API: ItemDetail gains additive videos/extras arrays (single + batch parity); library settings expose trailer_kinds. jellycompat now populates RemoteTrailers, LocalTrailerCount/SpecialFeatureCount, and serves real /LocalTrailers + /SpecialFeatures items playable through PlaybackInfo. Requires silo-plugin-sdk v0.9.0 (VideoRecord) before go.mod can bump; builds locally via go.work against the SDK feat/metadata-videos branch. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): trailers and extras sections, library trailer-kinds setting TrailersSection (YouTube thumbnails + youtube-nocookie modal) and ExtrasSection (plays extras through the standard watch controller) on movie and series detail pages; admin library form gains a trailer-kinds allow-list synced with the server default (all provider kinds), now also honored on library create. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): scan extra_id in scanMediaFiles; review cleanups scanMediaFiles (the plural row scanner behind GetByContentID/GetByFolder/ GetByExtraID and 20+ other queries) was missing the scan destination for the new extra_id column, which would have failed every media-file read at runtime with a column/destination count mismatch. Also: extend the batch equivalence test to seed item_videos/media_extras so the new videos/extras prefetch wiring is actually proven; drop the one-off pgxRows interface for the repo-wide pgx.Rows convention; reuse formatClock instead of a third duration formatter in ExtrasSection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump silo-plugin-sdk to v0.9.0 for VideoRecord Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(matching): exclude extras files from match queues and bulk content linking Dev verification caught extras media_files rows (content_id NULL by design) being swept into the movie/series match queues and the root-claim bulk relink: a '-featurette' suffix extra was matched onto its parent as a version, and a Trailers/ file minted a spurious local skeleton item that shadowed the extra's watch target. Add 'extra_id IS NULL' to the queue eligibility conditions, root/group claim relinks, observed-root content assignment, and the admin unmatched-files listing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): authorize local extras files through their parent item Dev verification: playback/start (and the shared MediaFileAuthorizer used by markers/subtitles/ebook reader) resolved file ownership only via episode_id/content_id, so extras files (extra_id only) 404ed. Add an ExtraLookup tier that resolves media_extras and gates on the parent item's access, mirroring the episode->series pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): resolve local extras through GetItemDetail for compat playback jellycompat PlaybackInfo (and any per-item consumer resolving arbitrary content ids) goes through GetItemDetail, which lacked the extras tier that GetWatchDetail has — so Jellyfin clients got zero MediaSources for extras. Add buildExtraItemDetail (minimal detail + ordinary playback surface, parent-gated access) as the fourth resolution tier, and map the extra type to Jellyfin's Video kind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): allow youtube-nocookie embeds in CSP; trailer modal a11y The frontend CSP's frame-src blocked the trailer modal's youtube-nocookie.com iframe (found on dev verification). Also add the missing sr-only DialogDescription and drop the redundant allowFullScreen attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address PR review findings for trailers/extras - Extras watch/item detail no longer stamp SeriesID/SeriesTitle for movie-owned extras (players key episodic post-roll flows off series_id); series-owned extras keep them (Codex). - processExtraFiles resolves the parent and upserts media_extras before the unchanged fast-path, and the fast-path now also compares mtime, so rematched parents / reclassified kinds / same-size replacements converge (Codex + CodeRabbit). - media_files upsert clears content/episode linkage atomically when extra_id is set (ownership mutual exclusion in one statement); the now-redundant MarkFileAsExtra helper is removed (CodeRabbit). - ScanFile's extras branch runs syncPresentLibraryState + reconcileLibraryMemberships so converting a primary file to an extra cleans stale library membership immediately (CodeRabbit). - media_extras migration adds the media_files FK as NOT VALID + VALIDATE to avoid a full-scan exclusive lock on large tables (CodeRabbit). - trailer_kinds input is trimmed/lowercased/deduped and unknown values are dropped instead of silently widening the allow-list to 'other' (CodeRabbit). - Extras authorization branches match the episode branch's posture: unconfigured lookup is a config error, nil extra is a 404 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0fb5afe479 |
feat(matching): split wrongly merged versions with watch-state reattribution; anchor group keys on provider tags (#319)
* feat(matching): split wrongly merged versions, reattribute watch state, anchor group keys on provider tags
Wrong merges (two titles normalizing to the same title+year key) stacked
different films as fake "versions" of one item with no in-app repair, and
explicit {tmdb-…}/[imdb-…] folder tags could not prevent it because the
content-group key ignored provider IDs entirely. Merges also silently
orphaned all per-user watch state.
- Anchor group keys on structured provider tags: same tag always groups,
different tags can never merge; untagged files keep title+year keys.
- media_identity_overrides: path-scoped (root/file) forced identities applied
during group inference, so admin splits survive rescans.
- internal/catalog/reattribute: shared user-state mover — exact moves for
file-linked rows, evidence-based user_watch_history classification via the
playback session log, newest-wins progress conflicts; wired into
rebindItemToExistingItem to stop merge orphaning (with S/E episode mapping).
- POST /admin/items/{id}/split (dry-run = full transaction + rollback, so
previews are exact), POST /admin/items/{id}/merge, GET /admin/items/{id}/files.
- Web admin: Split Versions dialog (files by folder → candidate search →
preview → split), Resolve link from ambiguous-roots diagnostics.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(reattribute): classify history before moving session log; cover managed downloads and series-scoped preferences
Review findings on #319, all reproduced against a migrated scratch database:
- moveFileSubset re-pointed playback_history_admin before the history
evidence query ran, erasing exactly the evidence proving a profile's plays
were all on moved files — their history stayed behind as ambiguous.
History classification now runs first; the pre-fix code demonstrably fails
TestRun_HistoryEvidenceClassification.
- Managed offline downloads (downloads.content_id/episode_id) were not
remapped on split or merge, stranding rows on the old id. Now moved per
file on splits and swept per id pair on merges/episode re-anchoring.
- Series merges left user_audio_preferences, user_subtitle_preferences,
user_series_playback_preferences (series_id-keyed) and the denormalized
user_home_item_dismissals.series_id behind. All four now move, mirroring
the provider-merge remap.
All five reattribute DB tests now verified green against PostgreSQL, with
new coverage for managed downloads, subtitle preferences, and dismissal
series ids.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
430224a1b9 |
perf: cut home-screen, Continue Watching, and Latest latency; cache shared home rails (#292)
* perf(jellycompat,sections): bound resume scan, batch leaf detail progress, widen section concurrency Three low-risk fixes from the section-fetch performance investigation (docs/superpowers/plans/2026-07-03-section-fetch-performance.md): - jellycompat: bound loadProgressPage at resumeScanMaxRows=300 so a single request never pages through more than that many in-progress rows. The cap is unconditional: it also covers the sparse-visible-set case (a heavy watcher whose recent rows are mostly dismissed/superseded, or a Series/Season-only request that matches no leaf in-progress row), where the page never fills and the loop would otherwise scan the entire history — previously an O(history) scan reaching tens of seconds. In the common case the loop exits far earlier, so the cap only bounds the pathological worst case; 300 leaves ample headroom to fill a ~20-item Continue Watching page. Beyond the cap the reported total is a clamped lower bound. Covered by TestLoadProgressPage_BoundsScanForSparseVisibleSet. - jellycompat: batch the leaf-item (movie/episode) progress lookup in GetItemDetailsByIDs via ListProgressWithCompletedHistory instead of a per-item GetProgressWithCompletedHistory (~100 sequential queries for a 50-item detail page). Series keep the per-item episode-rollup path (they own no progress row). Output is unchanged; a batch-lookup failure is now logged rather than silently dropping played state for the whole page. - sections: raise fetchAllMaxConcurrency 4 -> 6 to cut FetchAll wave count for large home layouts, staying within the default 20-conn pool. Part of the home/continue-watching latency work. AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat): keep Latest browse on the cross-library fast path under isPlayed /Items/Latest with isPlayed=false is the highest-frequency compat browse (~10.8k calls/day). The played overlay can't be pushed into SQL, so browse over-fetches and filters locally. The cross-library recently_added fast path (BrowseRecentlyAddedAcrossLibraries: one ~1ms index walk per library) was gated on Offset==0, so a heavy watcher who had already seen the newest items needed a 2nd chunk and fell through to BrowsePage — a whole-catalog MIN(first_seen_at) + GROUP BY HashAggregate over ~147k movies measured at ~755ms per call (0.8-1.6s observed end-to-end). Fetch the entire over-fetch budget (maxScannedRows) in a single merged fast-path walk instead of paging into BrowsePage, so the loop fills from one call. The clamp caveat (MaxLimit=1000 leaves a fall-through only for requestedLimit>200, off the Latest hot path) is documented inline. Part of the home/browse latency work. AI-use disclosure: implemented with AI (Claude) assistance. * fix(jellycompat): scope resume scan cap to resume path and bound the fast-path loop Addresses PR #292 review feedback: - Codex (P2): the resumeScanMaxRows cap was applied unconditionally in the general loop, which also paginates the completed (watched-items) list. Gate it on resumeFiltered so the completed path keeps exact TotalRecordCount and deep StartIndex pagination. Covered by TestLoadProgressPage_CompletedScanNotCapped. - CodeRabbit (Critical): the earlier raw-offset fast-path loop — the default Continue Watching shape and the sections-fallback route — had the same unbounded-scan bug and was not covered by the cap (the existing test forces EnableTotalRecordCount=true, routing around it). Bound it with the same resumeScanMaxRows guard. Covered by TestLoadProgressPage_BoundsFastPathScanForSparseVisibleSet. - CodeRabbit (Minor): tag the doc's fenced example blocks as text to satisfy markdownlint MD040. AI-use disclosure: implemented with AI (Claude) assistance. * perf(sections): cache shared user-agnostic home rails per access scope Home-screen rails that are identical for everyone who can see the same libraries (recently added, recently released, genre, trending on server, most watched, new to library, critically acclaimed, award winners, format showcase, seasonal, mood, trending discover, admin-curated lists, and library collections) were rebuilt from Postgres once per request, per user. Only the overlay on top of each row (watched flags, play position, presigned poster URLs) is actually per-user. Insert a process-global resolved-list cache at the FetchOne choke point in internal/sections. Each cacheable row is built once per access scope, held with a 15m TTL, and refreshed in the background 3m before expiry; singleflight collapses cold-miss stampedes into a single build. The per-user overlay still runs fresh in buildSectionsResponse, so no profile state is ever shared. Random and per-user rows (continue watching, next up, recommendations, hidden gems, forgotten favorites, activity feed, user collections) bypass the cache. The access-scope key captures every access boundary the fetch path enforces -- section identity (type + id + config hash) + item limit + accessible and disabled libraries + max content rating + excluded media types + name prefix + allowed-content-id allowlist -- and nothing per-user, so entries are safely shared. Empty membership is never cached (avoids freezing a transiently empty rail); background refreshes are bounded by a timeout. Scale (analytical, derived from the cache behavior -- not a measured latency): for the user-agnostic rows, Postgres section-query volume collapses from O(rows x concurrent requests) to O(rows x distinct access scopes) per 15m refresh window, because most users share a handful of access scopes. Illustrative -- 40 cacheable rows on a home screen, 1000 concurrent users falling into ~5 distinct access scopes: - before: ~40 x 1000 = ~40,000 section queries per wave of home loads - after: ~40 x 5 = ~200 builds per 15m window (plus one background refresh per row per scope), i.e. a warm home load runs zero section queries for these rows. That is a ~99% reduction in shared section-query load at that concurrency; the win grows with concurrency and shrinks as access-scope diversity rises. Design/plan doc added under docs/superpowers/plans/. * perf(jellycompat): serve per-library Latest via the cached recently-added section A jellyfin-compat per-library /Items/Latest rail is the same user-agnostic list as the native "recently added" library rail -- both order by mil.first_seen_at DESC. It was rebuilt on every request through directContentService.BrowseItems, missing the resolved-list cache entirely. Route per-library Latest for movies and series libraries through the native section fetch instead, so it reuses the shared cache. HandleLatest resolves the library's type once, and for a movies/series library builds a synthetic SectionRecentlyAdded with the same type + config + limit + access scope the native rail uses and calls FetchOne; the per-user overlay (favorites, progress, episode targets, presign) is extracted into buildLatestItemDTOs and shared by both the native and BrowseItems paths, so no overlay logic is duplicated. Cached *models.MediaItem values are read-only -- LocalizeItemModels deep-copies before any presign mutation. To let the two surfaces share one entry, resolvedListCacheKey no longer includes the arbitrary section ID: every cacheable section type derives its membership from type + config + limit + scope, never from its own ID (audited all 14 cacheable types plus the library-collection path; the sole s.ID read lives in the non-cacheable user-collection branch). A native recently-added rail and the compat Latest for the same library + scope now collapse to ONE cache entry, built once and reused. Access-scope isolation is unchanged -- the removed ID never carried access information, and every access boundary (libraries, rating cap, excluded types, content allow-list, name prefix) still keys the entry. Guardrails: the native path is restricted to movies and series libraries; every other library type (ebook, music, manga, mixed) is ignored and keeps its exact BrowseItems behavior -- important because an unfiltered recently-added fetch would otherwise surface non-video items to Jellyfin clients that only expect video. Deeper pages, played-filter and backdrop-required requests, a client asking for a type other than the library's own, and any FetchOne error also fall back to BrowseItems. Chosen over an alternative that gave the synthetic section a deterministic ID (which kept two separate cache entries): both returned identical data with similar complexity, so the shared-entry design won. * fix(sections,jellycompat): post-review fixes for the shared-list cache and Latest path Consolidates fixes from the branch's adversarial review and PR #292 review comments into one commit: - Latest fast path: fall back to BrowseItems when a request carries a genre, name-prefix, or person filter (the synthetic recently-added section cannot express these, so serving it unfiltered would return a wrong, broader set). Eligibility is decided by latestFastPathEligible and covered by a test. - Clamp the /Items/Latest page size to compatBrowseMaxLimit before building the section, matching the BrowseItems fallback, so a large client Limit can't drive an oversized recently-added fetch or explode the shared cache key with unbounded ItemLimit values. - Evict expired entries from the process-global resolvedListCache: resolvedListSet sweeps expired keys at most once per minute, bounding the map to scopes seen within one TTL window. Covered by TestResolvedListCacheEvictsExpiredEntries. - Log a short digest of the cache key (resolvedListLogKey) instead of the raw key in the background-refresh panic/error paths, since the key embeds user-controlled access-scope fields such as NamePrefix. Skipped review comments (verified already fixed or stale against current code): the resume fast-path scan bound and watched-items cap (04d2e795) and the docs fence-language tags (already addressed). Build, vet, and go test -race pass for internal/sections and internal/jellycompat. * perf(plugins): cache plugin installations in-memory, invalidated on lifecycle change ## Problem Every poster/image on a warm home rail re-read plugin_installations from Postgres to answer "is this plugin enabled?" and to acquire the plugin client (Source A: metadata chain buildProviders enabled-check; Source B: ensureClient -> loadInstallation). Plugin-resolved image URLs are never URL-cached, so the plugin source and the DB read behind it fired again on every identical warm request; 100% of images in the target library are plugin-backed. ## Solution - Guarded in-memory installation cache (map[int]*Installation + RWMutex) in plugins.Service. loadInstallation reads through it; the requireEnabled gate stays after the cache read so ErrInstallationDisabled semantics are unchanged. invalidateInstallationCache clears it and is self-registered as a lifecycle hook, so Service.OnLifecycleChange wipes it on install/enable/disable/update/ uninstall. - A generation counter closes an invalidate-vs-repopulate race: captured before installations.GetByID and re-checked under the write lock, so a row fetched before a lifecycle mutation is never written into a freshly cleared cache (would otherwise resurrect a just-disabled plugin). - Route the metadata chain enabled-check through the same cache via a structural InstallationEnabledChecker interface (nil-safe: falls back to the pool query when no checker is injected), wired in cmd/silo/main.go. ## Post-review fix (auto-update reliability blocker) AutoUpdateService mutated installations (new InstallPath, old dir deleted) on the default auto update policy without firing OnLifecycleChange, leaving the cache stale and breaking plugins with "stored plugin manifest mismatch" until restart. It now takes an onChange callback wired to Service.OnLifecycleChange and fires it once per Check run that mutated a row. ## Verification go build/vet, go test ./internal/plugins/... ./internal/metadata/... (-race). Tests: cache hit/invalidation, racing-invalidation guard, IsInstallationEnabled, auto-update fires onChange. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(jellycompat): batch per-item presign, and enrich series on the cached Latest path ## Problem List rails presigned each item's poster/backdrop/logo/still image individually (~160 singular resolver calls for a 40-item page where 4 batched calls suffice), and ItemsHandler carried a near-verbatim duplicate of the batch presigner. ## Solution (batching) Promote the batch presigner to a shared package-level presignCompatListItems (presign_list.go) with a generic collectImagePaths[T]; convert the per-item loops (cached home/Latest rail, favorites, batch loaders, userdata favorites) to one batched PresignImageURLsWithExpiry per image type per page; batch the season/episode collections; delete the three duplicate presign helpers. URL output is unchanged (verified byte-for-byte). ## Post-review fix (series Latest data-parity regression) The native cached Latest fast path built items via compatListItemsFromModels + buildLatestItemDTOs and never ran the series watch-state rollup, so a series library's Latest lost Played / UnplayedItemCount and page 1 disagreed with the BrowseItems fallback. enrichSeriesUserData is promoted to the ContentService interface and called on the native path (reused, not duplicated). ## Verification go build/vet, go test ./internal/jellycompat/... ./internal/catalog/... Tests: bounded presign invocation counts + per-item URL mapping; series rollup populated on the native Latest path. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(sections): gate personalized rails out of the shared cache; widen refresh lead ## Problem 1. The shared home-rail cache whitelisted custom_filter/genre sections by TYPE alone, but those route through fetchFiltered -> ParseQueryDefinition and can carry personalized (per-profile) rules/sorts (watched, favorited, in_watchlist, in_progress, last_watched; sorts progress/date_viewed/plays). Their membership is per-profile yet the cache key excludes userID/profileID, so a personalized rail built for one profile was served to others in the same access scope for up to 15m -- a cross-profile watchlist/watch-state leak. 2. The background-refresh lead was tuned so steady traffic is served a warm entry from a longer soft window. ## Solution - Add QueryDefinition.IsPersonalized() (reusing the existing QueryFieldRequiresProfile/QuerySortRequiresProfile helpers). isCacheableSectionType now parses the section QueryDefinition and refuses to cache custom_filter/genre when personalized; non-personalized definitions stay cacheable. Seasonal/mood/trending build their definitions server-side and stay unconditionally cacheable. - resolvedListRefreshLead 3m -> 10m (soft threshold builtAt+5min instead of builtAt+12min). ## Verification go build/vet, go test ./internal/sections/... ./internal/catalog/... (-race). Test: personalized custom_filter/genre not cacheable; non-personalized are. ## AI-use disclosure Implemented with AI assistance (Claude). * fix(sections,metadata): post-review fixes for shared cache and plugin chain staleness Addresses three review findings on PR #292: - sections: canonicalize section config JSON before hashing so configs differing only in whitespace/field order share a cache entry (native + jellycompat rail sharing). Added TestHashSectionConfigCanonicalizes. - metadata: invalidate the resolved-chain cache on plugin lifecycle changes; the installation-enabled check already reads the invalidated plugin cache, but resolveChainCached could serve a stale provider chain for up to chainCacheTTL after a provider's availability changed. - jellycompat: move ctx to the first parameter of presignCompatListItems for consistency with the other presign helpers. Skipped the episode-image presign batching nitpick: the resolver already dedupes+singleflights, so it is a Minor perf-only item not worth the two-pass refactor risk in this pass. * fix(sections,jellycompat): harden shared rail cache and Latest fast path per review Addresses the eight findings from the deep review of this PR: - Detach the blocking cold-miss rebuild from the singleflight leader's request context (context.WithoutCancel + the shared 30s build timeout) so one client disconnect no longer fails every collapsed waiter and leaves the entry uncached. - Stop client-controlled values minting unbounded cache entries: the compat Latest fast path now always fetches a fixed 100-row budget and slices to the requested limit (one entry per scope+library instead of one per Limit value), and an unrecognized MaxOfficialRating string disqualifies the fast path instead of entering the global cache key. - Add release_date to the sections item projection/scan so movies served via the Latest fast path keep PremiereDate (Jellyfin default-set field) in parity with the BrowseItems fallback. - Fall back to per-item progress lookups when the batched leaf progress query fails, restoring one-item-at-a-time degradation instead of blanking played state for the whole page. - Derive cache eligibility from a single source of truth: fetchSection and isCacheableSectionType now share the userAgnosticSectionFetcher table, whose no-userID/profileID signature makes a fetcher drop out of the cacheable set at compile time if it ever gains per-profile inputs. - Decide Latest fast-path eligibility off the actual browse params the fallback would receive, so any filter later added to buildBrowseParams automatically disqualifies the cached path; share one compatDefaultBrowseLimit constant between both paths. - Extract AccessFilter.WriteAccessScopeCacheKey as the shared, security- critical serializer for all access-scoped caches (resolved-list, editorial candidates, audiobook groups); the editorial key now captures ExcludedMediaTypes, which its loaders already applied in SQL. - Strip leaked agent-transcript markup from the section-fetch plan doc. go build ./..., go vet, gofmt clean; go test -race on internal/sections, internal/catalog, internal/jellycompat passes (TestBeginWebOperation* failures are the known pre-existing flakes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c0f209a936 |
feat(catalog): Latest Episodes sort — order series by newest episode file (#283)
* feat(catalog): Latest Episodes sort — order series by newest episode file Adds a latest_episode_added sort so users can see which shows received new episodes. Today's recently-added surfaces reflect when the SERIES was first added: linking a new episode file never bumps the series' media_item_libraries.first_seen_at (ON CONFLICT DO NOTHING), so a long-running show with a fresh episode sorts as stale (#202). - New denorm media_items.latest_episode_added_at (migration + backfill + partial series index), mirroring the last_air_date_at precedent. Source of truth is episode_libraries.first_seen_at; the three insert paths (UpdateEpisodeLink, BulkLinkEpisodesBySeries, scanner folder restore) bump the parent series atomically in the same statement, monotonically via GREATEST, and only for genuinely new links. - Sort registered in both frameworks: querySortDefs (sections + smart collections + /v1/catalog pick it up automatically via QuerySortFieldSet) and the browse buildOrderByPlan path. - Jellyfin compat: SortBy=DateLastContentAdded now maps to the new sort instead of silently collapsing to series creation date — Jellyfin clients already send this for the TV "Latest" shelf, so they get the correct behavior with no client changes. DatePlayed keeps its old created_at mapping instead of piggybacking. - Web sort picker gains "Latest Episode Added" (series scope). Additive-only per v1 API rules: new sort value, no field/status changes. Part of #202 Fixes #202 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): include latest_episode_added in the api QuerySort field union The picker-side QuerySortField gained the value but the api-layer QuerySort union did not, breaking the production tsc build. Part of #202 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): recompute latest_episode_added_at when episode memberships are removed The denorm was only ever bumped upward (GREATEST) at insert time, but UpdateEpisodeLink also deletes the old episode's library membership on re-link, and reconciliation/path-prefix clears remove memberships too — leaving a stale timestamp that kept the series sorting as recently updated. All removal paths now run in a transaction and finish with a shared full MAX() recompute (catalog.RecomputeSeriesLatestEpisodeAdded) that also resets to NULL when no memberships remain, mirroring the last_air_date_at maintenance pattern. Sequential statements are load-bearing here: data-modifying CTEs are invisible to reads in the same statement, which also silently no-op'd the old path-prefix membership delete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(jellycompat): keep DateLastContentAdded scoped to series-only requests mapSortBy runs for every /Items browse, so the latest_episode_added mapping leaked into movie and untyped requests where the column is always NULL, destroying the previous created_at ordering. The sort now falls back to created_at unless IncludeItemTypes is exactly Series. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
c75c519e3d |
fix(metadata): stop manual rematch from resurrecting recorded stale IDs (#276)
* fix(metadata): stop manual rematch from resurrecting recorded stale IDs The Apply Match flow (ModeIdentify) re-injected durable provider IDs into the identify request without checking stale_media_ids, so a known-dead tmdb ID rode along, 404ed again during the Phase-2 fetch, and was re-recorded with a fresh last_seen_at — the item never left the Stale External IDs list and jumped back to the top after every rematch. Filter recorded-stale IDs out of the injected durable set in prepareProcessRequest. Caller-supplied IDs are untouched, so an admin deliberately re-selecting a previously-stale ID still retries it (which is also why the ModeIdentify suppression guard in processInternal stays). Fixes #268 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): normalize provider-id keys so stale-ID suppression can't be bypassed by casing Review on PR #276 flagged that suppressRecordedStaleProviderIDs lowercases and trims the stored stale row's provider before looking it up in the incoming map, while the map keys are used verbatim, and that HandleApplyItemMatch passes req.ProviderIDs from the JSON body straight into metadata.Process without the normalization the search endpoint applies. A caller-supplied key like "TMDB" or " tmdb " therefore defeated the suppression. The same normalization gap was previously flagged on PR #182. Fix both layers: - HandleApplyItemMatch now runs req.ProviderIDs through normalizeMatchProviderIDs (same semantics as the search endpoint) and returns 400 when no non-blank entries remain, mirroring the existing empty-map rejection. - suppressRecordedStaleProviderIDs now indexes the incoming map by normalized key and deletes the matching original keys, so suppression is robust regardless of caller casing or padding. Adds regression tests at both layers: a metadata-level case where the durable row arrives as "TMDB " while the stale row records "tmdb", and handler-level cases asserting apply normalizes keys/values and rejects all-blank provider-id maps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
f983c54e87 |
fix(metadata): re-fetch titles/overviews when a library's metadata language changes (#278)
* fix(metadata): re-fetch titles/overviews when a library's metadata language changes An item's default_metadata_language was stamped once at first match and never updated: the canonical-language pin in mergeAndPersist routed any refresh in a different language into the localization tables, the upserts' COALESCE kept the old stamp forever, quick-mode library refresh skipped complete items entirely, and changing the language in HandleUpdateLibrary triggered nothing. Items stayed in the old language no matter how often the admin refreshed (#211). Four coupled changes: - ProcessRequest.AdoptLanguage: folder-scoped manual refreshes adopt the library's language as the item's new canonical language when it differs from the stamp, rewriting the base row instead of localizing to the side. Only ModeManualRefresh adopts — scheduled refreshes merge fill-empty and would restamp without rewriting the text. - Upsert language pins inverted (media_items, seasons, episodes): prefer the incoming non-empty default_metadata_language over the existing stamp. All existing callers send the unchanged stamp or empty, so behavior is unchanged outside adoption; the restamp is atomic with the canonical write. - Quick-mode refresh lister now includes complete items whose stamp differs from the library's configured language. - HandleUpdateLibrary enqueues a quick library metadata refresh when the metadata language changes, mirroring the paths-change rescan trigger. Fixes #211 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): gate language adoption on field locks and library agreement Two hardening fixes for AdoptLanguage from review of #278: - Locked fields defeat the restamp: title/overview merges honor per-field locks, but the restamp was unconditional. An item with both language- bearing fields locked kept its old-language text yet got stamped the new language, so the quick-refresh mismatch predicate never flagged it again. mergeAndPersist now skips adoption when both name and overview are locked, falling back to the non-adopting behavior: the stamp stays put, isCanonicalWrite goes false, and the fetch routes to the localization tables exactly like a non-adopting refresh in that language does today. One locked field still adopts — the other is actually rewritten. - Multi-library flip-flop: an item in libraries with different metadata languages had its canonical base row rewritten to whichever library refreshed last, oscillating forever. Process now requires every library containing the item (media_item_libraries) to resolve to the adoption target before setting AdoptLanguage, via the existing GetDistinctMetadataLanguagesForItem (which applies the same empty→en default as resolveFolderLanguage). Disagreement or a lookup failure keeps the current stamp — stable beats flip-flopping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
Quick
|
6df388b25b | fix(search): harden search index review paths | ||
|
Quick
|
6ca427096b | Add catalog search provider support | ||
|
QuickandGitHub
|
de931f784a |
Fix ebook match dialog to support generic provider IDs (#182)
* fix(metadata): match non-video items with content-aware providers * fix(web): guard invalid match year parsing |
||
|
|
14ffc91dfb |
[codex] Expand provider image cache queue (#176)
* feat(metadata): expand provider image cache queue * fix(metadata): harden provider image cache queue Addresses bug-review feedback from Codex/CodeRabbit on the metadata image cache pipeline. All findings validated against the code before fixing; false positives (rows/connection deadlock, PhotoSourcePath merge coupling) were confirmed non-issues and left unchanged. - Honor metadata.cache_images for the background processor. The cache_metadata_images task was registered whenever S3 was configured, so merely enabling object storage downloaded the entire provider-artwork catalog even with caching disabled. Add ImageCacheProcessor.SetEnabled, gate RunOnce/RunUntilIdle on it, and wire it (with hot reload) from cfg.Metadata.CacheImages in main.go. - Guard terminal job updates with lease ownership. EnqueueBatch can repurpose a running row with a new source; MarkSucceeded/MarkFailed keyed on id alone let a stale worker finalize the replacement job and drop the new artwork. Thread locked_by through and add status='running' AND locked_by=$n guards. - Avoid uploading stale jobs onto the live artwork key. Verify the target still references the job's source (CurrentTargetSourcePath) before CacheImage, so a job whose source an admin/refresh already replaced cannot overwrite the deterministic storage object. - COALESCE nullable external IDs in EnqueueExistingProviderArtwork. A NULL tmdb_id/tvdb_id/imdb_id on any candidate failed the scan and aborted the whole cache run; matches the existing item_repo pattern. - Stop re-downloading the catalog every 30 days. Discovery now skips targets whose *_path is already a cached relative path, making the cached row the durable dedup marker instead of the prunable job row. - Decouple catalog sweeps from queue draining. RunOnce no longer runs discovery per batch; RunUntilIdle sweeps only when the queue drains and throttles full sweeps to every 15m, so idle installs stop full-scanning every entity table each minute. - Requeue claimed-but-unstarted jobs on cancellation. Acquire the semaphore before spawning workers and RequeueClaimed any jobs not yet started, instead of leaving them locked until the 15m lease expires. - Skip the backoff sleep after the final upload attempt in putObjectWithRetry (saves ~1.5s on permanent failures). - Add the s3/file/local/upload/generated exclusion to the seasons and episodes backfill in migration 20260617184537 for consistency with the later migration (the bad backfill was inert downstream, but the asymmetry is removed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
9cd00a877b |
fix(metadata): cap un-completable episode refresh debt as terminal (#168)
Episodes a provider genuinely has no data for stay actionable forever and, via syncRefreshDebtForItem, pin the whole series item at RefreshDebtReasonEpisodeIncomplete -> priority 300 (the top band). Those debt rows never drain and always cut the queue ahead of legitimately-due content, every backoff interval, indefinitely. Classify episode-incomplete debt as terminal once its persistent attempt_count reaches a small cap (3, ~5 days under the existing stepped backoff): demote it off the priority-300 band (to a still-fixable reason's band if one is present, else a terminal floor) and re-check it rarely (90d) instead of every 30d. The signal is purely attempt_count on the existing row -- no schema change. Demote rather than delete: deleting would let the next series pass recreate the row at attempt_count 0 and restart the cycle. Giving up fast is safe: the library "refresh incomplete" and per-item refresh endpoints re-fetch on demand and bypass the backoff, and media_items.episode_metadata_incomplete is left untouched so they still find these items. Logs the terminal transition (no silent demotion). AI-use disclosure: implemented with AI assistance (Claude Code). |
||
|
|
13c5e0ba2f |
feat(catalog): deterministic cross-server content_id (#155)
* feat(catalog): deterministic cross-server content_id Replace per-server Sonyflake content_id with a structured natural key derived from provider IDs (movie:tmdb:…, series:tvdb:…, episode:…, local:… fallback), so two servers holding the same title share one anchor for artwork, watch history, progress, favorites and ratings. - internal/contentid: derivation core, SeriesIDFromContentID transform, frozen precedence, SchemeVersion=1, embedded-series-anchor invariant. - internal/metadata/service.go: deterministic id at every mint site. - internal/catalog/history_source.go: resolve show via string transform for anchored episode ids; skip the episodes_pkey probe. - migrations/sql/20260612130000: collision-safe value remap across the 65-column reference graph + COLLATE "C", FK/trigger handling, audit map, working down. Benchmarked against an exact-cardinality copy of cprod-postgres (1.93M episodes, 775k history rows): 2.57x faster history page, 1.7x throughput at 100 concurrent users, 2.7x cheaper per content_id probe. * feat(catalog): re-ID untagged items to deterministic content_id at first match Untagged libraries get a path-derived local: content_id at scan time and only learn their provider IDs later, when the match worker confirms a result. Previously that id was never folded back in, so untagged-then-matched items kept a per-server local: placeholder forever and never converged across servers (re-ID was deferred to a migration rerun). mergeAndPersist now promotes a local: skeleton to its deterministic provider-anchored id at the moment of first confirmed match, via a single new gate (canonicalizeLocalContentID): - target id already taken -> merge onto it (existing rebind machinery) - target id free -> rename in place The rename is a single SQL function (silo_rename_content_id); FK children follow via ON UPDATE CASCADE added to the content_id family, so a fresh skeleton moves a handful of rows rather than the full-table remap the bulk migration does. The guard is one IsLocal prefix check, so tagged content and all refreshes pay nothing, and the move is self-healing under retry. Verified: gofmt/vet/build clean; migrate-validate passes; migration applies on the real schema (up/down/up), FKs gain ON UPDATE CASCADE while keeping ON DELETE; functional test confirms series PK move + series_id cascade + provider-id sweep, and movie rename. Follow-ups (noted in docs): recomposeSeriesChildIDs for a series that accumulated episodes before matching; a lockstep test for the soft-ref list. * fix(catalog): harden content_id parsing and merge per review Address review feedback on the deterministic content_id work: - history_source.go: gate the anchored-episode display-id transform on the full five-part episode shape (split_part parts 2-5 non-empty), not just the 'episode:' prefix, so a malformed id can't transform to 'series:broken:' and vanish at the media_items join. Shared anchoredEpisodePredicate drives both the null-poisoned join key and the series-recovery expression. - contentid.go: unexport the provider-precedence slices so no package can mutate the frozen SchemeVersion ordering at runtime. - contentid.go: add parseAnchored to validate the exact per-kind arity and numeric season/episode suffixes; SeriesIDFromContentID and IsProviderAnchored now fail closed on truncated/malformed ids (e.g. "episode:tvdb:296762"). - canonicalize.go: distinguish catalog.ErrItemNotFound from transient lookup errors (a real error no longer masquerades as "target free"), and allow a matched local source to be consolidated onto the canonical row instead of orphaning a duplicate. * refactor(contentid): URL-safe "-" separator in content_id Use "-" instead of ":" to join content_id components (movie-tmdb-228064, episode-tvdb-296762-1-5, local-<hex>). "-" is an RFC 3986 unreserved character, so a content_id is URL-safe verbatim: encodeURIComponent is a no-op and the id is its own tidy path segment (/item/series-tvdb-296762) with no %3A escaping. The stored value equals the URL value, so there is no encode/decode boundary and an operator can grep the id straight out of a URL or log. Every component is [a-z0-9]+ (or "tt"+digits), so "-" is unambiguous. Pre-release format finalization: this branch is unmerged, so no deployed data carries ":" ids — the migration mints the "-" form fresh and no re-migration is needed. Still SchemeVersion 1. - contentid.go: single `sep` constant drives construction and parsing so the two can never drift; all constructors/parsers and doc examples updated. - history_source.go: split_part transform and the anchored-episode predicate use '-'; kept in lockstep with the package via a code comment. - 20260612130000_deterministic_content_id.sql: derivation and season/episode composition emit '-'; LIKE filters match 'series-%'. - docs/architecture/deterministic-content-id.md: format spec + rationale for the separator choice; this is the design doc the change is derived from. Client-side: the web frontend treats content_id as an opaque string (no splitting/regex), so no client changes are required; existing encodeURIComponent call sites simply stop emitting %3A. * docs(contentid): show why hash/bigint rejected in probe-cost table Add Cross-server deterministic / Zero-join show transform / Human-readable columns to the index-probe-cost comparison so the trade-off is legible at a glance: the 128-bit hash and bigint surrogate are faster but each give up a load-bearing property, and the structured key is the only all-checkmark row. * docs(contentid): order probe-cost table to end on the structured key * docs(contentid): label fenced blocks and drop stray EOF tags Per CodeRabbit review: add 'text' language to three fenced code blocks (MD040) and remove accidental </content></invoke> artifacts at EOF. * fix(catalog): remap array-valued content_id soft references in deterministic id migration The value-remap migration (20260612130000) enumerates the reference graph by FK plus a scalar name+type sweep (text/varchar/bpchar). That misses trending_discover_snapshots.content_ids: it is text[] (excluded by the type filter), named content_ids not content_id (excluded by the name list), and cannot carry an FK — so the bulk remap left those arrays holding stale Sonyflake ids that resolve to nothing until the snapshot regenerates. A counterexample to the migration's "self-protecting, cannot orphan" invariant. Remap the array element-wise in both directions (Up old->new, Down new->old), preserving order and leaving collision/unmatched elements untouched; a WHERE EXISTS guard skips empty/unaffected arrays so array_agg never collapses the NOT NULL column to NULL. Mirror the gap in silo_rename_content_id (20260614120000) with array_replace for the single-value runtime rename so the two stay in lockstep. Verified on PG18: mixed/collision/empty arrays remap correctly and round-trip clean; runtime array_replace preserves order. Surfaced reviewing #155. The jellycompat restart-decode regression and the atomicity-wording nit are posted as review comments, not addressed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(jellycompat): pack content_id into compat UUID reversibly so item ids survive restarts Addresses the restart-decode regression raised in review of #155. With content_id now a structured string instead of a numeric Sonyflake, EncodeStringID sent every item/season id down the one-way SHA1 path, making decode depend on an in-memory reverse map. That map is cold after a process restart (the codec is a process-lifetime singleton), so a client presenting a previously-issued item UUID — resume-from-home, deep link, detail page, image, userdata — got "unknown compat id" until the item was re-listed. Make the encoding reversible instead of stateful: - internal/contentid: add Pack/Unpack, a bit-packed, fixed-budget (<=15 byte) binary form of a structured or local content_id. digitCount preserves provider-id leading zeros (e.g. imdb tt0944947); structured forms are self-delimiting; the local form fills the budget exactly. Provider ids that overflow uint64 return ok=false. - Shrink ForLocal to a 112-bit (sha256(path)[:14]) hash so a local id packs losslessly into the 15-byte UUID payload. 112 bits is far beyond any single server's local-item count. No other code assumed the old width. - internal/jellycompat: EncodeStringID packs item/season content_ids into the UUID (byte 0 = kind, bytes 1..15 = packed, non-zero tag distinguishes it from the numeric encoding); DecodeStringID unpacks first and re-packs to confirm, so an opaque id whose bytes merely parse is rejected and falls through to the map. Numeric ids and arbitrary names (genres, studios) are unchanged. Net: item/season ids decode by pure computation — stable across restarts and across instances — with no lookup table. Only the rare unpackable content_id and non-content names still use the in-memory map. TDD: round-trip property tests in contentid (all kinds, leading zeros, reject cases) and a cross-instance decode test in jellycompat that fails on the old hash+map path. Full contentid + jellycompat suites green; production code golangci-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(migrate): make the migration run timeout configurable (SILO_MIGRATE_TIMEOUT) The boot-path migration runner hardcoded a 5-minute context timeout. The deterministic-content-id value-remap (20260612130000) does a full-table COLLATE rewrite + 65-column remap that needs ~20 min on a real dataset (615k items / 2M episodes), so it was cancelled at 5 min. Worse, Postgres keeps the orphaned backend running (holding AccessExclusive locks) until it notices the dead client at a statement boundary, while the goose session advisory lock releases on disconnect — so each 5-min boot retry piled a new attempt behind the previous one's locks. The migration never applied; the server boot-looped. Make the timeout configurable via SILO_MIGRATE_TIMEOUT (a Go duration like "60m"); 0 or negative disables the deadline for a one-off heavy migration. Default stays 5m. All three entry points (migrate-status, --migrate-only, boot) honor it. Required for the deterministic-content-id migration to apply on any real-sized database, not just dev — the 5m cap made the PR undeployable at scale. Follow-up (not here): on cancellation the runner should actively terminate its backend so a future timeout cannot orphan a lock-holding statement. TDD: MigrationTimeout parsing (default/override/zero/invalid) + MigrationContext deadline behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(contentid): require exact length for local ids in Unpack Tighten the tagLocal branch of Unpack from `len(body) < localHashLen` to an equality check. The local form fills the compat-UUID payload exactly (no padding), so a body of any other length is non-canonical; matching it exactly keeps Unpack a strict fail-closed inverse of Pack for the fixed-length branch, which decodes client-supplied UUIDs. Not applied to the structured branch (a review suggestion proposed the same change there): structured ids are self-delimiting and the compat layer pads them with trailing zeros to fill the 15-byte UUID payload, so ignoring trailing bytes is intentional and documented. Rejecting them would make every structured id fail to decode — the jellycompat cross-instance test guards against that. Not a live bug today (the only caller passes u[1:] from a 16-byte UUID, so body is always exactly localHashLen, and idcodec re-packs to verify), but it is the correct contract and zero-risk. Adds a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
799754dde7 |
feat(notifications): rich Discord embeds with posters, provider links, and ratings
Upgrade all outbound Discord surfaces (personal webhooks, bot DMs, server channels, request events) from bare title/description embeds to rich ones: poster thumbnail, overview teaser, TMDB/IMDb/TVDB links, rating and genre fields, content-rating footer, and a clickable title URL. Artwork respects the v1 privacy contract via a new admin poster mode (notifications.discord.poster_mode): "provider" (default) only emits public provider-CDN URLs, "server" additionally presigns locally cached posters from this server's image storage, "off" drops images entirely. Builders never derive artwork URLs themselves; the sender layer resolves PosterURL through System.discordPosterURL. To keep provider-CDN URLs derivable after image caching rewrites poster_path to a local storage key, media_items gains poster_source_path, captured during cacheItemImages, preserved across refreshes that keep the cached poster, and cleared on explicit poster overrides. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a17529fd6f |
feat(config): live admin settings + truthful restart-required banner (#128)
* feat(nodeconfig): harden config watcher for integrated-mode use - RequestReload(): non-blocking, coalescing reload nudge that runs on the poll goroutine, so concurrent requests can never swap a stale snapshot over a newer one (unlike ForceReload from request handlers) - Skip OnChange callbacks when the reloaded config is deep-equal to the previous one, so the 60s poll doesn't fire rebuild/log callbacks on no-op reloads - Add RedisURL to BootstrapOverrides; previously a reload clobbered an env-provided Redis URL in the live config - Split reload into fetchSettings/applySettings and add unit tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): hot-reload config watcher in integrated mode Start nodeconfig.Watcher in integrated/api mode (previously only proxy/ transcode worker modes hot-reloaded). Expose the live config to the API and jellycompat routers via func-typed LiveConfig/OnConfigChange fields with nil fallbacks to the startup snapshot, and wire the admin settings update hook to RequestReload so same-process changes apply immediately even without Redis. No consumer reads the live config yet — conversions land separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(admin): truthful restart-required banner for settings saves The settings UI showed 'restart required' after every save regardless of the key. The backend now classifies each key via a central registry (internal/config/restart_keys.go) and PUT /admin/settings/{key} reports restart_required per key; useSettingsForm only raises the banner when a saved key actually needs a restart (and keeps it raised until restart). The registry is conservative: every currently startup-frozen key is marked restart-required; subsequent hot-reload conversions shrink it. Settings read live from the settings repo (branding, overlays, markers, download.*, ...) default to no-restart. DownloadSettings/OverlaySettings drop their hardcoded restartRequired={false} special-casing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(logging): hot-reload server.log_level and server.log_quiet Share one slog.LevelVar across the handler chain and make logfilter.Handler's quiet-prefix list an atomic pointer shared with WithAttrs/WithGroup clones (New previously returned the inner handler unwrapped when the quiet list was empty, leaving nothing to update). The integrated-mode config watcher now applies both settings live; their keys leave the restart-required registry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(auth): hot-reload access/refresh token expiries JWTService stores expiries as atomics with a SetExpiries hook; all three instances (main API, ABS compat, jellycompat) re-apply them on config reload. Applies to newly issued tokens; outstanding tokens keep their original expiry. The JWT secret stays fixed for the process lifetime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(playback): read transcode config live at session start The playback and stream handlers pull ffmpeg path / hwaccel / transcode dir from the live config when starting a transcode or extracting subtitles, instead of values frozen at router construction. Each session snapshots the config once so its output dir and binary stay consistent. Also fixes a real bug: playback.hw_device was parsed into the config but never wired into the integrated-mode handler, so local transcodes always ran with an empty HWDevice while transcode nodes honored it. playback.transcode_dir leaves the restart-required registry (the handler is its only consumer); ffmpeg_path/hw_accel stay restart-required until scanner/chapterthumbs/audiobook consumers convert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(jellycompat): read compat identity settings live per request System/Auth handlers take a config provider instead of the startup snapshot, so jellyfin_compat.public_url, .server_name, and .emulated_server_version apply without restart. server_id stays restart-required (generate-once, baked into the resource mapper), as do the session-store TTLs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scanner,metadata,mdblist): hot-reload worker pools and API key scanner.workers, matcher.workers/batch_size, metadata.cache_images, and mdblist.api_key convert to atomic fields with setters wired to the config watcher. Worker counts apply on the next scan/match cycle (the loops read them per cycle); the MDBList key applies to the next request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): hot-reload AI connection, models, toggles, and quotas The shared llm.Client holds its config behind an atomic pointer (UpdateConfig; each request snapshots once), and the subtitle/metadata AI services gain UpdateConfig plus setters on the translator (batching) and Whisper transcriber (ffmpeg path, chunk seconds). The router derives their configs from shared helpers used both at construction and in OnConfigChange callbacks, re-evaluating the chat-only-gateway transcribe guard on each reload and warning only when it newly fires. Everything on the AI Services page now applies without restart except ai.max_concurrent_jobs (fixed-capacity dispatch semaphore). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): wire transcode_enabled; remove dead playback/scanner knobs playback.transcode_enabled was parsed into the config but the resolver always received a hardcoded true — the admin toggle did nothing. It now reads the live config per playback start, so disabling transcodes applies without restart. Remove settings that were wired to nothing so 'save + restart' stops pretending: playback.allow_hevc_encoding (resolver field never assigned), playback.transcode_ahead_segments and playback.segment_duration (parsed, never consumed — segment duration is per-session from the client), scanner.file_removal_grace (DeleteMissing is never called). UI fields removed and the config struct fields pruned so they don't resurrect; YAML import still tolerates the legacy keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6f716d3404 |
fix(scanner): skip misplaced TV episodes in movie libraries (#90)
* fix(scanner): skip misplaced TV episodes in movie libraries
A movie-type library containing a TV show laid out as
"Show/Season NN/SxxExx.mkv" (e.g. a fan "supercuts" pack) created one
bogus movie item per episode, all titled after the season folder
("Season 01"). They surfaced on Recently Added and never matched.
Add naming.IsMisplacedSeriesFile, which detects a file with an SxxExx
name inside an explicit "Season NN"/"Specials" directory, and guard
createOrFindSkeleton so such files in a strict movie library are recorded
as a skipped root (reason series_in_movie_library) instead of becoming
items. The movie match-queue worker now dequeues files the skeleton step
deliberately skips instead of erroring on the empty content id.
The guard fires on the structural signal alone, regardless of any parsed
provider id, because a "Season NN" folder otherwise yields a bogus tmdb
id (the season number). Movies whose release filename merely contains an
SxxExx substring but live in a proper "Title (Year)/" folder are
unaffected (covered by the new test).
* fix(metadata): durably exclude misplaced-series files from movie queue
Address review findings on the misplaced-TV skip:
- The skipped file's content_id is never set, so every library sync
re-enqueued it just for the worker to skip and dequeue it again.
Exclude files beneath a series_in_movie_library skipped root in the
movie match queue predicates; deleting the skipped root row makes the
files eligible again. The worker drain remains to flush rows claimed
before the root was recorded.
- Extract the eligibility predicate (previously duplicated verbatim in
eight queries) into movieQueueFileEligibleCond.
- Derive skipped-root file_count from media_files under the root via a
new UpsertObservedFile instead of hardcoding 1: a 34-episode pack now
reports 34 instead of each per-file upsert overwriting the count.
This also fixes the pre-existing missing_folder_ids undercount.
- Fold the two near-identical Upsert+log blocks in createOrFindSkeleton
into a recordSkippedRoot helper, normalize libraryType once, and name
the reason strings as constants.
- Drop a no-op filepath.Base on already-split path segments in
IsMisplacedSeriesFile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
39ba284c9d |
feat(ai): shared AI core — metadata translation, Whisper ASR, per-profile language, on-view translation (#127)
* docs: design + plan for shared AI core, metadata translation, Whisper ASR Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): shared LLM client, segment translator, and job runner packages internal/ai/llm: OpenAI-compatible chat client moved out of subtitles/ai, plus /v1/audio/transcriptions (verbose_json) for the ASR work; one shared retry/backoff loop for both. internal/ai/translate: the batched indexed-JSON translation protocol generalized to text segments. internal/ai/jobrunner: dispatch/heartbeat/reaper/cancel lifecycle extracted behind a minimal store interface, with a semaphore shareable across job services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(subtitles): consume shared AI core LLMTranslator becomes a thin cue<->segment adapter over aitranslate; the service delegates dispatch/heartbeat/reaper/cancel to jobrunner; the local OpenAI client is gone in favor of internal/ai/llm. Behavior (prompts, wire protocol, job rows, recovery semantics) is unchanged. NewService now takes the dispatch semaphore so all AI job services can share one bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): shared ai.* settings, metadata translation job table, localization provenance columns ai.* connection keys (chat + optional separate ASR endpoint) load with a fallback to the legacy subtitle_ai.* rows — those are never renamed in SQL because encrypted values are GCM-bound to their setting key. New toggles: subtitle_ai.transcribe_enabled, metadata_ai.enabled. Migration adds metadata_translation_jobs, per-field provenance (provider|ai|manual) on the localization tables, and media_folders.auto_translate_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): localization field provenance with provider/ai/manual precedence Provider upserts keep manual values and never blank a field with an empty incoming value; new UpsertAITranslation/UpsertAIOverview methods write AI fields only over empty or ai-sourced values (force adds provider, never manual) — all enforced in single-statement SQL. Serving now merges only non-empty localized fields onto the base item, since localization rows are legitimately partial (AI rows carry no titles/artwork). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(metadata): AI translation service, refresh auto-fallback, and admin API internal/metadata/translation: job service over the shared AI core that expands an item to its season/episode overviews, skips already-localized fields (zero model calls on repeat runs), batches paragraphs through the generic translator, and persists per batch with provenance-aware upserts. MetadataService gains an AutoTranslator seam invoked after each refresh for libraries with auto_translate_metadata. Admin endpoints under the metadata curation guard: enqueue, list (poll), cancel; plus a status probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(subtitles): Whisper ASR transcribe and transcribe_translate jobs New WhisperTranscriber: one ffmpeg pass extracts the audio track to 10-min 16kHz mono WAV chunks (temp dir cleaned on every exit path), each chunk goes to the OpenAI-compatible /v1/audio/transcriptions endpoint (verbose_json, per-request timeout sized to 3x chunk duration), segment timestamps are offset and built into wrapped cues. Chunks process playhead-first and stream live to the requesting session. The transcript is stored as an ordinary downloaded subtitle (provider 'transcribed'); transcribe_translate chains the existing translator and stores the translated track as the job result. Enqueue accepts an optional kind; status reports transcribe_enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): AI services settings, metadata translate action, library auto-translate, generate-from-audio New AI Services admin page hosts the shared endpoint config (reads fall back to legacy subtitle_ai.* values, writes target ai.*) and the three feature toggles; the AI card moves out of Subtitles settings. The metadata editor gains a Translate-with-AI panel with job polling and force/re-translate. The library form gains the auto-translate toggle (threaded through the libraries API). The player translate modal gains a From-audio mode that lists audio tracks and submits transcribe / transcribe_translate jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt import grouping in router and translation tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): per-profile metadata language and viewer-triggered description translation user_profiles.preferred_metadata_language threads through the access scope into catalog serving: presentation language now resolves explicit param -> profile preference -> library metadata language (native API and jellycompat). ItemDetail gains pending_translation_language when the viewer's language is missing a localized overview. New metadata_ai.on_view setting (off|button| auto) gates POST /items/{id}/translate-description: any profile with item access may request its language, with in-flight dedup and a 15-minute failure cooldown so page views never hammer a broken endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): on-view description translation with per-profile metadata language Profile playback settings gain a Metadata language picker (library default inherit). Detail pages: when the server reports pending_translation_language and metadata_ai.on_view is 'auto', the description translates on view with a pulse animation until the refetched detail comes back localized (45s timeout); in 'button' mode a small Translate chip triggers the same flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): expose metadata_ai.on_view in AI Services settings The on-view translation mode had no UI control, so it could only ever be 'off' — viewers got neither the auto translation nor the fallback button. Adds the off/button/auto selector to the Features card, and the config loader now warns and falls back to 'off' on a bad row instead of refusing to start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): clear configuration hint when the transcription endpoint is chat-only A blank Transcription base URL falls back to the chat endpoint; chat-only gateways reject the multipart upload with an opaque 400 that reads like a pipeline bug. 400/404/405 transcription failures now carry a hint to set a Whisper-compatible endpoint in AI Services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): wrap ASR cue text by rune count, not bytes Arabic/Cyrillic/Greek text is 2+ bytes per character in UTF-8, so byte-based wrapping broke lines at roughly half the intended visual width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(web): steer transcription base URL hint away from chat-only gateways Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): block chat-only gateways for transcription, add endpoint presets llm.IsChatOnlyGateway (OpenRouter et al — no timestamped transcription API) is enforced in three layers: the settings API rejects ai.asr_base_url values pointing at one, the router disables ASR with a warning when the blank-URL fallback would land on one, and llm.Transcribe refuses outright. The AI Services page gains one-click transcription presets (Groq turbo/accurate, OpenAI, self-hosted speaches) plus the mirrored client-side check, and the settings API now also validates metadata_ai.on_view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): tighten ASR subtitle sync Three systematic timing-error sources addressed: cue offsets now use the segment muxer's exact per-chunk start times (segment_list CSV) instead of assuming index*chunk_seconds; the audio stream's start delay relative to the container timeline (common in TS remuxes) is probed via ffprobe and added to every cue; and the chunk length is now operator-tunable via subtitle_ai.asr_chunk_seconds (60-600s, default 600) since shorter chunks bound Whisper's within-chunk timestamp drift. Playhead-first ordering now pivots on real chunk starts, and a beyond-end playhead starts at the final chunk instead of restarting from zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): tolerate base URLs that already include the /v1 segment Providers like DeepInfra expose their OpenAI-compatible API under a base that contains the version segment (api.deepinfra.com/v1/openai); always appending /v1/... mangled those. endpointURL now appends bare paths when the base already carries /v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): prefer self-hosted transcription in presets and hints Preset order becomes self-hosted (recommended) -> Groq turbo -> Groq large-v3 -> OpenAI, and the settings hint plus the job-error hint lead with the self-hosted option. The self-hosted preset now fills the turbo CT2 model to match the recommended speaches setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): request VAD and word timestamps for ASR cue accuracy Without vad_filter, faster-whisper servers report wall-to-wall segment times: cues linger on screen through silence (verified up to 91s) and paragraph-length segments become single 400+ char cues. Request vad_filter=true (skipped for hosted providers that reject non-OpenAI fields and run VAD server-side) plus timestamp_granularities word+segment, and rebuild cues from word timings: split at speech pauses, sentence ends, text capacity, and a 7s max duration; cap word-less segments instead of trusting their reported end; stretch sub-second cues to a readable minimum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
75b476d124 |
fix(naming,metadata): anchor identity on provider IDs, drop bare-numeric IDs, back off match queue retries (#112)
* fix(naming,metadata): anchor identity on provider IDs, drop bare-numeric IDs, back off match queue retries
Three scanner/matching fixes validated against dev data:
- Group identity: explicit structured provider tags ({tmdb-...},
[tvdbid-...]) now anchor a group's identity, so folder/file title
conflicts (renamed releases in Radarr-tagged folders) no longer mark
groups ambiguous and silently exclude them from matching. 3,049 of
3,121 ambiguous groups on dev carried explicit tags.
- ParseFolderIDs: remove bare trailing numeric ID parsing entirely,
mirroring Jellyfin's path-attribute model (bracketed key tags plus
unambiguous tt-prefixed IMDb ids only). Titles ending in numbers
("District 9", "Beverly Hills 90210", "Season 01") were misparsed as
trusted IDs, which suppresses title search and silently mismatches.
The folderType parameter existed only to type bare numerics, so it
is gone too.
- Match queues: replace the constant 15s/30s retry delay with shared
exponential backoff capped at 24h. Terminal failures ("no metadata
found from any provider") had rows at 15k+ attempts hot-looping
every 15s on dev.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(naming): merge trailing bare IMDb id with structured folder tags
ParseFolderIDs returned early on any structured tag, so a folder like
"Show [tvdbid-81189] tt1375666" lost the trailing IMDb id. Parse both and
merge, with an explicit structured imdb tag still taking precedence over a
trailing bare id. Matches Jellyfin, which resolves each provider key
independently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
zZebrahzandGitHub
|
96bae2f346 |
fix(library): clean orphaned provisional items (#76)
* fix(library): clean orphaned provisional items * test(metadata): use item not found sentinel in fake repo * fix(library): preserve abs references during orphan cleanup * fix(library): avoid dropped abs cleanup tables * fix(library): harden provisional orphan cleanup * fix(library): preserve home dismissal series orphans * fix(library): Delete matched items created after the first orphan sweep |
||
|
QuickandGitHub
|
c16d876007 | fix(metadata): gate episode refresh debt on provenance (#60) | ||
|
Quick
|
356c014f41 | fix(metadata): preserve provider ids on child 404 | ||
|
|
807384f4af |
fix(metadata): record post-refresh stale IDs against the canonical item (#38)
processInternal cleared and re-recorded stale provider IDs against req.ContentID after mergeAndPersist. But mergeAndPersist can canonicalize the item into an existing one (provider-ID dedup), deleting req.ContentID and returning a different result.ContentID. In that case the post-merge DeleteByContentID/Upsert targeted the now-deleted source: the upsert hit the stale_media_ids content_id foreign key and the still-404ing providers were never recorded on the surviving canonical item — so the same providers get re-attempted (and re-404) on every subsequent refresh. Target the canonical ID (result.ContentID, falling back to req.ContentID) for the stale-ID follow-up via refreshFollowUpContentID, matching what the adjacent refresh-debt sync already uses. The block stays guarded on provider404s != nil (allocated only when req.ContentID was set) so a content-id-less refresh that canonicalizes into an existing item does not clear that item's stale rows without re-recording any. Unit-tested for the nil/empty/canonicalized cases. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
c5daad0dcb |
fix(metadata): tolerate missing content item when recording stale IDs (#37)
StaleMediaIDRepository.Upsert inserts (content_id, provider, ...) into stale_media_ids, which has a foreign key to media_items.content_id. A metadata refresh can start for an item that is then deleted or merged away (e.g. by provider-ID canonicalization) before the provider 404 lands and triggers the upsert. The parent row is gone by then, so the insert fails with a 23503 foreign-key violation (stale_media_ids_content_id_fkey) and the refresh surfaces a spurious error — observed in production postgres logs. When the referenced item no longer exists there is nothing left to track, so treat that specific FK violation as a logged no-op (at Info, with provider_id, so a deletion wave stays visible). Other errors still propagate. The swallow decision is extracted to resolveStaleUpsertError so it is unit-testable without a database, and the pgconn error-classification boilerplate is consolidated into a shared isPgConstraintViolation helper that isProviderIDUniqueConflict now also uses. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
1e36d42bcc |
fix(metadata): don't promote season/episode 404s to item-level stale provider IDs (#35)
* fix(metadata): don't promote season/episode 404s to item-level stale provider IDs
The main-refresh path in MetadataService.processInternal passes
`provider404s` to handleProvider404 for season (line 1140) and episode
(line 1173) fetch errors. handleProvider404 records the item's provider
ID as stale whenever provider404s is non-nil. A 404 fetching seasons or
episodes is a child-resource failure (e.g. TMDB returns 404 for
`/tv/{id}/season/0` when a series has no specials season) — it must not
mark the item's TMDB ID itself as stale.
The equivalent target-refresh paths (fetchTargetSeasonResults at line
2305 and fetchTargetEpisodeResults at line 2339) already pass `nil` for
this exact reason. The main-refresh path was the missing half of that
fix.
In our deployment this bug caused 13,524 series items to have their
`media_items.tmdb_id` cleared and recorded as stale during a single
refresh sweep — every series whose TMDB entry doesn't have a 'specials'
season was affected. Once recorded, `suppressRecordedStaleProviderIDs`
locked the items out of future TMDB enrichment.
Tested by curl-ing 5 affected items against TMDB directly:
- /tv/{id} returns 200 for all 5
- /season/1 returns 200 for all 5
- /season/0 returns 404 for 4 of 5 (the ones that don't have specials)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
|
||
|
Quick
|
0e73e6c240 | Merge branch 'main' of https://github.com/Silo-Server/silo-server | ||
|
Silo Server Developer
|
6c071e7248 |
diag(matcher): debug-log provider search query + per-provider result counts
Adds DEBUG logs in the ModeInitialMatch search path: each provider's result
count for the query, and the assembled raw/candidate totals. Lets us see when a
provider search returns zero ('no metadata found') vs a scoring/tie issue.
Zero behavior change.
|
||
|
Silo Server DeveloperandClaude Opus 4.7
|
b5e4213e81 |
feat(matcher): resolve cross-source ties by library provider priority
Accept a year-corroborated single distinct show when the TOP tie-group (within 15 pts of best) is one show across providers, ignoring low-score noise below it, and pick the winner by the library's metadata-provider chain order (providerPriority, highest-first; falls back to top-scored). Recovers items like '100 Days Wild' that are returned identically by TVDB and TMDB. Thresholds (55/70/15) unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Quick
|
2637061601 | feat(calendar): show local episode airtimes | ||
|
Quick
|
bba3177fc9 |
fix(metadata): break duplicate provider candidate ties
- Score candidate metadata completeness and auto-match the richer duplicate when title/year/type tie - Enrich near-duplicate candidates via the provider chain before initial match selection - Seed both movie and series match queues for mixed-type libraries and wait for TV queue settle - Add taskmanager worker test coverage and a plan doc for the tie-breaker work |
||
|
Silo Server MigrationandClaude Opus 4.7
|
ab520d2743 |
fix(metadata): skip refresh-failure mask when provider id is missing
syncRefreshDebtFailure unconditionally set RefreshDebtReasonRefreshFailure on matched items, even when the refresh failed because the item is missing a provider id. The priority logic already preferred ProviderIDIncomplete (priority 240) over RefreshFailure (priority 190), so scheduling was correct — but the persisted reason mask combined both bits and the reason-count metric reported these items as "refresh failures" instead of "provider id incomplete". Guard the OR so the failure bit only lands on items that actually have a provider id and failed for a different reason. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Silo Server Migration
|
c085b12fd1 | Initial Silo migration |