Files
silo-server/internal/plugins/service_installation_cache_test.go
T
430224a1b9 perf: cut home-screen, Continue Watching, and Latest latency; cache shared home rails (#292)
* perf(jellycompat,sections): bound resume scan, batch leaf detail progress, widen section concurrency

Three low-risk fixes from the section-fetch performance investigation
(docs/superpowers/plans/2026-07-03-section-fetch-performance.md):

- jellycompat: bound loadProgressPage at resumeScanMaxRows=300 so a single
  request never pages through more than that many in-progress rows. The cap is
  unconditional: it also covers the sparse-visible-set case (a heavy watcher
  whose recent rows are mostly dismissed/superseded, or a Series/Season-only
  request that matches no leaf in-progress row), where the page never fills and
  the loop would otherwise scan the entire history — previously an O(history)
  scan reaching tens of seconds. In the common case the loop exits far earlier,
  so the cap only bounds the pathological worst case; 300 leaves ample headroom
  to fill a ~20-item Continue Watching page. Beyond the cap the reported total
  is a clamped lower bound. Covered by TestLoadProgressPage_BoundsScanForSparseVisibleSet.
- jellycompat: batch the leaf-item (movie/episode) progress lookup in
  GetItemDetailsByIDs via ListProgressWithCompletedHistory instead of a
  per-item GetProgressWithCompletedHistory (~100 sequential queries for a
  50-item detail page). Series keep the per-item episode-rollup path (they own
  no progress row). Output is unchanged; a batch-lookup failure is now logged
  rather than silently dropping played state for the whole page.
- sections: raise fetchAllMaxConcurrency 4 -> 6 to cut FetchAll wave count for
  large home layouts, staying within the default 20-conn pool.

Part of the home/continue-watching latency work.

AI-use disclosure: implemented with AI (Claude) assistance.

* perf(jellycompat): keep Latest browse on the cross-library fast path under isPlayed

/Items/Latest with isPlayed=false is the highest-frequency compat browse
(~10.8k calls/day). The played overlay can't be pushed into SQL, so browse
over-fetches and filters locally. The cross-library recently_added fast path
(BrowseRecentlyAddedAcrossLibraries: one ~1ms index walk per library) was
gated on Offset==0, so a heavy watcher who had already seen the newest items
needed a 2nd chunk and fell through to BrowsePage — a whole-catalog
MIN(first_seen_at) + GROUP BY HashAggregate over ~147k movies measured at
~755ms per call (0.8-1.6s observed end-to-end).

Fetch the entire over-fetch budget (maxScannedRows) in a single merged
fast-path walk instead of paging into BrowsePage, so the loop fills from one
call. The clamp caveat (MaxLimit=1000 leaves a fall-through only for
requestedLimit>200, off the Latest hot path) is documented inline.

Part of the home/browse latency work.

AI-use disclosure: implemented with AI (Claude) assistance.

* fix(jellycompat): scope resume scan cap to resume path and bound the fast-path loop

Addresses PR #292 review feedback:

- Codex (P2): the resumeScanMaxRows cap was applied unconditionally in the
  general loop, which also paginates the completed (watched-items) list. Gate it
  on resumeFiltered so the completed path keeps exact TotalRecordCount and deep
  StartIndex pagination. Covered by TestLoadProgressPage_CompletedScanNotCapped.
- CodeRabbit (Critical): the earlier raw-offset fast-path loop — the default
  Continue Watching shape and the sections-fallback route — had the same
  unbounded-scan bug and was not covered by the cap (the existing test forces
  EnableTotalRecordCount=true, routing around it). Bound it with the same
  resumeScanMaxRows guard. Covered by
  TestLoadProgressPage_BoundsFastPathScanForSparseVisibleSet.
- CodeRabbit (Minor): tag the doc's fenced example blocks as text to satisfy
  markdownlint MD040.

AI-use disclosure: implemented with AI (Claude) assistance.

* perf(sections): cache shared user-agnostic home rails per access scope

Home-screen rails that are identical for everyone who can see the same
libraries (recently added, recently released, genre, trending on server,
most watched, new to library, critically acclaimed, award winners, format
showcase, seasonal, mood, trending discover, admin-curated lists, and
library collections) were rebuilt from Postgres once per request, per user.
Only the overlay on top of each row (watched flags, play position, presigned
poster URLs) is actually per-user.

Insert a process-global resolved-list cache at the FetchOne choke point in
internal/sections. Each cacheable row is built once per access scope, held
with a 15m TTL, and refreshed in the background 3m before expiry;
singleflight collapses cold-miss stampedes into a single build. The per-user
overlay still runs fresh in buildSectionsResponse, so no profile state is
ever shared. Random and per-user rows (continue watching, next up,
recommendations, hidden gems, forgotten favorites, activity feed, user
collections) bypass the cache.

The access-scope key captures every access boundary the fetch path enforces
-- section identity (type + id + config hash) + item limit + accessible and
disabled libraries + max content rating + excluded media types + name prefix
+ allowed-content-id allowlist -- and nothing per-user, so entries are safely
shared. Empty membership is never cached (avoids freezing a transiently empty
rail); background refreshes are bounded by a timeout.

Scale (analytical, derived from the cache behavior -- not a measured
latency): for the user-agnostic rows, Postgres section-query volume collapses
from O(rows x concurrent requests) to O(rows x distinct access scopes) per
15m refresh window, because most users share a handful of access scopes.
Illustrative -- 40 cacheable rows on a home screen, 1000 concurrent users
falling into ~5 distinct access scopes:

  - before: ~40 x 1000 = ~40,000 section queries per wave of home loads
  - after:  ~40 x 5    = ~200 builds per 15m window (plus one background
            refresh per row per scope), i.e. a warm home load runs zero
            section queries for these rows.

That is a ~99% reduction in shared section-query load at that concurrency;
the win grows with concurrency and shrinks as access-scope diversity rises.

Design/plan doc added under docs/superpowers/plans/.

* perf(jellycompat): serve per-library Latest via the cached recently-added section

A jellyfin-compat per-library /Items/Latest rail is the same user-agnostic
list as the native "recently added" library rail -- both order by
mil.first_seen_at DESC. It was rebuilt on every request through
directContentService.BrowseItems, missing the resolved-list cache entirely.

Route per-library Latest for movies and series libraries through the native
section fetch instead, so it reuses the shared cache. HandleLatest resolves
the library's type once, and for a movies/series library builds a synthetic
SectionRecentlyAdded with the same type + config + limit + access scope the
native rail uses and calls FetchOne; the per-user overlay (favorites,
progress, episode targets, presign) is extracted into buildLatestItemDTOs and
shared by both the native and BrowseItems paths, so no overlay logic is
duplicated. Cached *models.MediaItem values are read-only -- LocalizeItemModels
deep-copies before any presign mutation.

To let the two surfaces share one entry, resolvedListCacheKey no longer
includes the arbitrary section ID: every cacheable section type derives its
membership from type + config + limit + scope, never from its own ID (audited
all 14 cacheable types plus the library-collection path; the sole s.ID read
lives in the non-cacheable user-collection branch). A native recently-added
rail and the compat Latest for the same library + scope now collapse to ONE
cache entry, built once and reused. Access-scope isolation is unchanged --
the removed ID never carried access information, and every access boundary
(libraries, rating cap, excluded types, content allow-list, name prefix)
still keys the entry.

Guardrails: the native path is restricted to movies and series libraries;
every other library type (ebook, music, manga, mixed) is ignored and keeps
its exact BrowseItems behavior -- important because an unfiltered
recently-added fetch would otherwise surface non-video items to Jellyfin
clients that only expect video. Deeper pages, played-filter and
backdrop-required requests, a client asking for a type other than the
library's own, and any FetchOne error also fall back to BrowseItems.

Chosen over an alternative that gave the synthetic section a deterministic ID
(which kept two separate cache entries): both returned identical data with
similar complexity, so the shared-entry design won.

* fix(sections,jellycompat): post-review fixes for the shared-list cache and Latest path

Consolidates fixes from the branch's adversarial review and PR #292 review
comments into one commit:

- Latest fast path: fall back to BrowseItems when a request carries a genre,
  name-prefix, or person filter (the synthetic recently-added section cannot
  express these, so serving it unfiltered would return a wrong, broader set).
  Eligibility is decided by latestFastPathEligible and covered by a test.
- Clamp the /Items/Latest page size to compatBrowseMaxLimit before building the
  section, matching the BrowseItems fallback, so a large client Limit can't drive
  an oversized recently-added fetch or explode the shared cache key with unbounded
  ItemLimit values.
- Evict expired entries from the process-global resolvedListCache: resolvedListSet
  sweeps expired keys at most once per minute, bounding the map to scopes seen
  within one TTL window. Covered by TestResolvedListCacheEvictsExpiredEntries.
- Log a short digest of the cache key (resolvedListLogKey) instead of the raw key
  in the background-refresh panic/error paths, since the key embeds
  user-controlled access-scope fields such as NamePrefix.

Skipped review comments (verified already fixed or stale against current code):
the resume fast-path scan bound and watched-items cap (04d2e795) and the docs
fence-language tags (already addressed).

Build, vet, and go test -race pass for internal/sections and internal/jellycompat.

* perf(plugins): cache plugin installations in-memory, invalidated on lifecycle change

## Problem
Every poster/image on a warm home rail re-read plugin_installations from
Postgres to answer "is this plugin enabled?" and to acquire the plugin client
(Source A: metadata chain buildProviders enabled-check; Source B: ensureClient
-> loadInstallation). Plugin-resolved image URLs are never URL-cached, so the
plugin source and the DB read behind it fired again on every identical warm
request; 100% of images in the target library are plugin-backed.

## Solution
- Guarded in-memory installation cache (map[int]*Installation + RWMutex) in
  plugins.Service. loadInstallation reads through it; the requireEnabled gate
  stays after the cache read so ErrInstallationDisabled semantics are unchanged.
  invalidateInstallationCache clears it and is self-registered as a lifecycle
  hook, so Service.OnLifecycleChange wipes it on install/enable/disable/update/
  uninstall.
- A generation counter closes an invalidate-vs-repopulate race: captured before
  installations.GetByID and re-checked under the write lock, so a row fetched
  before a lifecycle mutation is never written into a freshly cleared cache
  (would otherwise resurrect a just-disabled plugin).
- Route the metadata chain enabled-check through the same cache via a structural
  InstallationEnabledChecker interface (nil-safe: falls back to the pool query
  when no checker is injected), wired in cmd/silo/main.go.

## Post-review fix (auto-update reliability blocker)
AutoUpdateService mutated installations (new InstallPath, old dir deleted) on
the default auto update policy without firing OnLifecycleChange, leaving the
cache stale and breaking plugins with "stored plugin manifest mismatch" until
restart. It now takes an onChange callback wired to Service.OnLifecycleChange
and fires it once per Check run that mutated a row.

## Verification
go build/vet, go test ./internal/plugins/... ./internal/metadata/... (-race).
Tests: cache hit/invalidation, racing-invalidation guard, IsInstallationEnabled,
auto-update fires onChange.

## AI-use disclosure
Implemented with AI assistance (Claude).

* perf(jellycompat): batch per-item presign, and enrich series on the cached Latest path

## Problem
List rails presigned each item's poster/backdrop/logo/still image individually
(~160 singular resolver calls for a 40-item page where 4 batched calls suffice),
and ItemsHandler carried a near-verbatim duplicate of the batch presigner.

## Solution (batching)
Promote the batch presigner to a shared package-level presignCompatListItems
(presign_list.go) with a generic collectImagePaths[T]; convert the per-item
loops (cached home/Latest rail, favorites, batch loaders, userdata favorites) to
one batched PresignImageURLsWithExpiry per image type per page; batch the
season/episode collections; delete the three duplicate presign helpers. URL
output is unchanged (verified byte-for-byte).

## Post-review fix (series Latest data-parity regression)
The native cached Latest fast path built items via compatListItemsFromModels +
buildLatestItemDTOs and never ran the series watch-state rollup, so a series
library's Latest lost Played / UnplayedItemCount and page 1 disagreed with the
BrowseItems fallback. enrichSeriesUserData is promoted to the ContentService
interface and called on the native path (reused, not duplicated).

## Verification
go build/vet, go test ./internal/jellycompat/... ./internal/catalog/...
Tests: bounded presign invocation counts + per-item URL mapping; series rollup
populated on the native Latest path.

## AI-use disclosure
Implemented with AI assistance (Claude).

* perf(sections): gate personalized rails out of the shared cache; widen refresh lead

## Problem
1. The shared home-rail cache whitelisted custom_filter/genre sections by TYPE
   alone, but those route through fetchFiltered -> ParseQueryDefinition and can
   carry personalized (per-profile) rules/sorts (watched, favorited,
   in_watchlist, in_progress, last_watched; sorts progress/date_viewed/plays).
   Their membership is per-profile yet the cache key excludes userID/profileID,
   so a personalized rail built for one profile was served to others in the same
   access scope for up to 15m -- a cross-profile watchlist/watch-state leak.
2. The background-refresh lead was tuned so steady traffic is served a warm
   entry from a longer soft window.

## Solution
- Add QueryDefinition.IsPersonalized() (reusing the existing
  QueryFieldRequiresProfile/QuerySortRequiresProfile helpers).
  isCacheableSectionType now parses the section QueryDefinition and refuses to
  cache custom_filter/genre when personalized; non-personalized definitions stay
  cacheable. Seasonal/mood/trending build their definitions server-side and stay
  unconditionally cacheable.
- resolvedListRefreshLead 3m -> 10m (soft threshold builtAt+5min instead of
  builtAt+12min).

## Verification
go build/vet, go test ./internal/sections/... ./internal/catalog/... (-race).
Test: personalized custom_filter/genre not cacheable; non-personalized are.

## AI-use disclosure
Implemented with AI assistance (Claude).

* fix(sections,metadata): post-review fixes for shared cache and plugin chain staleness

Addresses three review findings on PR #292:

- sections: canonicalize section config JSON before hashing so configs
  differing only in whitespace/field order share a cache entry (native +
  jellycompat rail sharing). Added TestHashSectionConfigCanonicalizes.
- metadata: invalidate the resolved-chain cache on plugin lifecycle
  changes; the installation-enabled check already reads the invalidated
  plugin cache, but resolveChainCached could serve a stale provider chain
  for up to chainCacheTTL after a provider's availability changed.
- jellycompat: move ctx to the first parameter of presignCompatListItems
  for consistency with the other presign helpers.

Skipped the episode-image presign batching nitpick: the resolver already
dedupes+singleflights, so it is a Minor perf-only item not worth the
two-pass refactor risk in this pass.

* fix(sections,jellycompat): harden shared rail cache and Latest fast path per review

Addresses the eight findings from the deep review of this PR:

- Detach the blocking cold-miss rebuild from the singleflight leader's
  request context (context.WithoutCancel + the shared 30s build timeout)
  so one client disconnect no longer fails every collapsed waiter and
  leaves the entry uncached.
- Stop client-controlled values minting unbounded cache entries: the
  compat Latest fast path now always fetches a fixed 100-row budget and
  slices to the requested limit (one entry per scope+library instead of
  one per Limit value), and an unrecognized MaxOfficialRating string
  disqualifies the fast path instead of entering the global cache key.
- Add release_date to the sections item projection/scan so movies served
  via the Latest fast path keep PremiereDate (Jellyfin default-set field)
  in parity with the BrowseItems fallback.
- Fall back to per-item progress lookups when the batched leaf progress
  query fails, restoring one-item-at-a-time degradation instead of
  blanking played state for the whole page.
- Derive cache eligibility from a single source of truth: fetchSection
  and isCacheableSectionType now share the userAgnosticSectionFetcher
  table, whose no-userID/profileID signature makes a fetcher drop out of
  the cacheable set at compile time if it ever gains per-profile inputs.
- Decide Latest fast-path eligibility off the actual browse params the
  fallback would receive, so any filter later added to buildBrowseParams
  automatically disqualifies the cached path; share one
  compatDefaultBrowseLimit constant between both paths.
- Extract AccessFilter.WriteAccessScopeCacheKey as the shared, security-
  critical serializer for all access-scoped caches (resolved-list,
  editorial candidates, audiobook groups); the editorial key now captures
  ExcludedMediaTypes, which its loaders already applied in SQL.
- Strip leaked agent-transcript markup from the section-fetch plan doc.

go build ./..., go vet, gofmt clean; go test -race on
internal/sections, internal/catalog, internal/jellycompat passes
(TestBeginWebOperation* failures are the known pre-existing flakes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 02:01:30 -04:00

173 lines
6.3 KiB
Go

package plugins
import (
"context"
"errors"
"testing"
)
// countingInstallationStore wraps the shared fake store and counts GetByID
// calls so tests can assert the in-memory installation cache absorbs repeat
// reads.
type countingInstallationStore struct {
*fakeServiceInstallationStore
getByIDCalls int
// onGetByID, when set, runs after the call is counted but before the row is
// returned, so tests can simulate an invalidation racing an in-flight read.
onGetByID func()
}
func (s *countingInstallationStore) GetByID(ctx context.Context, id int) (*Installation, error) {
s.getByIDCalls++
if s.onGetByID != nil {
s.onGetByID()
}
return s.fakeServiceInstallationStore.GetByID(ctx, id)
}
// newCachedInstallationService builds a Service backed by the counting store and
// wires the installation-cache invalidation exactly as NewService does, so the
// OnLifecycleChange -> invalidate path is exercised.
func newCachedInstallationService(installations ...*Installation) (*Service, *countingInstallationStore) {
store := &countingInstallationStore{
fakeServiceInstallationStore: newFakeServiceInstallationStore(installations...),
}
svc := &Service{installations: store}
svc.AddLifecycleHook(func(context.Context) { svc.invalidateInstallationCache() })
return svc, store
}
func TestLoadInstallationCachesAndInvalidatesOnLifecycleChange(t *testing.T) {
ctx := context.Background()
svc, store := newCachedInstallationService(&Installation{ID: 7, PluginID: "silo.metadb", Enabled: true})
// First read hits the store.
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("first loadInstallation err = %v", err)
}
if store.getByIDCalls != 1 {
t.Fatalf("after first read GetByID calls = %d, want 1", store.getByIDCalls)
}
// Subsequent reads are served from the cache.
for i := 0; i < 5; i++ {
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("cached loadInstallation err = %v", err)
}
}
if store.getByIDCalls != 1 {
t.Fatalf("after cached reads GetByID calls = %d, want still 1", store.getByIDCalls)
}
// A lifecycle change wipes the cache and forces a re-read.
svc.OnLifecycleChange(ctx)
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("post-invalidate loadInstallation err = %v", err)
}
if store.getByIDCalls != 2 {
t.Fatalf("after lifecycle change GetByID calls = %d, want 2", store.getByIDCalls)
}
}
func TestIsInstallationEnabledReflectsCacheAndInvalidation(t *testing.T) {
ctx := context.Background()
svc, store := newCachedInstallationService(&Installation{ID: 7, PluginID: "silo.metadb", Enabled: true})
enabled, err := svc.IsInstallationEnabled(ctx, 7)
if err != nil {
t.Fatalf("IsInstallationEnabled err = %v", err)
}
if !enabled {
t.Fatal("IsInstallationEnabled = false, want true")
}
if store.getByIDCalls != 1 {
t.Fatalf("GetByID calls = %d, want 1", store.getByIDCalls)
}
// Disable the underlying row. Until a lifecycle change, the cache still
// reports the stale (enabled) value and issues no further reads.
falseVal := false
if err := store.Update(ctx, 7, UpdateInstallationInput{Enabled: &falseVal}); err != nil {
t.Fatalf("store.Update err = %v", err)
}
enabled, err = svc.IsInstallationEnabled(ctx, 7)
if err != nil {
t.Fatalf("cached IsInstallationEnabled err = %v", err)
}
if !enabled {
t.Fatal("cached IsInstallationEnabled = false, want stale true before invalidation")
}
if store.getByIDCalls != 1 {
t.Fatalf("GetByID calls = %d, want still 1 (served from cache)", store.getByIDCalls)
}
// After a lifecycle change the cache is wiped and the new value is read.
svc.OnLifecycleChange(ctx)
enabled, err = svc.IsInstallationEnabled(ctx, 7)
if err != nil {
t.Fatalf("post-invalidate IsInstallationEnabled err = %v", err)
}
if enabled {
t.Fatal("post-invalidate IsInstallationEnabled = true, want false")
}
if store.getByIDCalls != 2 {
t.Fatalf("GetByID calls = %d, want 2 after invalidation", store.getByIDCalls)
}
}
// TestCachedInstallationSkipsWriteOnRacingInvalidation proves the generation
// guard: if a lifecycle invalidation lands while GetByID is in flight, the
// fetched (potentially pre-mutation) row is returned to the caller but is not
// written back into the freshly-cleared cache, so the next read re-fetches
// instead of serving a resurrected stale row.
func TestCachedInstallationSkipsWriteOnRacingInvalidation(t *testing.T) {
ctx := context.Background()
svc, store := newCachedInstallationService(&Installation{ID: 7, PluginID: "silo.metadb", Enabled: true})
// Simulate the race by invalidating the cache from inside the store read,
// i.e. between the generation capture and the write-back.
store.onGetByID = func() { svc.invalidateInstallationCache() }
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("racing loadInstallation err = %v", err)
}
if store.getByIDCalls != 1 {
t.Fatalf("GetByID calls = %d, want 1", store.getByIDCalls)
}
// The racing read must not have populated the cache; the next read re-fetches.
store.onGetByID = nil
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("post-race loadInstallation err = %v", err)
}
if store.getByIDCalls != 2 {
t.Fatalf("GetByID calls = %d, want 2 (racing write skipped, cache empty)", store.getByIDCalls)
}
// A subsequent read with no interference is served from the cache.
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("cached loadInstallation err = %v", err)
}
if store.getByIDCalls != 2 {
t.Fatalf("GetByID calls = %d, want still 2 (served from cache)", store.getByIDCalls)
}
}
func TestLoadInstallationRequireEnabledGateAppliesAfterCache(t *testing.T) {
ctx := context.Background()
svc, store := newCachedInstallationService(&Installation{ID: 7, PluginID: "silo.metadb", Enabled: false})
// requireEnabled=false caches the disabled row.
if _, err := svc.loadInstallation(ctx, 7, false); err != nil {
t.Fatalf("loadInstallation err = %v", err)
}
// requireEnabled=true returns ErrInstallationDisabled without a second read,
// proving the gate is applied after the cache lookup.
if _, err := svc.loadInstallation(ctx, 7, true); !errors.Is(err, ErrInstallationDisabled) {
t.Fatalf("loadInstallation requireEnabled err = %v, want ErrInstallationDisabled", err)
}
if store.getByIDCalls != 1 {
t.Fatalf("GetByID calls = %d, want 1 (gate applied after cache)", store.getByIDCalls)
}
}