Commit Graph
628 Commits
Author SHA1 Message Date
CoffeeKnyteandGitHub 1dbcf2cd9a fix(jellycompat): guard aux search paths and index-back person search (#252)
* fix(jellycompat): guard aux search paths and index-back person search

Short, recursive type-ahead terms (e.g. a single "a") against the
PostgreSQL people index and the in-memory collection/box-set filter
produced multi-second scans that pegged the server when a client fired
one search per keystroke. Every jellycompat search path except the
Meilisearch-backed /Items media search now rejects a provided SearchTerm
shorter than 3 runes without touching any backend, and caps results at
20 regardless of the client-requested Limit:
- /Persons        (PostgreSQL people scan)
- /Search/Hints   (catalog search)
- /Items BoxSet   (in-memory collection filter)

Shared policy + helpers live in search_guard.go (auxSearchMinTermLen=2,
i.e. reject 1-2 runes / allow 3+, and auxSearchMaxResults=20) with unit
coverage. The 3-rune floor is deliberate: 3 runes is the point where a
pg_trgm trigram index becomes usable, so the gate lines up with the index
and still lets legitimate short titles/names ("300", "Saw", 3-letter
actors) and 3-char type-ahead hints through.

PersonRepository.Search filters with `name ILIKE '%'||$1||'%'` rather
than `LOWER(name) LIKE '%'||LOWER($1)||'%'`. The old expression filtered
on LOWER(name), which the trigram GIN index idx_people_name_trgm (built
on name) could not serve, so every search fell back to an ordered index
scan on idx_people_name that walked the whole table for rare terms
(~300-400ms on the 889k-row production people table). ILIKE on name lets
pg_trgm serve rare 3+ char terms from the trigram index, while the planner
still picks the ordered btree scan with early termination for common
terms. ILIKE is case-insensitive, so behavior is preserved (including the
pre-existing treatment of % and _ in the term as LIKE wildcards).

Verified on production silo-postgres (889,302 people), parameterized query
under both custom and generic plan cache modes:
  rare 3-char 'qzx':  304ms -> 1-8ms   (trgm bitmap index)
  rare 4-char 'zzzz': 329ms -> 1-2ms   (trgm bitmap index)
  common 3-char 'ann':  15ms -> 7-74ms (btree early-stop / trgm bitmap)
New worst case across all terms is ~83ms (generic-plan common 3-char).

* fix(jellycompat): stop gating meilisearch hints, clamp box-set search

Two review follow-ups on the aux-search guards:

- HandleSearchHints is served by the catalog (Meilisearch-backed) search
  provider, which already bounds and short-term-handles its own results.
  Gating it with auxSearchTermTooShort contradicted the guard's own
  "non-Meilisearch paths only" policy and hid valid 1-2 char titles
  ("Up", "It") from global type-ahead. Drop the too-short gate there;
  keep the empty-term check and the result clamp.

- handleBoxSetsList gated short terms but passed query.limit straight to
  slicePage, which treats <=0 as no cap, so a box-set *search* was
  uncapped unlike Persons/Hints. Clamp the limit when a search term is
  present; empty-term browse keeps its client paging window.
2026-07-01 09:05:41 -04:00
Quick 75cf5e3b11 fix(nextup): include completed or partially watched items in NextUp query 2026-06-29 10:46:22 -04:00
QuickandGitHub 8afe783277 fix(jellycompat): honor StartItemId episode queues (#242)
* fix(jellycompat): honor StartItemId episode queues

* fix(jellycompat): compact episode queue models
2026-06-27 22:02:54 -04:00
QuickandGitHub 9cea2bb4f0 fix(metadata): enable AI translation on season and episode pages (#238) 2026-06-27 21:44:28 -04:00
QuickandGitHub a0fbe2cdda [codex] feat(jellycompat): use catalog search provider (#236)
* feat(jellycompat): use catalog search provider

* fix(jellycompat): route video search buckets to provider
2026-06-27 20:10:48 -04:00
QuickandClaude Opus 4.8 4d2a990de1 chore(skills): harden issue-to-pr with researched best practices
Folds in findings from Anthropic's Claude Code guidance and the agentic-SWE
literature:

- Manual-only trigger (disable-model-invocation) for this side-effecting,
  PR-creating workflow, per Anthropic's fix-issue template.
- Reproduce-first hard gate: confirm the bug exists on current main (red test
  before the fix); stop and report if it doesn't reproduce, rather than patching
  correct/stale code.
- Verification is the primary correctness signal; full-suite run for regressions;
  forbid reaching green by weakening/skipping tests or relaxing CI.
- Adversarial-review loop: confirm each finding is real/grounded before acting,
  fix only correctness/requirement-affecting findings (no gold-plating), and cap
  the loop at 2 (max 3) iterations before escalating.
- Guards against hallucinated APIs, plan-proportional-to-scope, and long-run
  context-loss (progress + failed-approaches notes, re-assert guardrails after
  compaction).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:07:36 -04:00
QuickandClaude Opus 4.8 96a279819e chore(skills): add issue-to-pr skill (worktree → fix → adversarial review → PR)
Adds a committed, shared Claude Code skill that takes a GitHub issue number,
creates an isolated worktree off the latest main, finds the root cause or
scopes the feature, implements and verifies the change, runs a Codex
adversarial review, and opens a ready PR.

Carves a narrow exception into the .gitignore /.claude/ rule so only this
skill directory is tracked; everything else under .claude/ stays ignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:07:36 -04:00
443445f0fe fix(catalog): make media_items scalar columns NOT NULL to stop NULL-scan crashes (#228)
media_items had many columns the Go model (models.MediaItem) declares as
non-pointer string/int fields, but migration 001 left them nullable. Five
scanners (catalog item_repo, catalog browse, jellycompat, sections, and
the catalog API handler) read these straight into the non-pointer fields,
so a NULL row panics with "cannot scan NULL into *string" (or *int).
item_repo papers over a subset (poster/backdrop/logo/metadata paths) with
COALESCE in its SELECT, but the other four scanners read the same columns
raw and crash; sort_title/original_title/etc. are not coalesced anywhere.

No writer stores a meaningful NULL (every insert/upsert passes the Go
field, '' or 0 at worst) and all sort/filter SQL already collapses NULL
and '' (e.g. COALESCE(NULLIF(BTRIM(sort_title), ''), title); "poster_path
IS NULL OR poster_path = ''"). Enforce the invariant the code already
assumes at the schema level rather than scattering COALESCE across every
current and future scanner. Mirrors what later migrations already did for
original_language, show_status, default_metadata_language, and the
*_source_path columns (all NOT NULL DEFAULT '').

- Migration: backfill existing NULLs, then NOT NULL DEFAULT '' on 15 text
  columns (sort_title, original_title, content_rating, overview, tagline,
  imdb_id/tmdb_id/tvdb_id, poster_path, poster_thumbhash, backdrop_path,
  backdrop_thumbhash, logo_path, metadata_s3_path, metadata_etag) and
  NOT NULL DEFAULT 0 on year/runtime.
- item_repo: SetLocalPoster and UpdateArtworkIfSourceMatches now store ''
  for empty thumbhashes (was NULLIF($,'')) — the only deliberate NULL
  writers — matching the upsert path.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:06:01 -04:00
02e62767a1 feat(watchsync): sync watchlists with Trakt/Simkl/MDBList (#227)
* feat(watchsync): sync watchlists with Trakt/Simkl/MDBList

Extend the watch-providers feature to sync a user's watchlist, generalizing
the existing favorites pipeline rather than duplicating it.

What changed
- Generalize the favorites sync into one ListKind-parameterized pipeline
  (internal/watchsync/lists.go) driving both favorites and watchlist; the
  per-favorites service methods are replaced by kind-generic ones. The shadow
  table watch_provider_favorite_items becomes watch_provider_list_items with a
  list_kind discriminator.
- Providers: Trakt gains watchlist sync (/sync/watchlist, distinct from
  favorites); Simkl gains plan-to-watch sync; MDBList is re-mapped from
  favorites to watchlist (its only list is a watchlist) — its capabilities now
  report import_favorites=false / import_watchlist=true, and the migration
  re-binds existing MDBList connections.
- Auto-remove watched items from the watchlist: a standalone, default-on
  profile preference (user_profiles.remove_watched_from_watchlist) removes a
  movie when watched and a series once every episode is watched. Implemented as
  watchstate.CompletionObserver (internal/watchlist.Maintainer), wired into the
  manual mark-watched, playback-stop, and jellycompat mark-played paths.
- Optional MDBList sort-order mirroring: an opt-in, capability-gated toggle
  mirrors MDBList's watchlist order into Silo via user_watchlist.sort_index;
  ListWatchlist orders by sort_index then added_at, so both /api/v1/watchlist
  and the catalog watchlist view inherit it.
- Real-time + scheduled: local add/remove pushes to connected providers
  immediately (removals gated by the opt-in removals toggle); the hourly job is
  the inbound/import + retry/reconcile path.
- Web: watch-provider settings gain watchlist import/export/removals and
  "mirror watchlist order" toggles plus watchlist sync stats.

Why
- The favorites and watchlist pipelines are ~90% identical; generalizing keeps
  one code path (per CLAUDE.md's anti-duplication guidance) instead of cloning.

API/compat
- All new fields on ConnectionStatus/Capabilities/ConnectionUpdate/SyncRun and
  the web types are additive (Silo v1 additive-only rule). No existing field is
  renamed, removed, or retyped.

Risks / follow-up
- MDBList capability flip is intentional and client-visible: silo-android /
  silo-apple may need to surface MDBList under the watchlist (not favorites) UI.
- MDBList existing users: their MDBList list previously mirrored Silo favorites
  and now mirrors Silo watchlist; the first post-migration sync is a union
  (removals default off), so nothing is destructively purged.
- Order mirroring reflects the order MDBList returns from /watchlist/items
  (couldn't confirm against their docs — Cloudflare-blocked); if it ever
  diverges from the UI sort, a sort param is the small follow-up.

Tests: new maintainer (auto-remove) and watchlist-order unit tests; provider +
service tests updated. go build, go test (affected pkgs), migrate-validate,
verify-local-paths, web prettier/eslint/tsc all pass.

AI-use disclosure: implemented with Claude Code (Claude Opus 4.8).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(watchsync): update list shadow table references

* fix(watchsync): address review — retry/progress + error propagation

Addresses CodeRabbit review on #227:
- maintainer: propagate transient catalog lookup errors instead of silently
  treating every items.GetByID failure as "maybe an episode".
- exportList: mark every queued item not confirmed sent (not_found, failed, or
  omitted) so the pending loop always advances; the next run's upsert clears the
  error and re-attempts, so transient failures still retry.
- removePendingListItems + realtime removal: treat Sent and NotFound as
  reconciled; leave true failures pending (no last_error, which would strand
  them from the removal query) so the scheduled run retries, using in-memory
  dedupe to terminate the loop.
- exportLocalListItems: send the normalized items (with computed
  ProviderItemKey), not the original event slice.
- UpdateConnection: clear mirrored watchlist order before persisting the disable
  and propagate failures, so a failed clear can't report "disabled" while
  sort_index ordering is still active.
- web: include favorite + watchlist removal counts in the exported "sent" total.
- test: align serviceFakeRepo list-state with Postgres (clear last_error on
  successful transitions); add maintainer error-propagation test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:05:53 -04:00
b4a66b29e2 fix(web): preserve untouched metadata levels when toggling a provider (#226)
toggleLevelProvider merged the toggled level into the levelChains state
object, which starts empty. After the first toggle the state collapsed to
a single level, so once chainDirty flipped true activeLevelChains dropped
every untouched level and the dialog rendered them empty.

Seed from the full resolved activeLevelChains map and write it back the
same way reorderLevel already does, so disabling a source on one TV level
no longer blanks the others on the first toggle.

The earlier redesign (b9a9613) only patched the per-level source fallback,
which kept the toggled level's items but did not preserve the rest of the
map; builds containing it still reproduced the issue.

Part of #213

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 12:46:40 -04:00
6854eac80e fix(web/deps): bump vulnerable transitive deps via pnpm overrides (#225)
Resolves 8 open Dependabot alerts in web/pnpm-lock.yaml (all dev-only
transitive dependencies) by pinning patched versions through the existing
pnpm.overrides mechanism:

- undici  7.24.7 -> 7.28.0  (via jsdom; alerts #43-#49, incl. 2 high:
  TLS cert validation bypass & cross-origin routing via SOCKS5 ProxyAgent)
- js-yaml 4.1.1  -> 4.2.0   (via @eslint/eslintrc; alert #39, quadratic DoS)
- @babel/core 7.29.0 -> 7.29.6 (via @vitejs/plugin-react; alert #38,
  arbitrary file read via sourceMappingURL)

No vulnerable versions remain in the lockfile. Web build and lint pass;
the only failing tests (ServerStorageStep ResizeObserver) are pre-existing
on main and unrelated to these bumps.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 12:37:43 -04:00
CoffeeKnyteandGitHub 6414061022 fix(collections): correct multi-library sync — match all bound libraries and dedupe items (#184)
* fix(collections): sync matches across all bound libraries

Collection sync resolved item membership against only the legacy
library_collections.library_id, so a collection bound to multiple
libraries (N:N library_collection_libraries) had its items collapsed to a
single library on every sync.

Switch the MDBList/TMDB/Trakt sync matchers to the existing
LibraryItemRepository.GetItemsInFolders over collection.LibraryIDs.
GetByID already populates LibraryIDs from the N:N table (falling back to
[LibraryID]), so single-library collections are unaffected. No schema
change; listing and the catalog open path were already N:N-aware.

Unblocks merging duplicated per-library collections into single
multi-library collections.

* fix(collections): dedupe matched items by media_item_id before ReplaceItems

A collection sync resolves source entries (external IDs) to library
content_ids and appends one item per matched entry without deduping by
media_item_id. When two different source entries resolve to the same
library item, the same content_id is appended twice; ReplaceItems then
hits the library_collection_items composite PK (collection_id,
media_item_id) with a duplicate-key violation, failing the whole sync
with an HTTP 500 and zero items.

The multi-library union added in the previous commit (GetItemsInFolders
over collection.LibraryIDs) widens the matched set and so raises the
collision probability, making this latent bug more likely to fire.

Dedupe at the single choke point in ReplaceItems (first occurrence wins,
preserving rank order) so all five sync paths are covered at once, and
renumber position over the surviving rows so it stays dense.

Also log the underlying error in HandleSyncAdminCollection, which
previously mapped every non-sentinel sync error to a generic 500 with
no log line, making this failure invisible on the server.
2026-06-26 12:28:14 -04:00
a3aa93534d fix(jellycompat): authenticate Android TV token-less direct play (#200)
* fix(jellycompat): authenticate Android TV token-less direct play

Stock Jellyfin Android TV ignores the api_key-bearing DirectStreamUrl
returned from PlaybackInfo and builds its own direct-play URL with no
auth header, no api_key/ApiKey query param, and no PlaySessionId. The
media request arrives via the player's HTTP stack (okhttp) with
auth_kind=none, so PlaybackSessionAuth 401s it — the client retries,
falls back to a transcode that stalls, and surfaces "player error".

Add a third fallback in PlaybackSessionAuth, scoped strictly to the
direct-play video stream routes (/Videos/{id}/stream[.{container}]) via
the chi route pattern so /Items/{id}/Download stays protected: anchor
auth on the PlaybackSession negotiated during the preceding (already
authenticated) PlaybackInfo, looked up by mediaSourceId when present
(else the route item id), and resolve its CompatToken.

Covered by tests: token-less direct play with a matching session
succeeds, no matching session 401s, and Download is not loosened.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jellycompat): require session item match + expand direct-play tests

Address CodeRabbit review on PR #200:
- Require the matched PlaybackSession's RouteItemID to equal the requested
  route item before authorizing, so a mediaSourceId cannot authorize a
  stream for a different item.
- Seed the compat session in the 401 tests so they fail on route/session
  scoping rather than a missing session.
- Table-drive the positive test across /Videos/{id}/stream and
  /Videos/{id}/stream.{container}, plus the route-item lookup branch when
  mediaSourceId is absent; add a cross-item denial test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 12:27:37 -04:00
QuickandGitHub d83f7fadef fix(audiobooks): mirror ABS playback into live sessions (#203) 2026-06-26 12:27:12 -04:00
185a90b048 fix(jellycompat): honor client subtitle selection in PlaybackInfo (#222)
* fix(jellycompat): honor client subtitle selection in PlaybackInfo

Jellyfin clients send the chosen subtitle as SubtitleStreamIndex on
PlaybackInfo, but the compat layer only ever advertised the media's
embedded default subtitle (or none). Picking a subtitle before playback
had no effect and the video started without it — and an explicit
"subtitles off" was ignored when the media had a default subtitle.

Mirror the existing audio-selection plumbing for subtitles:
- parse SubtitleStreamIndex from the PlaybackInfo body and query
- persist it as SelectedSubtitleStreamIndex on the play-session source
- advertise it via DefaultSubtitleStreamIndex and flip IsDefault onto the
  chosen stream (embedded, external, or Silo-downloaded)
- honor a negative index as an explicit "subtitles off"

The selection is validated against streamable tracks (bitmap subs that
require burn-in are excluded, matching delivery) and the downloaded
subtitle range, falling back to the media default when invalid.

Out of scope (follow-ups): bitmap-subtitle burn-in transcode wiring and
mid-playback subtitle changes on progress reports.

Part of #217

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(jellycompat): handle downloaded-subtitle lookup failure in selection

Don't let a transient ListDownloadedSubtitles error masquerade as "no
downloaded subtitles", which silently downgraded a valid subtitle
selection to the media default. The lookup error is now logged, and
resolution honors a requested index it cannot validate (it may be a
downloaded subtitle) instead of discarding the user's choice, while
embedded/external selections continue to resolve normally.

Addresses CodeRabbit review feedback on #222.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 12:26:27 -04:00
QuickandGitHub 78d4f9acbf Merge pull request #223 from Silo-Server/codex/silodiscordtriage-httpsdiscordappcomchannels148783
Fix semantic-disabled Meilisearch settings
2026-06-26 12:12:31 -04:00
Quick 7f67bb80ad Fix semantic-disabled Meilisearch settings
- Omit Meilisearch embedders unless semantic search is enabled
- Default semantic ratio setting to 0.50
2026-06-26 12:09:45 -04:00
QuickandGitHub b21c3eea37 Merge pull request #221 from Silo-Server/codex/meilisearch-short-title-search
fix(search): tighten short meilisearch title queries
2026-06-26 10:53:11 -04:00
Quick f57c95e813 fix(search): update default semantic ratio for meilisearch settings 2026-06-26 10:47:02 -04:00
Quick a181586139 fix(search): keep meilisearch active during pending sync 2026-06-26 10:46:42 -04:00
Quick dd2784c0ee fix(search): tighten short meilisearch title queries 2026-06-26 10:13:15 -04:00
QuickandGitHub 4473f8c60b Merge pull request #220 from Silo-Server/codex/search-provider-interface
feat(search): add provider interface with initial Meilisearch support
2026-06-26 09:32:53 -04:00
Quick 125e94b429 fix(search): allow zero meilisearch task uid 2026-06-26 09:29:40 -04:00
QuickandClaude Opus 4.8 2a943550d4 fix(search): fold semantic toggle into catalog index schema version
catalogSearchMeilisearchSchemaVersion hashed the embedder, canonical dimensions, and index types but not SemanticEnabled. Because attachDocumentVectors omits _vectors when semantic is off, toggling semantic on without a full rebuild left previously indexed (and incrementally synced) documents without vectors while the Postgres coverage gate still reported the type "ready" — silently degrading hybrid ranking until a manual rebuild.

Folding SemanticEnabled into the schema version makes a toggle diverge from the stored version, so the provider falls back to keyword, sync skips, the admin status surfaces the schema mismatch, and a rebuild is required to restore hybrid — consistent with how embedder and index-type changes already behave.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:43:09 -04:00
QuickandClaude Opus 4.8 6b45cfd855 fix(scanner): track podcast show audio paths during folder reconcile
Extract listPodcastShowAudioFiles and record each show's audio paths in
seenPaths during ScanPodcastFolder so reconciliation does not treat
still-present episodes as removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:34:18 -04:00
QuickandClaude Opus 4.8 ed6c084c68 feat(search): gate index events by active provider and harden rebuild reconcile
Completes the search-provider-interface wiring that the catalog hardening
commits already call into:

- Skip the transactional search-index-event write path when Meilisearch is
  not the active provider (ItemRepository.WithActiveSearchProvider /
  SearchIndexEventRepository.disabledByActiveProvider).
- Dead-letter catalog_search_index_events after 10 attempts instead of
  retrying forever.
- Track the rebuild high-water mark (MaxEventID / MarkProcessedThrough) and
  persist last_processed_event_id in UpdateStateAfterRebuild so a rebuild
  reconciles events enqueued during the rebuild.
- Validate (read-only) the embedding lock when embedding a search query
  instead of establishing/mutating it.
- Surface total_exact on the legacy /items browse response.
- Wire the active catalog search provider into the scanner and item repo at
  startup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:34:04 -04:00
QuickandClaude Opus 4.8 91b5105f3c docs(search): add hybrid semantic search hardening plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:24:14 -04:00
QuickandClaude Opus 4.8 7080491f89 feat(web): surface semantic coverage and embedder capability in search admin
Render the new `semantic` block from the admin catalog-search status
endpoint in the Search Settings page: semantic readiness (with
disabled reason), overall vector coverage %, coverage-updated
timestamp, embedder capability, and a compact per-type coverage list.
Reuses the existing StatusRow/FieldGroup/Badge patterns and adds a
formatPercent helper; keeps the existing Vectorized Documents row.

The TS `semantic?` field is optional and the whole block is guarded so
a new frontend talking to an older backend during rollout degrades
gracefully.

Part of the hybrid-semantic-search hardening work (Task 8).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:10:33 -04:00
QuickandClaude Opus 4.8 13ac4b753c perf(recommendations): split embedding backfill into cheap and text-stale passes
EmbedAll previously ran the full text-staleness CTE on every page, paying
five item_people LATERAL joins per eligible row just to detect whether an
item's canonical text had drifted - even when the real work was embedding
brand-new (missing) items during active backfill.

Restructure EmbedAll into two passes:

- Pass 1 (cheap): drain missing/model-stale items via the repurposed
  ItemsNeedingEmbedding query (single LEFT JOIN, no LATERAL), paged by a
  content_id cursor so a failed/skipped item is retried next run instead of
  stalling the page.
- Pass 2 (expensive): only once Pass 1 fully drains, run one bounded
  ListEmbeddingTextCandidates scan (LIMIT embeddingTextStaleQuotaPerRun=200)
  to re-embed text-drifted items. Re-embedding refreshes canonical_text, so
  handled rows drop out next run - no Pass 2 cursor needed.

Coverage-first tradeoff (documented in EmbedAll): under steady state Pass 1
drains every run so text-stale items stay fresh; only under pathological
continuous heavy ingest does Pass 2 get skipped, deliberately prioritizing
covering new items over re-embedding changed ones.

Supporting changes:
- ItemsNeedingEmbedding gains an afterID cursor + ORDER BY; SQL extracted to
  buildItemsNeedingEmbeddingSQL for a cheap-shape unit test (no item_people /
  LATERAL).
- Add an `embedder` interface seam (Engine.embClient) so EmbedAll is testable
  with a fake; *embeddings.Client still satisfies it.
- Extract embedBatch to DRY both passes, preserving quota/billing early-return,
  single-item fallback, ensureEmbeddingLock-before-upsert, and skip-on-store-error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:02:20 -04:00
QuickandClaude Opus 4.8 a5ef464d95 feat(search): validate embedder capability and expose semantic status
Add a rate-limited Meilisearch embedder capability check and surface a
semantic-coverage block in the admin search status.

Capability validation confirms the active index declares the configured
embedder with source="userProvided" and dimensions matching the canonical
embedding dimension, then runs a unit-vector hybrid probe to confirm the
index accepts the request shape. The whole path (GetSettings + evaluate +
probe) is cached for 5m and NEVER trips the circuit, marks a fallback, or
modifies unhealthyUntil, so a misconfigured embedder cannot take keyword
search down. The pure evaluator (evaluateEmbedderSettings) is unit-tested
across missing/wrong-source/wrong-dimensions/ok with distinct reasons.

The admin status gains a Semantic block read entirely from the in-memory
coverage snapshot (Snapshot + CoverageReady) with no new DB query in
Status(); document_count/vector_document_count are unchanged (additive
only). buildSemanticStatus is pure and unit-tested (nil snapshot and a
populated, deterministically-sorted snapshot).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:43:21 -04:00
QuickandClaude Opus 4.8 dadd6f4769 feat(search): surface search diagnostics on catalog responses
Add per-query search observability to /api/v1/catalog. The internal
CatalogSearchResult now carries Mode + SemanticUsed alongside the existing
Provider + FallbackReason; all four are plumbed through CatalogResult (which
previously dropped them in resolveDirectSearchSource) and exposed as an
additive, omitempty search_diagnostics object on the catalog response.

Two correctness points:
- For Meilisearch, Mode/SemanticUsed are derived from the POST-downgrade
  request (baseSearchReq.Hybrid != nil) at the end of searchMeilisearch, so a
  hybrid->keyword self-downgrade reports mode="keyword", semantic_used=false.
- search_diagnostics is emitted iff a provider search actually ran
  (result.Provider != ""), which naturally omits browse, preview,
  non-relevance-sort q=, and group=work paths (none set Provider).

API additive-only: existing response fields keep declaration order and stay
byte-stable; only the new omitempty field is added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:32:30 -04:00
QuickandClaude Opus 4.8 4cc92c1579 feat(search): gate hybrid on vector coverage readiness
Wire the semantic-coverage building blocks together so the Meilisearch
provider only goes hybrid once the active embedding model covers enough of
the requested item types.

- Add Coverage SemanticCoverageGate to MeilisearchProviderConfig and consult
  it in buildMeilisearchSearchRequest after the wordcount check; when
  not-ready, record "semantic_not_ready: <reason>" as a diagnostic fallback
  and stay keyword-only (shouldUseSemanticSearch's bool contract unchanged).
- Build and own a semanticCoverageTracker in
  NewCatalogSearchServiceFromSettings, but only when semantic is enabled and
  a real pool is present. Derive the model provider via a panic-safe comma-ok
  helper (semanticModelProvider) so a nil or non-implementing vectorizer
  degrades to a not-ready gate instead of asserting a nil interface.
- Add StartCoverageRefresh and call it from NewRouter on deps.AppContext so
  the refresher runs for the process lifetime.

Coverage == nil preserves today's behavior exactly; existing hybrid and
SkipTotal tests stay green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:17:47 -04:00
QuickandClaude Opus 4.8 f40611409d feat(search): semantic coverage tracker with hysteresis gate
Add the in-memory coverage gate the search hot path will consult to
decide whether semantic results may be served for a set of item types.

The tracker reads an atomic.Pointer snapshot with no DB and no lock on
the read path, refreshes single-flight under a mutex on a 2-minute
ticker, and applies per-type hysteresis (enable 0.90 / disable 0.80,
holding the previous latch inside the band) to avoid flapping. It
collapses to not-ready when the active embedding model changes so a
stale latch cannot leak across models, and fails safe: a not-yet-
computed snapshot reports not-ready (never panics), and a model-lookup
or count-query error retains the last-good snapshot instead of
publishing zeros.

Refresh counts through the existing catalogSemanticCoverageByType via an
injectable fetch seam, so the behavior is unit-testable with canned
counts and a fake model provider (no pgx.Rows faking). Reuses the
Task-1 coverageQuerier and extends catalogTypeCoverage with Ratio/Ready,
filled by the pure computeCoverageSnapshot helper. Adds the
SemanticCoverageGate interface beside the other provider interfaces;
Task 4 wires the gate and starts the refresher.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:06:51 -04:00
QuickandClaude Opus 4.8 e21c35cb7f feat(search): expose active embedding model from lock
Add recommendations.Engine.ActiveEmbeddingModel, which reads the
embedding lock from server_settings and returns the locked model (or ""
when no lock is established). Define the catalog
CatalogSemanticModelProvider interface so a later task can scope vector
coverage checks to the active model; the recommendations Engine already
satisfies it. No wiring yet (Task 4 injects recEngine as this provider).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 22:51:36 -04:00
QuickandClaude Opus 4.8 927d7764a4 feat(search): model-filtered embed-eligible coverage counts
Establish how semantic vector "coverage" is counted: current-model
embeddings over embed-eligible items, per media type, from Postgres.

- Move the embed-eligibility predicate to a single source of truth in
  embeddingvectors.ItemEligibilityWhereClause; recommendations now
  delegates to it (output byte-identical).
- Add catalogSemanticCoverageByType (per-type eligible/vectorized) and
  define the coverageQuerier interface in catalog. Both numerator and
  denominator apply the eligibility predicate, so vectorized never
  exceeds eligible (C1 guarantee) even with a stale embedding on a
  now-unmatched item.
- countCatalogSearchVectorDocuments now takes a coverageQuerier and a
  model, applies the eligibility + model filter, and is the per-type
  numerator summed across types. Update all four callers (pass "").
- Add idx_media_item_embeddings_model (CONCURRENTLY, self-healing
  guard) so the model filter does not scan the embeddings table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 22:43:23 -04:00
Quick 20f36fdc31 fix(search): address provider review comments 2026-06-25 21:31:20 -04:00
Quick 5089e17ee6 fix(scanner): reconcile podcast rescans 2026-06-25 20:57:28 -04:00
Quick 07f2dd5a8f fix(admin): bound autoscan activity backlog 2026-06-25 20:50:45 -04:00
Quick 40329f616d perf(search): speed up catalog query results 2026-06-25 20:46:08 -04:00
Quick fb3a21c332 fix(search): batch orphan delete index events 2026-06-25 17:26:19 -04:00
Quick 86bc69c0a2 fix(search): defer meilisearch key check to runtime 2026-06-25 17:24:08 -04:00
Quick 6df388b25b fix(search): harden search index review paths 2026-06-25 17:00:45 -04:00
Quick 6ca427096b Add catalog search provider support 2026-06-25 16:20:14 -04:00
QuickandGitHub 89c435f046 Merge pull request #219 from Silo-Server/codex/fix-image-resolver-build
fix(plugins): restore image resolver startup build
2026-06-25 15:09:18 -04:00
Quick af52ea7bcc fix(plugins): use scoped installation store for image resolver 2026-06-25 15:08:55 -04:00
QuickandGitHub 13c0dde538 Merge pull request #218 from Silo-Server/codex/image-resolver-capability
feat(plugins): add image resolver registry
2026-06-25 15:01:07 -04:00
Quick 2045b7a0b2 feat(plugins): add image resolver registry 2026-06-25 14:47:48 -04:00
Quick 5c0027b54b fix(metadata): dedupe availability during re-id 2026-06-25 13:29:30 -04:00
87159b0a38 feat(collections): add profile-scoped display filters (#191)
* feat(collections): add profile-scoped display filters

* refactor(collections): dedup display-filter helpers per review

Address code-review feedback on the profile-scoped display filters
without changing behavior:

- Widen CompletedHistoryItemMap to accept ProgressCompletionStore and
  drop the duplicate completedHistoryItemMapForProgress copy.
- Extract the duplicated MDBList candidate retry loop into a generic
  collectionutil.FetchMDBListWithFallback helper, used by both the user
  and library collection syncers, and cover it with unit tests.
- Reuse validateOptionalLibraryIDs in HandleUpdateCollection instead of
  an inline positive-ID loop.
- Import the shared COLLECTION_{WATCH,MEDIA}_FILTER_OPTIONS in the
  template config form rather than redefining them locally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(collections): sanitize query_definition library_ids fallback

readSourceConfigLibraryIDs validated source_config.library_ids (finite,
positive, truncated, deduplicated) but returned the query_definition
fallback raw, so legacy rows could surface zero/negative/duplicate IDs
that the backend now rejects on save. Extract a shared sanitizer and
apply it to both paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(docs): This makes the agents annoying to work with

* Improve playback session handling

* Support collection source order in catalog filters

* fix(collections): address display filter review feedback

* refactor(catalog): remove duplicate collection query params

* Hide episode media scope for collection overlays

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 11:03:38 -04:00
Quick 227986094b Propagate playback client metadata through session sync 2026-06-23 14:26:14 -04:00