Files
silo-server/migrations/sql/20260626023809_media_item_embeddings_model_index.sql
QuickandClaude Opus 4.8 927d7764a4 feat(search): model-filtered embed-eligible coverage counts
Establish how semantic vector "coverage" is counted: current-model
embeddings over embed-eligible items, per media type, from Postgres.

- Move the embed-eligibility predicate to a single source of truth in
  embeddingvectors.ItemEligibilityWhereClause; recommendations now
  delegates to it (output byte-identical).
- Add catalogSemanticCoverageByType (per-type eligible/vectorized) and
  define the coverageQuerier interface in catalog. Both numerator and
  denominator apply the eligibility predicate, so vectorized never
  exceeds eligible (C1 guarantee) even with a stale embedding on a
  now-unmatched item.
- countCatalogSearchVectorDocuments now takes a coverageQuerier and a
  model, applies the eligibility + model filter, and is the per-type
  numerator summed across types. Update all four callers (pass "").
- Add idx_media_item_embeddings_model (CONCURRENTLY, self-healing
  guard) so the model filter does not scan the embeddings table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 22:43:23 -04:00

32 lines
1.0 KiB
SQL

-- +goose NO TRANSACTION
-- +goose Up
-- Semantic vector "coverage" counts filter media_item_embeddings by the active
-- embedding model (e.model = $2). Without an index on model that filter scans
-- the whole embeddings table on every coverage/status refresh. A leftover
-- INVALID index can survive a crashed CREATE INDEX CONCURRENTLY, which then
-- blocks the IF NOT EXISTS retry below; drop it first.
-- +goose StatementBegin
DO $$
BEGIN
IF EXISTS (
SELECT 1
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
JOIN pg_index i ON i.indexrelid = c.oid
WHERE n.nspname = 'public'
AND c.relname = 'idx_media_item_embeddings_model'
AND NOT i.indisvalid
) THEN
DROP INDEX public.idx_media_item_embeddings_model;
END IF;
END;
$$;
-- +goose StatementEnd
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_media_item_embeddings_model
ON public.media_item_embeddings (model);
-- +goose Down
DROP INDEX CONCURRENTLY IF EXISTS idx_media_item_embeddings_model;