fix(catalog): harden Meilisearch search integration (#291)

Findings from a full review of the Meilisearch implementation:

- Give the indexer its own 2m HTTP timeout instead of reusing the 800ms
  search-path timeout_ms, so large document uploads to a non-loopback
  Meilisearch stop timing out.
- Delete superseded indexes after a rebuild (previous active + leftover
  <prefix>_rebuild_* partials); every rebuild previously leaked a full
  copy of the catalog on the Meilisearch instance.
- Cache the index state row + pending count for 3s on the search hot
  path (was two Postgres round trips per search request); a failed
  search invalidates the cache immediately.
- Swap the active-index pointer before marking outbox events processed
  so a crash between the two replays events instead of losing them.
- End pagination only on a short page; estimatedTotalHits is an
  estimate and could truncate results.
- Latch the startup-resolved provider process-wide so package-level
  enqueue helpers stop querying server_settings in write transactions.
- Surface dead-lettered outbox events in the admin status + web UI.
- Remove unwired provider config knobs, dedupe the manga-chapter
  exclusion predicate, split the vector cache onto its own mutex, real
  rebuild progress percentages, and expand client/coalesce test
  coverage.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Quick
2026-07-05 00:18:14 -04:00
committed by GitHub
co-authored by Claude Fable 5
parent cb2581af4a
commit ce09d2b6fc
14 changed files with 712 additions and 78 deletions
+1
View File
@@ -42,6 +42,7 @@ export interface CatalogSearchStatus {
document_count: number;
vector_document_count: number;
pending_events: number;
dead_lettered_events: number;
last_rebuild_at?: string;
last_sync_at?: string;
last_processed_event_id: number;
@@ -289,6 +289,13 @@ export default function SearchSettings() {
</>
)}
<StatusRow label="Pending Events" value={String(status.index.pending_events)} />
{status.index.dead_lettered_events > 0 && (
<StatusRow
label="Dead-lettered Events"
value={String(status.index.dead_lettered_events)}
badge="stale until rebuild"
/>
)}
<StatusRow
label="Last Sync"
value={formatStatusDate(status.index.last_sync_at) || "Never"}