Files
silo-server/internal/audiobooks/abs/items_handler.go
T
9b111649f3 perf+fix(audiobooks): detail page, browse, sessions & scanner (#169)
* perf(catalog): fix audiobook detail N+1 + slow people facets

Audiobook detail pages were slow in proportion to track count (up to 433
files/book). Root causes, found by EXPLAIN ANALYZE on the live DB:

- effectiveAudioSelection ran 3-4 user-store queries (profile, audio pref,
  library pref) per file inside buildPlaybackInfo's loop, though the results
  are invariant across a request. Introduce a request-scoped audioPrefResolver
  that memoizes the store lookups (library prefs keyed by folder); a 400-file
  audiobook now issues each query once instead of per file. Selection logic is
  unchanged (audioPreference returns a copy so the original-language sentinel
  is still resolved per file).
- buildAudiobookExtension ran its four independent related-content queries
  serially; run them concurrently so latency is the slowest, not the sum.
- author/narrator browse facets did a full people-table scan; add a
  (kind, content_id, person_id) index so the facet resolves from an index-only
  scan of just that kind's credits (~112ms -> ~49ms on the live library).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(catalog): cache audiobook author/narrator group browse

The Authors/Narrators audiobook pages were slow on cold load and slow again
after a hard refresh (fast only while the React Query client cache was warm).

Root cause (EXPLAIN ANALYZE on live, 31K-audiobook library): the grouped
browse query is ~234ms/page, there are ~13K distinct authors, and the client
pages through the entire list on every load (sequential 500-row requests). With
no server-side cache each of the ~20 pages re-ran the full aggregation
(COUNT(*) OVER() forces it), so a cold load was ~20x234ms. The client's 60s
staleTime was the only thing making a warm revisit fast; a refresh wiped it.

Fix: AudiobookGroupsCache caches the full sorted group list per (library,
group_by, sort, viewer) for 60s (matching the client staleTime, so no extra
staleness) and serves every page as an in-memory slice — one aggregation per
window instead of one per page, and a refresh is a cache hit. Also raise the
client page size 500->2000 so fewer sequential round-trips are needed now that
a larger page is a cheap slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(settings): throttle per-request device last_seen upserts

Device-setting reads (HandleGetDeviceSetting, HandleGetEffectiveSettings,
HandleGetEffectiveSubtitleAppearance) each registered the request's device — an
INSERT ... ON CONFLICT upsert of last_seen_at on a single per-device row. A page
that fetches many settings fired hundreds of these concurrently; they serialized
on that row's lock (observed 100-237ms each, ~250 per page load in the slow
query log), taxing every settings fetch.

Throttle device registration to one upsert per (profile, device) per 5 minutes
via an in-process TTL cache, marking the device seen before the upsert so a
concurrent burst collapses to a single write. last_seen_at stays fresh to within
the window. Reads no longer issue a contended write on the hot path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf+fix(audiobooks): probe-repair, resume position, cache storm, groups reveal, hot-row + stats index

From the full audiobook code review (EXPLAIN + slow-query trace on live):

- #1 (P0, detail-page killer): NeedsCriticalProbeRepair required video codec/
  resolution/tracks, which audio-only files never have, so PlaybackProbeEnsurer
  re-ran ffprobe per file on every detail/watch load (up to N serial spawns for
  an N-track book) and never converged. Gate video-field checks on the file
  actually having a video stream. TDD.
- #3 (P0): abs session-sync rewound the resume cursor — UpdateProgressPosition
  did an unconditional SET with no monotonic guard, ignored its error, and
  no-op'd when no row existed (first-listen resume lost). Now a finish-preserving
  GREATEST upsert; caller logs failures.
- #4 (P0 perf): progress reports fired every ~10s invalidated all of
  catalogKeys.all → refetched every active browse/detail query incl the 13k
  audiobook group lists. Scope invalidation to the reported item's detail.
- #6 (P1 perf): Authors/Narrators page rendered all ~13k groups + cover images
  at once (main-thread freeze). Incremental reveal: render a capped window, grow
  on scroll via IntersectionObserver.
- #10: throttle abs TouchToken last_seen upsert (one per token per 5min) — same
  hot-row contention class as the device fix.
- #8: index abs_playback_sessions (user_id, profile_id, started_at) for the
  listening-stats aggregations.

Deferred (need contract/validation): listening-time idempotency (client delta-vs-
cumulative), scanner deleted-file reconcile, abs session retention job, abs list-
handler batch fetch, scanner-output P2s (need re-backfill).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(audiobooks): batch-fetch abs list/shelf handlers (kill N+1)

handleSimilarItems, handleItemsInProgress, and handleGetMyProgress called
MediaStore.GetAudiobookByID once per row — up to ~500 single fetches (each a
few queries) on app open. Add GetAudiobooksByIDs (one access-scoped fetch +
people/series hydrated once for the whole set) and look results up from the
returned map, preserving order. Underlying primitives (GetByIDsWithAccess,
hydratePeople/Series) were already batch-capable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(audiobooks): reconcile deleted files on scan + prune session history

#5: ScanAudiobookFolder only ever upserted — deleted/renamed books leaked
media_items/media_files/memberships forever. Mirror the ebook reconcile: collect
seenPaths during the walk, MarkMissing files no longer on disk, then
reconcileLibraryMemberships. Safety mirrors ebooks/video: an inaccessible root
(unmounted source) is skipped entirely, and a walk that saw zero files while the
DB has rows only reconciles after operator cleanup confirmation
(ebookEmptyCleanupAllowed) — so a flapping mount can't wipe the catalog. Soft
mark only; the existing grace-period purge hard-deletes later. Reconcile runs
only on a fully-completed (non-cancelled) scan. (#9 coarse case already handled:
audiobookFolderShouldSkip skips unchanged folders; per-file reuse deferred.)

#8-retention: abs_playback_sessions grew unbounded (one row per play-start, never
deleted) and fed every listening-stats scan. Add an hourly sweep in
SessionCleaner: close abandoned open sessions (no /close, stopped syncing >24h)
and delete closed sessions older than 90 days. Mirrors the recommendation_cache /
missing-files prune pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(audiobooks): address max-effort code-review findings

From /code-review max on the pre-PR diff:
- DATA RACE (P0): SessionCleaner.lastABSSessionPrune is read+written by both the
  15s ticker goroutine and the shutdown-path CleanStale call (main.go defers
  Stop() to after that call). Guard the prune-due gate with a mutex. (CleanStale
  was stateless before this branch, so concurrent calls were previously safe.)
- ScanAudiobookFolder hardcoded fullScan=true into the empty-walk cleanup guard,
  but it's also called from ScanSubtree (incremental scans). An empty subtree
  scan would wrongly consume the operator's one-shot empty-cleanup allowance and
  warn. Thread a real fullScan flag (true from ScanFolder, false from the two
  subtree call sites), mirroring the ebook path.
- Revert UpdateProgressPosition to UPDATE-only (drop the INSERT-on-missing):
  keep the monotonic GREATEST + finish guard that fixes the resume rewind, but
  restore the no-op-on-missing contract so a stray sync tick can't resurrect
  just-cleared progress or create a zero-duration continue-listening row.
- Clamp the audiobook-groups handler limit (paging moved into the cache, leaving
  the old 500/page bound stranded); also gofmt the Scanner struct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(audiobooks): address review feedback for scanner and stats

* fix(audiobooks): address review feedback

* fix(audiobooks): retry failed session prune

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-06-17 09:51:43 -04:00

199 lines
6.1 KiB
Go

package abs
import (
"context"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/Silo-Server/silo-server/internal/catalog"
)
// resolveDefaultLibrary returns the first audiobook library (the canonical
// "default" for response embedding) or a virtual fallback when the store
// is empty or errors. Centralizes the snippet that handleItem,
// handleSimilarItems, and handleItemsInProgress all need so the fallback
// shape stays consistent across all three response paths.
func (h *Handler) resolveDefaultLibrary(ctx context.Context, filters ...catalog.AccessFilter) AudiobookLibrary {
access := emptyAccessFilter()
if len(filters) > 0 {
access = filters[0]
}
if libs, err := h.deps.MediaStore.ListAudiobookLibraries(ctx, access); err == nil && len(libs) > 0 {
return libs[0]
}
return AudiobookLibrary{ID: 0, Name: VirtualLibraryName, Type: "audiobooks"}
}
// handleItem — GET /abs/api/items/{id} (and /api/items/{id})
//
// Returns the full ABS LibraryItem with audio track details for the given
// audiobook. The ABS mobile app fetches this when the user opens the
// item-detail page; it reads media.tracks.length to decide whether to render
// the play button and uses the track metadata for offline-download decisions.
func (h *Handler) handleItem(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
contentID := chi.URLParam(r, "id")
if contentID == "" {
http.Error(w, "id required", http.StatusBadRequest)
return
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
item, err := h.deps.MediaStore.GetAudiobookByID(r.Context(), contentID, access)
if err != nil || item == nil {
http.Error(w, "item not found", http.StatusNotFound)
return
}
files, err := h.deps.MediaStore.GetMediaFiles(r.Context(), contentID, access)
if err != nil {
http.Error(w, "load files: "+err.Error(), http.StatusInternalServerError)
return
}
lib := h.resolveDefaultLibrary(r.Context(), access)
baseURL := h.absBaseURL(r)
result := siloItemToLibraryItemDetail(item, files, lib, baseURL)
writeJSON(w, http.StatusOK, result)
}
// handleSimilarItems — GET /abs/api/items/{id}/similar
//
// Returns similar audiobooks in the canonical ABS paged envelope so
// mobile clients can render the "Similar" rail. Sort metadata is
// "relevance" desc to match continuum-plugin-audiobooks; the envelope
// is emitted even when empty so clients that iterate
// `results`/`total` don't crash.
func (h *Handler) handleSimilarItems(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
contentID := chi.URLParam(r, "id")
if contentID == "" {
http.Error(w, "id required", http.StatusBadRequest)
return
}
const limit = 10
emptyEnvelope := pagedEnvelope([]any{}, 0, limit, 0, "relevance", true, "", false, "")
if h.deps.Recommender == nil {
writeJSON(w, http.StatusOK, emptyEnvelope)
return
}
ids, err := h.deps.Recommender.Similar(r.Context(), contentID, limit)
if err != nil || len(ids) == 0 {
writeJSON(w, http.StatusOK, emptyEnvelope)
return
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
lib := h.resolveDefaultLibrary(r.Context(), access)
baseURL := h.absBaseURL(r)
byID, err := h.deps.MediaStore.GetAudiobooksByIDs(r.Context(), ids, access)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
out := make([]LibraryItem, 0, len(ids))
for _, id := range ids { // preserve recommender order
si := byID[id]
if si == nil {
continue
}
out = append(out, siloItemToLibraryItem(si, lib, baseURL))
}
writeJSON(w, http.StatusOK, pagedEnvelope(out, len(out), limit, 0, "relevance", true, "", false, ""))
}
// handleItemsInProgress — GET /abs/api/me/items-in-progress
//
// Returns the Continue Listening shelf. Queries the ProgressStore for in-
// progress rows, then hydrates each with a summary LibraryItem from the
// catalog. Items without a matching catalog entry are skipped silently.
func (h *Handler) handleItemsInProgress(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
if h.deps.ProgressStore == nil {
writeJSON(w, http.StatusOK, map[string]any{"libraryItems": []any{}})
return
}
rows, err := h.deps.ProgressStore.ListProgressForAudiobooks(r.Context(), a.UserID, a.ProfileID, 25)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
lib := h.resolveDefaultLibrary(r.Context(), access)
baseURL := h.absBaseURL(r)
ids := make([]string, 0, len(rows))
for _, p := range rows {
if !p.IsFinished && p.CurrentSeconds > 0 {
ids = append(ids, p.ContentID)
}
}
byID, err := h.deps.MediaStore.GetAudiobooksByIDs(r.Context(), ids, access)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
items := make([]any, 0, len(rows))
for _, p := range rows {
if p.IsFinished || p.CurrentSeconds <= 0 {
continue
}
si := byID[p.ContentID]
if si == nil {
continue
}
li := siloItemToLibraryItem(si, lib, baseURL)
items = append(items, map[string]any{
"id": li.ID,
"libraryId": li.LibraryID,
"folderId": li.FolderID,
"mediaType": li.MediaType,
"media": li.Media,
"numTracks": li.NumTracks,
"addedAt": li.AddedAt,
"updatedAt": li.UpdatedAt,
"userMediaProgress": map[string]any{
"id": a.UserID + "-" + p.ContentID,
"libraryItemId": p.ContentID,
"currentTime": p.CurrentSeconds,
"progress": p.ProgressPct,
"isFinished": p.IsFinished,
"lastUpdate": p.UpdatedAt.UnixMilli(),
},
})
}
writeJSON(w, http.StatusOK, map[string]any{"libraryItems": items})
}