* perf(catalog): fix audiobook detail N+1 + slow people facets Audiobook detail pages were slow in proportion to track count (up to 433 files/book). Root causes, found by EXPLAIN ANALYZE on the live DB: - effectiveAudioSelection ran 3-4 user-store queries (profile, audio pref, library pref) per file inside buildPlaybackInfo's loop, though the results are invariant across a request. Introduce a request-scoped audioPrefResolver that memoizes the store lookups (library prefs keyed by folder); a 400-file audiobook now issues each query once instead of per file. Selection logic is unchanged (audioPreference returns a copy so the original-language sentinel is still resolved per file). - buildAudiobookExtension ran its four independent related-content queries serially; run them concurrently so latency is the slowest, not the sum. - author/narrator browse facets did a full people-table scan; add a (kind, content_id, person_id) index so the facet resolves from an index-only scan of just that kind's credits (~112ms -> ~49ms on the live library). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(catalog): cache audiobook author/narrator group browse The Authors/Narrators audiobook pages were slow on cold load and slow again after a hard refresh (fast only while the React Query client cache was warm). Root cause (EXPLAIN ANALYZE on live, 31K-audiobook library): the grouped browse query is ~234ms/page, there are ~13K distinct authors, and the client pages through the entire list on every load (sequential 500-row requests). With no server-side cache each of the ~20 pages re-ran the full aggregation (COUNT(*) OVER() forces it), so a cold load was ~20x234ms. The client's 60s staleTime was the only thing making a warm revisit fast; a refresh wiped it. Fix: AudiobookGroupsCache caches the full sorted group list per (library, group_by, sort, viewer) for 60s (matching the client staleTime, so no extra staleness) and serves every page as an in-memory slice — one aggregation per window instead of one per page, and a refresh is a cache hit. Also raise the client page size 500->2000 so fewer sequential round-trips are needed now that a larger page is a cheap slice. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(settings): throttle per-request device last_seen upserts Device-setting reads (HandleGetDeviceSetting, HandleGetEffectiveSettings, HandleGetEffectiveSubtitleAppearance) each registered the request's device — an INSERT ... ON CONFLICT upsert of last_seen_at on a single per-device row. A page that fetches many settings fired hundreds of these concurrently; they serialized on that row's lock (observed 100-237ms each, ~250 per page load in the slow query log), taxing every settings fetch. Throttle device registration to one upsert per (profile, device) per 5 minutes via an in-process TTL cache, marking the device seen before the upsert so a concurrent burst collapses to a single write. last_seen_at stays fresh to within the window. Reads no longer issue a contended write on the hot path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf+fix(audiobooks): probe-repair, resume position, cache storm, groups reveal, hot-row + stats index From the full audiobook code review (EXPLAIN + slow-query trace on live): - #1 (P0, detail-page killer): NeedsCriticalProbeRepair required video codec/ resolution/tracks, which audio-only files never have, so PlaybackProbeEnsurer re-ran ffprobe per file on every detail/watch load (up to N serial spawns for an N-track book) and never converged. Gate video-field checks on the file actually having a video stream. TDD. - #3 (P0): abs session-sync rewound the resume cursor — UpdateProgressPosition did an unconditional SET with no monotonic guard, ignored its error, and no-op'd when no row existed (first-listen resume lost). Now a finish-preserving GREATEST upsert; caller logs failures. - #4 (P0 perf): progress reports fired every ~10s invalidated all of catalogKeys.all → refetched every active browse/detail query incl the 13k audiobook group lists. Scope invalidation to the reported item's detail. - #6 (P1 perf): Authors/Narrators page rendered all ~13k groups + cover images at once (main-thread freeze). Incremental reveal: render a capped window, grow on scroll via IntersectionObserver. - #10: throttle abs TouchToken last_seen upsert (one per token per 5min) — same hot-row contention class as the device fix. - #8: index abs_playback_sessions (user_id, profile_id, started_at) for the listening-stats aggregations. Deferred (need contract/validation): listening-time idempotency (client delta-vs- cumulative), scanner deleted-file reconcile, abs session retention job, abs list- handler batch fetch, scanner-output P2s (need re-backfill). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(audiobooks): batch-fetch abs list/shelf handlers (kill N+1) handleSimilarItems, handleItemsInProgress, and handleGetMyProgress called MediaStore.GetAudiobookByID once per row — up to ~500 single fetches (each a few queries) on app open. Add GetAudiobooksByIDs (one access-scoped fetch + people/series hydrated once for the whole set) and look results up from the returned map, preserving order. Underlying primitives (GetByIDsWithAccess, hydratePeople/Series) were already batch-capable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): reconcile deleted files on scan + prune session history #5: ScanAudiobookFolder only ever upserted — deleted/renamed books leaked media_items/media_files/memberships forever. Mirror the ebook reconcile: collect seenPaths during the walk, MarkMissing files no longer on disk, then reconcileLibraryMemberships. Safety mirrors ebooks/video: an inaccessible root (unmounted source) is skipped entirely, and a walk that saw zero files while the DB has rows only reconciles after operator cleanup confirmation (ebookEmptyCleanupAllowed) — so a flapping mount can't wipe the catalog. Soft mark only; the existing grace-period purge hard-deletes later. Reconcile runs only on a fully-completed (non-cancelled) scan. (#9 coarse case already handled: audiobookFolderShouldSkip skips unchanged folders; per-file reuse deferred.) #8-retention: abs_playback_sessions grew unbounded (one row per play-start, never deleted) and fed every listening-stats scan. Add an hourly sweep in SessionCleaner: close abandoned open sessions (no /close, stopped syncing >24h) and delete closed sessions older than 90 days. Mirrors the recommendation_cache / missing-files prune pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): address max-effort code-review findings From /code-review max on the pre-PR diff: - DATA RACE (P0): SessionCleaner.lastABSSessionPrune is read+written by both the 15s ticker goroutine and the shutdown-path CleanStale call (main.go defers Stop() to after that call). Guard the prune-due gate with a mutex. (CleanStale was stateless before this branch, so concurrent calls were previously safe.) - ScanAudiobookFolder hardcoded fullScan=true into the empty-walk cleanup guard, but it's also called from ScanSubtree (incremental scans). An empty subtree scan would wrongly consume the operator's one-shot empty-cleanup allowance and warn. Thread a real fullScan flag (true from ScanFolder, false from the two subtree call sites), mirroring the ebook path. - Revert UpdateProgressPosition to UPDATE-only (drop the INSERT-on-missing): keep the monotonic GREATEST + finish guard that fixes the resume rewind, but restore the no-op-on-missing contract so a stray sync tick can't resurrect just-cleared progress or create a zero-duration continue-listening row. - Clamp the audiobook-groups handler limit (paging moved into the cache, leaving the old 500/page bound stranded); also gofmt the Scanner struct. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): address review feedback for scanner and stats * fix(audiobooks): address review feedback * fix(audiobooks): retry failed session prune --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
236 lines
8.2 KiB
Go
236 lines
8.2 KiB
Go
package audiobooks
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"strconv"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
|
|
"github.com/Silo-Server/silo-server/internal/audiobooks/abs"
|
|
)
|
|
|
|
// ABSProgressStore implements abs.ProgressStore directly against the
|
|
// user_watch_progress table using a shared pgxpool. Using the pool directly
|
|
// (rather than the per-user-scoped PostgresUserStore) lets us query by
|
|
// (user_id, profile_id) without needing a ForUser call, which would require
|
|
// knowing the integer user_id at construction time. The ABS handlers carry
|
|
// user_id as a string and resolve it inline here.
|
|
type ABSProgressStore struct {
|
|
Pool *pgxpool.Pool
|
|
}
|
|
|
|
var _ abs.ProgressStore = (*ABSProgressStore)(nil)
|
|
|
|
// GetProgress returns the progress row for (userID, profileID, contentID).
|
|
// Returns (nil, nil) when no row exists.
|
|
func (s *ABSProgressStore) GetProgress(ctx context.Context, userID, profileID, contentID string) (*abs.ProgressRow, error) {
|
|
uid, err := strconv.Atoi(userID)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: invalid user_id %q: %w", userID, err)
|
|
}
|
|
var row abs.ProgressRow
|
|
var updatedAt time.Time
|
|
var positionSeconds, durationSeconds float64
|
|
var completed bool
|
|
var progressPct *float64
|
|
|
|
// Completed rows store position_seconds = 0 (no resume point), so the
|
|
// percentage must come from the completed flag, not the position.
|
|
dbRow := s.Pool.QueryRow(ctx, `
|
|
SELECT media_item_id, position_seconds, duration_seconds, completed,
|
|
CASE WHEN completed THEN 1.0
|
|
WHEN duration_seconds > 0 THEN position_seconds / duration_seconds
|
|
ELSE 0 END AS progress_pct,
|
|
updated_at
|
|
FROM user_watch_progress
|
|
WHERE user_id = $1 AND profile_id = $2 AND media_item_id = $3`,
|
|
uid, profileID, contentID,
|
|
)
|
|
err = dbRow.Scan(
|
|
&row.ContentID, &positionSeconds, &durationSeconds, &completed,
|
|
&progressPct, &updatedAt,
|
|
)
|
|
if errors.Is(err, pgx.ErrNoRows) {
|
|
return nil, nil
|
|
}
|
|
if err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: get progress: %w", err)
|
|
}
|
|
|
|
row.UserID = userID
|
|
row.ProfileID = profileID
|
|
row.CurrentSeconds = positionSeconds
|
|
row.DurationSeconds = durationSeconds
|
|
row.IsFinished = completed
|
|
if progressPct != nil {
|
|
row.ProgressPct = *progressPct
|
|
}
|
|
row.UpdatedAt = updatedAt
|
|
return &row, nil
|
|
}
|
|
|
|
// ListProgressForAudiobooks returns all progress rows for (userID, profileID)
|
|
// that join to media_items with type = 'audiobook'. Ordered by updated_at DESC,
|
|
// capped at limit rows.
|
|
func (s *ABSProgressStore) ListProgressForAudiobooks(ctx context.Context, userID, profileID string, limit int) ([]abs.ProgressRow, error) {
|
|
uid, err := strconv.Atoi(userID)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: invalid user_id %q: %w", userID, err)
|
|
}
|
|
if limit <= 0 {
|
|
limit = 500
|
|
}
|
|
rows, err := s.Pool.Query(ctx, `
|
|
SELECT wp.media_item_id,
|
|
wp.position_seconds,
|
|
wp.duration_seconds,
|
|
wp.completed,
|
|
CASE WHEN wp.completed THEN 1.0
|
|
WHEN wp.duration_seconds > 0 THEN wp.position_seconds / wp.duration_seconds
|
|
ELSE 0 END,
|
|
wp.updated_at
|
|
FROM user_watch_progress wp
|
|
JOIN media_items mi ON mi.content_id = wp.media_item_id
|
|
WHERE wp.user_id = $1
|
|
AND wp.profile_id = $2
|
|
AND mi.type = 'audiobook'
|
|
ORDER BY wp.updated_at DESC
|
|
LIMIT $3`,
|
|
uid, profileID, limit,
|
|
)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: list progress: %w", err)
|
|
}
|
|
defer rows.Close()
|
|
|
|
var result []abs.ProgressRow
|
|
for rows.Next() {
|
|
var p abs.ProgressRow
|
|
var updatedAt time.Time
|
|
if err := rows.Scan(
|
|
&p.ContentID,
|
|
&p.CurrentSeconds,
|
|
&p.DurationSeconds,
|
|
&p.IsFinished,
|
|
&p.ProgressPct,
|
|
&updatedAt,
|
|
); err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: scan progress row: %w", err)
|
|
}
|
|
p.UserID = userID
|
|
p.ProfileID = profileID
|
|
p.UpdatedAt = updatedAt
|
|
result = append(result, p)
|
|
}
|
|
if err := rows.Err(); err != nil {
|
|
return nil, fmt.Errorf("abs_progress_store: iterate progress rows: %w", err)
|
|
}
|
|
return result, nil
|
|
}
|
|
|
|
// UpsertProgress inserts or updates a user_watch_progress row. Conflict
|
|
// updates merge monotonically so concurrent writes cannot un-finish an item or
|
|
// rewind progress with a stale position.
|
|
func (s *ABSProgressStore) UpsertProgress(ctx context.Context, row abs.ProgressRow) error {
|
|
uid, err := strconv.Atoi(row.UserID)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: invalid user_id %q: %w", row.UserID, err)
|
|
}
|
|
updatedAt := row.UpdatedAt
|
|
if updatedAt.IsZero() {
|
|
updatedAt = time.Now().UTC()
|
|
}
|
|
// Completed rows hold no resume point (position_seconds = 0) — the same
|
|
// invariant as the userstore write paths — so finished books can't leak
|
|
// into position-based continue-watching/listening queries. A later
|
|
// non-finished report (re-listen) moves position forward from 0 while
|
|
// the completed latch stays.
|
|
position := row.CurrentSeconds
|
|
if row.IsFinished {
|
|
position = 0
|
|
}
|
|
_, err = s.Pool.Exec(ctx, `
|
|
INSERT INTO user_watch_progress
|
|
(user_id, profile_id, media_item_id, position_seconds, duration_seconds, completed, updated_at)
|
|
VALUES ($1, $2, $3, $4, $5, $6, $7)
|
|
ON CONFLICT (user_id, profile_id, media_item_id) DO UPDATE SET
|
|
position_seconds = CASE WHEN EXCLUDED.completed THEN 0
|
|
ELSE GREATEST(user_watch_progress.position_seconds, EXCLUDED.position_seconds) END,
|
|
duration_seconds = GREATEST(user_watch_progress.duration_seconds, EXCLUDED.duration_seconds),
|
|
completed = user_watch_progress.completed OR EXCLUDED.completed,
|
|
updated_at = GREATEST(user_watch_progress.updated_at, EXCLUDED.updated_at)`,
|
|
uid, row.ProfileID, row.ContentID,
|
|
position, row.DurationSeconds, row.IsFinished, updatedAt,
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: upsert progress: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// UpdateProgressPosition advances the resume cursor from a session-sync tick.
|
|
// It is monotonic and finish-preserving: position only moves forward
|
|
// (GREATEST) and a completed row is never moved or un-finished. Without the
|
|
// GREATEST guard an out-of-order tick or a second device rewound the saved
|
|
// position. It deliberately stays UPDATE-only (no row created when none
|
|
// exists): the row is created by the explicit progress-report path, so a stray
|
|
// sync tick can't resurrect progress the user just cleared, nor create a
|
|
// zero-duration row.
|
|
func (s *ABSProgressStore) UpdateProgressPosition(ctx context.Context, userID, profileID, contentID string, positionSeconds float64) error {
|
|
uid, err := strconv.Atoi(userID)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: invalid user_id %q: %w", userID, err)
|
|
}
|
|
_, err = s.Pool.Exec(ctx, `
|
|
UPDATE user_watch_progress
|
|
SET position_seconds = GREATEST(position_seconds, $4),
|
|
updated_at = now()
|
|
WHERE user_id = $1 AND profile_id = $2 AND media_item_id = $3
|
|
AND NOT completed`,
|
|
uid, profileID, contentID, positionSeconds,
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: update progress position: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// DeleteProgress removes the progress row entirely. Idempotent on missing-row.
|
|
// Used by the ABS "Reset Progress" / clear-progress affordance.
|
|
func (s *ABSProgressStore) DeleteProgress(ctx context.Context, userID, profileID, contentID string) error {
|
|
uid, err := strconv.Atoi(userID)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: invalid user id %q: %w", userID, err)
|
|
}
|
|
if _, err := s.Pool.Exec(ctx, `
|
|
DELETE FROM user_watch_progress
|
|
WHERE user_id = $1 AND profile_id = $2 AND media_item_id = $3`,
|
|
uid, profileID, contentID,
|
|
); err != nil {
|
|
return fmt.Errorf("abs_progress_store: delete progress: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// SetHideFromContinue sets the hide_from_continue flag for the given
|
|
// progress row. Idempotent on missing-row.
|
|
func (s *ABSProgressStore) SetHideFromContinue(ctx context.Context, userID, profileID, contentID string, hide bool) error {
|
|
uid, err := strconv.Atoi(userID)
|
|
if err != nil {
|
|
return fmt.Errorf("abs_progress_store: invalid user id %q: %w", userID, err)
|
|
}
|
|
if _, err := s.Pool.Exec(ctx, `
|
|
UPDATE user_watch_progress
|
|
SET hide_from_continue = $4
|
|
WHERE user_id = $1 AND profile_id = $2 AND media_item_id = $3`,
|
|
uid, profileID, contentID, hide,
|
|
); err != nil {
|
|
return fmt.Errorf("abs_progress_store: set hide_from_continue: %w", err)
|
|
}
|
|
return nil
|
|
}
|