haeaeae/silo-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audiobooks): audiobookshelf support — ABS conformance, perf, ebooks (#289)

Browse Source 
* fix(ebooks): fold author hint into metadata search query

The ebook enricher loaded each item's author but buildEbookSearchQuery
dropped it, and metadata.SearchQuery had no field to carry it — so the
plugin only ever received the title. Title-only searches collide or miss,
leaving items without metadata or a cover.

Add SearchQuery.Author and fold it into the plugin search query text
(the SearchMetadataRequest contract carries a single free-text Query, so
no proto change is needed). Gated to callers that set Author (ebooks);
movie/TV search is unchanged.

Verified live against OpenLibrary/GoogleBooks: improves disambiguation on
clean titles. Note: messy filename-derived titles (series prefixes,
trailing "(… Book N)") still need title normalization, and a large tail
of niche/self-published ebooks is simply absent from the free sources —
neither is addressed here.

AI-use disclosure: authored with Claude Code.

(cherry picked from commit ba1265909c4fb87e1a8eab64b0b0c183aa95acc1)

* feat(scanner): extract MOBI/AZW/AZW3 metadata from EXTH headers

These formats previously had no parser — parseEbookFile returned only the
format string, so title fell back to the filename with no author and no
ISBN, leaving ~21k books unmatchable by the metadata enricher.

Parse the Palm Database container (PDB header → record 0 → PalmDOC +
MOBI header → EXTH block) and extract title, authors, ISBN, publisher,
and language. EXTH is located by its magic rather than the header flag,
and field offsets (encoding @12, full-name @0x44/0x48) were verified
against real .mobi/.azw3 files.

Verified live against real library files:
  azw3 → title "The Sea", author "A H Lee"
  mobi → title "Brotherband 3: The Hunters", author "John Flanagan",
         ISBN 9781742750637

AI-use disclosure: authored with Claude Code.

(cherry picked from commit 7af194b711de97bc79855f08a9a4f9732c49db74)

* fix(ebooks): recover author from path and clean provider search title

- ebookAuthorFromPath: recover an author for ".../<Author>/<Title>/<Title> -
  <Author>.ext" layouts when the file embeds none, gated on two agreeing
  path signals (grandparent dir == filename suffix) so magazines/courses
  never get a junk author; strip the suffix from a path-derived title.
- cleanEbookSearchTitle: normalize filesystem-mangled titles before search
  (underscore->space, drop trailing " - <author>") to lift hit rate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 36a16cb58c3e5276aa4c0bdf8577008070f6abea)

* fix(scanner): gate path-author on person-name shape

ebookAuthorFromPath's grandparent==suffix corroboration also matched
inverted layouts ("<Title>/<Author>/<Author> - <Title>"), assigning the
title as the author. Require the candidate directory to look like a person
name (comma form, or all-capitalized tokens plus name particles) so series
and title folders ("De legenden van de Alfen") are rejected, and return the
canonical directory form for proper casing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit ff720bd268a23bff0e94c70f15cb7ecfb8efcb1f)

* fix(ebooks): strip series/book-number parentheticals from search title

cleanEbookSearchTitle now peels trailing "(... Book N)", "[#3]", "(2019)"
groups that don't belong in a provider title query, while leaving
meaningful parentheticals ("(Illustrated)") intact. Enrichment-side only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2273636c04fd9ef483003a9558972a2104fdd3a6)

* fix(ebooks): keep volume number in search title and dedup provider IDs

Two distinct ebooks (e.g. series volumes named only by series + book
number) were collapsing onto a single provider work, then fighting over
the same media_item_provider_ids row:

- cleanEbookSearchTitle stripped trailing "(... Book N)" / "[#3]" groups
  entirely, so every volume of a series searched as the bare series name
  and matched the same provider work. The plugin search contract carries
  only a single free-text Query, so the volume number is now UNWRAPPED
  into the query (brackets dropped, words kept) instead of discarded,
  giving distinct volumes distinct searches. Bare-year groups are still
  dropped (SearchQuery.Year carries them); meaningful parentheticals
  ("(Illustrated)") still survive.

- collectEbookMetadata now consults FindContentIDByProviderIDs before
  accumulating a search-result provider ID. An ID already owned by a
  different content item is skipped, so the loser is not mis-tagged with
  the winner's metadata and ReplaceByContentID no longer violates the
  (provider, provider_id, item_type) unique constraint. The previous
  behavior logged duplicate-key errors every sweep and re-enriched the
  failing item forever (CPU/RAM churn). A failed ownership check is
  surfaced as a provider error so the item retries rather than terminally
  stamping as "no match".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 942fdef6cb0e167b2d9223e9a968b3010b7b3ec8)

* fix(ebooks): address CodeRabbit review on PR #185

- cleanEbookSearchTitle: anchor author-suffix strip to a trailing match
  (optionally followed by a series/volume parenthetical) so a mid-title
  " - <token>" no longer truncates valid title text
- ebook scan: strip the recovered author suffix using normalized comparison
  so case/spacing variants (e.g. "a. f.  carter") don't leave a duplicate
- parseMOBIEXTH: bound parsing to the declared EXTH length so a corrupt
  record count can't read full-text bytes as junk metadata
- add regression test for a non-trailing " - <token>" in the title

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 0f45af8143e04dfd5b4a5ee3e949dcd943eedbd1)

* fix(audiobooks): pass author in search query and retry on provider errors

Set SearchQuery.Author so the host adapter folds author into the
plugin free-text query (parity with ebooks). Track provider errors
during enrichment; when nothing matched and a provider errored, return
an error without stamping last_refreshed so the sweep retries instead
of terminally burning the item on a transient failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit f45dd2104c5d6415324e80f91f6684bc39858459)

* fix(scanner): consolidate fragmented multi-file audiobook content_ids on rescan

audiobookFolderShouldSkip used ListByObservedRootPath which returns all
files for a root path regardless of content_id. When a multi-file audiobook
had files fragmented across multiple content_ids (e.g. from concurrent
refreshes), the file count matched disk so the skip check returned true
and the reconcile never ran to merge them.

Now verifies all DB files share the same content_id before skipping; any
fragmentation forces a full reconcile which consolidates to one content_id
via FindContentIDByRootPath → upsertAudiobookMediaFiles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit e91c33e88a7d09e802e6afd8af246c6c954d0498)

* fix(ingest): skip concurrent match drainer for audiobook/podcast/ebook/manga libraries

The concurrent scoped match drainer ran during scan for all library types.
For audiobook libraries, the scanner assigns content_ids by folder root
(one item per multi-file folder). Running the drainer concurrently caused
it to process files with content_id=NULL (cleared by complete refresh)
as individual items, creating one media_item per file instead of one per
folder. This manifested as 41-file audiobooks fragmenting into dozens of
orphaned single-file content_ids on every refresh.

These library types use scanner-driven grouping; the post-scan drain step
handles them correctly. Returning nil matchScopes skips the concurrent
drainer entirely for these types.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit 93ae9d22ce315874fa22a958b88ca1766075695f)

* fix(abs): match real audiobookshelf auth + session-sync contract

Align the ABS-compat auth flow with real audiobookshelf (v2.26+) so
third-party clients (yaabsa, Plappa, native iOS) authenticate and sync
playback correctly:

- login/refresh: always emit user.accessToken; x-return-tokens gates
  only the refresh token (body vs HttpOnly refresh_token cookie)
- /auth/refresh returns the full login envelope (was a thin token map)
- /me returns the full user object (toOldJSONForBrowser), shared with
  login/authorize via a single absUserObject() builder
- /logout returns 200 {redirect_url:null} and clears the cookie (was 204)
- add POST /session/{sid}/sync (real ABS heartbeat path); it was
  PATCH-only, so the official client's sync POST 404'd and playback
  progress never synced

Verified against advplyr/audiobookshelf server/{Auth.js,models/User.js,
controllers,routers}. Unit tests updated/added; full abs suite green.
Not yet live-verified.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 336e932471d4be021d82299106a120611783836a)

* fix(abs): conform browse/list items to real audiobookshelf minified shape

Strict ABS clients (yaabsa, Plappa) crash or drop items when the browse
list shape only approximates real audiobookshelf. Match the serializers:

- add media.id + media.libraryItemId (= ContentID) to LibraryItemMedia;
  yaabsa BookMedia.id is required non-null and was missing → the whole
  item failed to parse ("Null is not a subtype of String")
- rebuild the minified list shape to LibraryItem.toOldJSONMinified +
  Book.toOldJSONMinified + oldMetadataToJSONMinified key-for-key (ino,
  path, isFile, numFiles/size, media.{id,tags,numTracks,numAudioFiles,
  numChapters,size,ebookFormat}, flat author/series metadata)
- force media.numTracks/numAudioFiles >= 1 in the browse projection so
  Plappa doesn't drop items reporting 0 audio files
- default /items list to minified (real ABS list is always minified);
  minified=0 opts into the full shape

Verified against advplyr/audiobookshelf models/{Book,LibraryItem}.js.
Adds minified_test.go key-set conformance guards; abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6c9387a8c4b60be3dbe541049ed8c06344b10717)

* fix(abs): conform /items/{id} detail to real audiobookshelf expanded shape

Match real audiobookshelf LibraryItem.toOldJSONExpanded +
Book.toOldJSONExpanded + oldMetadataToJSONExpanded so strict clients
decode the item-detail page with the same model they use elsewhere:

- add expanded outer keys to LibraryItem (oldLibraryItemId, lastScan,
  scanVersion, libraryFiles, size) and populate libraryFiles + summed
  size from the item's media files in the detail builder
- add media.size (Book.toOldJSONExpanded)
- make the typed Metadata the full expanded superset: subtitle,
  titleIgnorePrefix, authorName, authorNameLF, narratorName, seriesName,
  descriptionPlain, publishedDate, asin, language, abridged; drop the
  omitempty that previously dropped description/publishedYear/isbn/
  publisher when empty (a missing key crashes strict clients)

Verified against advplyr/audiobookshelf models/Book.js + LibraryItem.js.
Adds items_detail_test.go expanded key-set guard; abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8bd485f0291e9db9f32e13a68609a68ee8a945ec)

* fix(abs): conform authors/series endpoints to real audiobookshelf shapes

Match the real audiobookshelf serializers so strict clients decode the
authors/series browse + detail responses:

- GET /libraries/{id}/authors now branches like LibraryController.getAuthors:
  bare { authors: [...] } when not paginated, paged { results, total, ... }
  only when limit+page are present (was always paged → clients keying on
  `authors` got keyNotFound)
- author objects carry the full Author.toOldJSON key set (id, asin, name,
  description, imagePath, libraryId, addedAt, updatedAt, numBooks); silo has
  no analog for asin/description/imagePath/timestamps so they are null/0
- series objects carry the full Series.toOldJSON key set (adds
  nameIgnorePrefix, description, libraryId, addedAt, updatedAt)
- series/author books are now FULL minified library items (real ABS shape)
  instead of thin {id,media:{metadata:{title}}} stubs that crash Plappa;
  author items moved to the real-ABS `libraryItems` key

Verified against advplyr/audiobookshelf controllers/LibraryController.js and
models/{Author,Series}.js. Tests updated + envelope-branch guard added; abs
suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8a22eb0900ed881d500ded315a508e1a07da14f3)

* fix(abs): add libraryId to collection/playlist objects (real ABS shape)

Real audiobookshelf Collection.toOldJSON and Playlist.toOldJSON both carry
a libraryId; silo's emitters omitted it, so a strict client modeling the
object with a required libraryId crashed. silo collections/playlists are
cross-library user-personal, so emit the virtual audiobook library id.

The books[]/items[] entries already carry the full LibraryItem shape and
inherit the browse-conformance fixes (media.id etc.). Envelopes were
already correct (paged for library-scoped, {collections}/{playlists} for
global).

Verified against advplyr/audiobookshelf models/{Collection,Playlist}.js.
Envelope key-set tests updated; abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit f7d2ff0565f05c3a6ef7f36f2b1f252bd373fa7a)

* fix(abs): conform library object + /libraries/{id} to real audiobookshelf

The library object was only {id,name,mediaType}; real audiobookshelf
Library.toOldJSON has 12 keys, so a strict client decoding the library
model crashed on the missing ones. Also GET /libraries/{id} always wrapped
the object in { library: ... }, but real ABS returns it directly unless
?include=filterdata is requested.

- audiobookLibraryMap now emits the full Library.toOldJSON shape (folders[]
  as LibraryFolder.toOldJSON, displayOrder, icon, provider, settings,
  lastScan, lastScanVersion, createdAt, lastUpdate). This also enriches the
  libraries[] on the login envelope, which shares the builder.
- handleLibraryDetail returns the library object DIRECTLY without include,
  and wraps in { filterdata, issues, numUserPlaylists,
  customMetadataProviders, library } (adds the missing
  customMetadataProviders) with include=filterdata.

GET /libraries already returned { libraries: [...] } (correct). Verified
against advplyr/audiobookshelf models/Library.js +
controllers/LibraryController.js. Adds libraries_shape_test.go; abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit d05a2f2af1caf21a4ad04577a4787ba02cff091c)

* fix(abs): conform personalized recent-series shelf to real ABS series shape

The /libraries/{id}/personalized "Recent Series" shelf emitted thin
{id,name,numBooks,libraryId,books:[]} entities with an always-empty cover
stack. Emit the full real-ABS series object (seriesObjectABS, adds
nameIgnorePrefix/description/addedAt/updatedAt) with minified book items
(seriesBookMinified) — the same shape as /libraries/{id}/series so the
shelf card decodes identically and shows real covers.

Book shelves already used full minified items; the shelves array is a bare
array (matches real ABS getUserPersonalizedShelves). abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7c586f8c923cbc481f6f94e304af48dee379a999)

* fix(abs): conform listening-sessions to real audiobookshelf PlaybackSession shape

silo's /me/listening-sessions returned a thin 5-field session object
(id, libraryItemId, userId, timeListening, currentTime) wrapped in the
generic pagedEnvelope shape ({results,sortBy,filterBy,minified}). Real
audiobookshelf clients (Flutter/Swift strict decoders) expect the
MeController.getListeningSessions envelope
({total,numPages,page,itemsPerPage,sessions}) and each session to carry
the full PlaybackSession.toJSON() key set, so the missing keys (notably
mediaType, mediaMetadata, displayTitle, displayAuthor, coverPath,
duration, chapters, deviceInfo, playMethod, mediaPlayer, serverVersion,
date, dayOfWeek, startTime, startedAt, updatedAt, libraryId, bookId,
episodeId) crashed with keyNotFound errors.

Both handleListeningSessions and handleListeningSessionDetail now build
the response via a shared sessionToABS() that reuses
buildSiloPlayMediaMetadata (already used by /play) to hydrate
mediaMetadata/displayTitle/displayAuthor from MediaStore, batching
lookups via GetAudiobooksByIDs for the list endpoint. Lookups are
best-effort: a missing/inaccessible item falls back to a stub
MediaItem so every key is still emitted, never a crash.

Verified against advplyr/audiobookshelf server/controllers/MeController.js
(getListeningSessions) and server/objects/PlaybackSession.js (toJSON())
on GitHub master.

Known placeholders (real ABS fields we can't populate without extra
cost): chapters (empty array — would require a per-session media-files
fetch), duration (0 — total book duration isn't tracked on the session
row), startTime (0 — not persisted separately from currentTime),
deviceInfo (static "unknown" device, matching the /play endpoint's
existing placeholder — no device info is persisted per session).

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9471497c99b96c8d3defc6c8112c913f7c55924b)

* feat(abs): add offline session sync endpoints (/session/local, /session/local-all)

The official ABS mobile app records playback while offline and POSTs those
PlaybackSession objects back on reconnect via SessionController.syncLocal and
syncLocalSessions. silo was missing both endpoints, so offline listening
progress was silently lost. Add them to the bearerAuth-protected session group
(both /abs/api and /api prefixes) alongside /session/{sid}/sync and /close.

POST /session/local decodes one PlaybackSession and updates the caller's resume
position via ProgressStore.UpdateProgressPosition (the same call handleSessionSync
uses), emitting user_item_progress_updated. POST /session/local-all decodes
{sessions:[...]} and loops each robustly — a malformed or unknown item marks that
one result failed without sinking the batch — returning {results:[...]}. No new
store persistence or migration; podcast/episode sessions are accepted as no-ops.

Verified against advplyr/audiobookshelf server/controllers/SessionController.js
and server/managers/PlaybackSessionManager.js.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 008a4df948d855a4bfe62b24f89bfc484f088033)

* fix(abs): conform library search + items-in-progress to real audiobookshelf

Real ABS's libraryItemsBookFilters.search() (delegated from
LibraryController.search) returns { book, narrators, tags, genres,
series, authors } with no "podcast" key for a book library, and each
book entry is only { libraryItem } — no matchKey/matchText, which our
handler was inventing. Search now matches those keys, drops the
fabricated matchKey/matchText fields, and best-effort populates
authors/series buckets via client-side substring filtering over the
existing aggregate listers (narrators/tags/genres stay empty-but-present
since silo has no backing aggregation query for them yet).

MeController.getAllLibraryItemsInProgress wraps items as
{ ...libraryItem.toOldJSONMinified(), progressLastUpdate }; our handler
was emitting a hand-rolled subset of fields plus a nested
userMediaProgress object that doesn't exist in the real response.
items-in-progress now reuses the existing Minify() projection and merges
a flat progressLastUpdate (ms) field to match.

Verified against advplyr/audiobookshelf controllers/{Library,Me}Controller.js
and server/utils/queries/{libraryItemsBookFilters,authorFilters}.js.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 998ff55f3cf27d504f0e5aec8c7c29fa57f10247)

* fix(abs): /ping returns success:true and /status carries authMethods

The ABS apps validate a server address by reading response.success from
GET /ping; silo returned {pong:true,...} with no `success`, so the app
reported "unable to reach" even though the server responded 200. Also
/status was missing authMethods/authFormData, which the app reads to render
the login form.

- /ping now includes {"success": true} (pong/server/version kept as extras)
- /status now returns {app,serverVersion,isInit,language,authMethods,
  authFormData} matching real audiobookshelf Server.js

Verified against advplyr/audiobookshelf server/Server.js. Adds
ping_status_test.go; abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit df732d09355303608bc4d5fc2138555e37497a02)

* fix(abs): mount login + auth/refresh under /api prefix

Clients that post to /api/login (and /api/auth/refresh) got a 404 because
login/refresh were only mounted at root and /abs/api — while the rest of the
authenticated ABS surface (/api/me, /api/authorize, /api/libraries, covers)
is served under both /api and /abs/api. The 404 surfaced in the client as a
generic "unknown error occurred" on sign-in.

Mount /login and /auth/refresh under all three prefixes ("", /api, /abs/api),
matching the authenticated groups.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 18071e180b02131cda354303eadd1ff3a0708065)

* fix(abs): accept form-encoded login bodies (not just JSON)

Real audiobookshelf (express body-parser + passport local) accepts both
application/json and application/x-www-form-urlencoded credential bodies.
Silo only json-decoded the body, so a form-encoded client got 400 "invalid
request body" — surfaced in the app as a generic "unknown error" on sign-in
(confirmed live: JSON creds -> 200, identical form-encoded creds -> 400).

Buffer the body once, try JSON, then fall back to url.ParseQuery for the
form-encoded case.

Adds login_body_test.go (form + JSON both reach the validator). abs suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 408dc33debb7a7ee363778094511ccfdd1ee70d2)

* fix(abs): emit full real-ABS serverSettings (OpenID/auth fields)

silo's login/authorize serverSettings omitted the auth + OpenID fields that
real audiobookshelf ServerSettings.toJSONForBrowser includes
(authLoginCustomMessage, authOpenID*, rateLimitLogin*, backupPath,
allowedOrigins). OIDC-aware strict clients (Prologue, iOS/Swift) decode
serverSettings into a model that requires those keys, so their absence throws
keyNotFound and the ENTIRE login response fails to decode — the client stays
on the login screen with a generic "unknown error" even though the server
returned 200. Simpler clients that don't model OpenID were unaffected.

Emit real ABS's OIDC-disabled defaults; authActiveAuthMethods still advertises
only "local" so no client initiates the OpenID flow.

Diagnosed from a packet capture (Prologue posts /login? with X-Return-Tokens
and gets a 200 it can't decode) + real ABS ServerSettings.js. Verified against
advplyr/audiobookshelf.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 283826c952824e97233e46e057f37385ddc3054b)

* fix(abs): GET /me returns the real display username, not the userID

/me built its user object from the token claims and passed the numeric
userID as the username, so clients saw "98" instead of "puksthepirate".
Login gets the display name from the credential validator, but /me only has
the token, so it needs a lookup.

Add an optional UsernameResolver to the abs Dependencies; wire it from the
concrete SiloCredValidator (which holds the pgx pool) via a new
ResolveUsername method that mirrors Validate's display-name logic — the
profile name when a profile is set and named, else the account username.
handleMe uses it and falls back to the userID when unresolved.

abs package compiles + tests pass; the audiobooks package (service.go,
cred_validator.go) could not be linked locally (pre-existing bimg/libvips
pkg-config gap) and is validated at the Docker build.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 39ff3e3350aad087757f7fb0e94d5a7f10c08ae5)

* fix(abs): always emit AudioTrack keys + correct media.duration

Two item-detail issues that made Prologue report "Unable to load book
contents" (can't press Start Listening):

- AudioTrack used omitempty on chapters/metaTags/format/bitRate/codec/
  metadata/etc, so empty values dropped those keys. Real ABS AudioFile/
  AudioTrack always emit them; strict clients (Prologue, yaabsa) decode
  tracks into a required-field model and throw keyNotFound on the missing
  keys, failing the whole track decode. Removed omitempty and emit
  chapters/metaTags as [] / {} (non-nil) in both track builders.
- media.duration used the item's Runtime, which is often stale/mis-scanned
  (e.g. 222s for a 3.7h book) and desyncs the player scrubber. Now sum the
  track durations (real ABS: sum of audio file durations), falling back to
  Runtime only when there are no tracks.

Verified against advplyr/audiobookshelf models/Book.js (AudioFile/AudioTrack)
via a live packet capture of Prologue's item-detail decode failure. abs
suite green.

AI-use: implemented with Claude Code (Opus 4.8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8210ed2fc63e782f158b4168ef693e671ae19638)

* perf(abs): push down library browse filters + author counts MV

The ABS audiobook library-serving path was slow on large libraries
(~255k items): /libraries/{id}/items?filter=authors.{id} loaded and
hydrated the whole library into Go before filtering (~4.8s each), and
/libraries/{id}/authors ran a full GroupAggregate + COUNT(DISTINCT)
per page (~53s full sync) — slow enough to trip ABS client sync
timeouts (e.g. Prologue).

- Push author/series/narrator/no-series filters into indexed SQL
  EXISTS predicates in ListAudiobooks; paginate + COUNT in SQL.
  Semantically equivalent to the prior Go-side filter (kind=7 author,
  kind=8 narrator, exact-case match, no-series sentinel).
- Add covering index media_items(content_id, type) so the count/list
  type check runs index-only (CONCURRENTLY, NO TRANSACTION — no
  write-lock on the live table).
- Serve /authors from a materialized view (abs_audiobook_author_counts)
  refreshed every 15min, with a live-query fallback when the view is
  empty/unrefreshed so the endpoint never blanks on a fresh deploy.

Conformance preserved: keeps authorObjectABS/seriesObjectABS shapes and
the limit&&page envelope decision; adds a regression test for the
bare {authors:[...]} envelope on limit-only requests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 0d55754051dd3ad016b3cec6a0921307071d3219)

* perf(abs): index-back audiobook search via trigram GIN

SearchAudiobooks matched the raw media_items.title with ILIKE '%q%'
OR'd with an author/narrator EXISTS. The un-indexed raw-title column
plus the OR forced a full seq scan of the ~255k-item library on every
search (~560ms on library 18).

Reshape into a UNION of two index-driven arms that reuse the search
infrastructure the rest of the catalog already relies on: the title arm
matches media_items.title_normalized (idx_media_items_title_normalized_trgm)
via the shared normalize_search_text(), the people arm matches people.name
(idx_people_name_trgm). GROUP BY content_id keeps the best rank when an
item matches both; a normalize_search_text($2) <> '' guard stops a
punctuation-only query from degenerating into ILIKE '%%'.

No new index or migration — the trigram indexes already existed and were
simply unused. ~560ms -> ~35ms, both indexes engaged, no seq scan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 98bbd1712719ecd03e4db87f840774cec788f177)

* perf(abs): index-ordered item paging + cached library count

The unfiltered /libraries/{id}/items path that ABS clients page through
to sync a library recomputed COUNT(*) over the whole library on every
page (~150ms each) and ordered by LOWER(sort_title), LOWER(title) — an
expression matching no index, forcing a full in-memory sort of all
~255k rows per page (~324ms shallow, ~543ms deep). A full sync is
thousands of pages, so both costs dominated indexing time.

- Order by lower(coalesce(nullif(btrim(sort_title),''), title)),
  content_id so the page is served by an ordered index scan on the
  existing idx_media_items_sort_key (~324ms -> ~1ms). content_id (PK)
  is a stable tiebreaker, making sequential pagination deterministic —
  the prior ordering could skip/repeat rows when sort keys collided.
- Memoize the per-page COUNT in a 60s TTL cache keyed on the fully
  rendered count SQL + bound args, so it covers every input the WHERE
  depends on (library, pushed-down filter, all access predicates) and
  can't drift as access logic evolves. Expired entries swept on write.

No new index or migration — reuses idx_media_items_sort_key.
total may lag up to 60s during an active scan; clients re-sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 32e26c2f99a1ffc071f600071c2ea7ddcd3397b4)

* fix(abs): address PR review — access-aware authors, offline progress create, cookie refresh, body limits

- media_store: ListLibraryAuthors bypassed per-item access when reading the
  author materialized view (keyed by library only), leaking authors of books
  hidden by a content-rating cap or excluded media types. Take the access-aware
  live path whenever the filter carries an item-level predicate.
- session_local: offline sync used UPDATE-only UpdateProgressPosition, so a book
  listened to entirely offline (no progress row yet) had its position silently
  dropped while still reporting progressSynced. Create the row via UpsertProgress
  when none exists; keep the monotonic update path for existing rows.
- login: handleRefresh never read the refresh_token cookie, so cookie-flow ABS
  clients got 400 refreshToken required once the access token expired. Read the
  cookie as a third source after header and body.
- session_local: cap /session/local and /session/local-all request bodies at
  1 MiB via io.LimitReader, matching the rest of the package.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Puks The Pirate
2026-07-05 00:16:34 -04:00
committed by GitHub
co-authored by Claude Opus 4.8
parent c0f209a936
commit 97ac2b4eed
47 changed files with 3569 additions and 511 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/audiobooks/abs/author_series_handler.go
+84 -17
View File
@@ -32,7 +32,8 @@ func (h *Handler) handleAuthorDetail(w http.ResponseWriter, r *http.Request) {
http.Error(w, "author get failed", http.StatusInternalServerError)
return
}
writeJSON(w, http.StatusOK, authorToABS(author))
lib := h.resolveDefaultLibrary(r.Context(), access)
writeJSON(w, http.StatusOK, authorToABS(author, lib, h.absBaseURL(r)))
}
func (h *Handler) handleSeriesDetail(w http.ResponseWriter, r *http.Request) {
@@ -62,31 +63,97 @@ func (h *Handler) handleSeriesDetail(w http.ResponseWriter, r *http.Request) {
http.Error(w, "series get failed", http.StatusInternalServerError)
return
}
writeJSON(w, http.StatusOK, seriesToABS(series))
lib := h.resolveDefaultLibrary(r.Context(), access)
writeJSON(w, http.StatusOK, seriesToABS(series, lib, h.absBaseURL(r)))
}
func authorToABS(a Author) map[string]any {
books := make([]map[string]any, 0, len(a.Books))
// authorObjectABS builds the real ABS Author.toOldJSON(+numBooks) shape
// (server/models/Author.js). silo does not track asin/description/imagePath/
// timestamps, so those are emitted as null/0 — nullable in real ABS, and a
// present key (not its value) is what keeps strict clients from crashing.
func authorObjectABS(id, name, libraryID string, numBooks int) map[string]any {
return map[string]any{
"id": id,
"asin": nil,
"name": name,
"description": nil,
"imagePath": nil,
"libraryId": libraryID,
"addedAt": 0,
"updatedAt": 0,
"numBooks": numBooks,
}
}
func authorToABS(a Author, lib AudiobookLibrary, baseURL string) map[string]any {
libID := audiobookLibraryID(lib)
obj := authorObjectABS(a.ID, a.Name, libID, len(a.Books))
// Author-detail books are full minified library items (not thin stubs) so
// any strict client decodes them with its LibraryItem model.
books := make([]MinifiedLibraryItem, 0, len(a.Books))
for _, b := range a.Books {
books = append(books, map[string]any{"id": b.ContentID, "media": map[string]any{"metadata": map[string]any{"title": b.Title}}})
books = append(books, Minify(siloItemToLibraryItem(b, lib, baseURL)))
}
obj["libraryItems"] = books
return obj
}
// seriesObjectABS builds the real ABS Series.toOldJSON shape
// (server/models/Series.js). description/timestamps are absent in silo's
// catalog → null/0.
func seriesObjectABS(id, name, libraryID string, numBooks int) map[string]any {
return map[string]any{
"id": a.ID,
"name": a.Name,
"numBooks": len(a.Books),
"books": books,
"id": id,
"name": name,
"nameIgnorePrefix": titleIgnorePrefix(name),
"description": nil,
"addedAt": 0,
"updatedAt": 0,
"libraryId": libraryID,
"numBooks": numBooks,
}
}
func seriesToABS(s Series) map[string]any {
books := make([]map[string]any, 0, len(s.Books))
func seriesToABS(s Series, lib AudiobookLibrary, baseURL string) map[string]any {
libID := audiobookLibraryID(lib)
obj := seriesObjectABS(s.ID, s.Name, libID, len(s.Books))
books := make([]MinifiedLibraryItem, 0, len(s.Books))
for _, b := range s.Books {
books = append(books, map[string]any{"id": b.ContentID, "media": map[string]any{"metadata": map[string]any{"title": b.Title}}})
books = append(books, Minify(siloItemToLibraryItem(b, lib, baseURL)))
}
return map[string]any{
"id": s.ID,
"name": s.Name,
"numBooks": len(s.Books),
"books": books,
obj["books"] = books
return obj
}
// seriesBookMinified builds a full real-ABS minified library item from the
// limited fields the series-LIST query carries (no full MediaItem). Every
// required minified key is present with a safe placeholder so strict clients
// (Plappa) decode the series card's books[] without crashing.
func seriesBookMinified(contentID, title, libID, baseURL string, updatedAtMs int64) MinifiedLibraryItem {
return MinifiedLibraryItem{
ID: contentID,
Ino: contentID,
LibraryID: libID,
FolderID: VirtualFolderID,
IsFile: true,
MtimeMs: updatedAtMs,
CtimeMs: updatedAtMs,
BirthtimeMs: updatedAtMs,
AddedAt: updatedAtMs,
UpdatedAt: updatedAtMs,
MediaType: LibraryMediaType,
Media: minifiedMedia{
ID: contentID,
Metadata: minifiedMetadata{
Title: title,
TitleIgnorePrefix: titleIgnorePrefix(title),
Genres: []string{},
},
CoverPath: baseURL + "/api/items/" + contentID + "/cover",
Tags: []string{},
NumTracks: 1,
NumAudioFiles: 1,
},
NumFiles: 1,
}
}
internal/audiobooks/abs/author_series_handler_test.go
+113 -3
View File
@@ -50,10 +50,108 @@ func TestAuthor_Detail_ReturnsBooks(t *testing.T) {
if got["name"] != "Brandon Sanderson" {
t.Errorf("name = %v", got["name"])
}
books, _ := got["books"].([]any)
if len(books) != 2 {
t.Errorf("books len = %d, want 2", len(books))
// Real ABS Author.toOldJSON key set (+ numBooks).
for _, k := range []string{"id", "asin", "name", "description", "imagePath", "libraryId", "addedAt", "updatedAt", "numBooks"} {
if _, ok := got[k]; !ok {
t.Errorf("author object missing key %q", k)
}
}
// Author items are real-ABS minified library items under libraryItems.
items, _ := got["libraryItems"].([]any)
if len(items) != 2 {
t.Errorf("libraryItems len = %d, want 2", len(items))
}
if len(items) > 0 {
b0, _ := items[0].(map[string]any)
if _, ok := b0["ino"]; !ok {
t.Errorf("author libraryItem missing minified key 'ino' (thin stub regression)")
}
}
}
type libAuthorsStub struct {
noopMediaStore
authors []AuthorSummary
}
func (s *libAuthorsStub) ListLibraryAuthors(_ context.Context, _ int64, _, _ int, _ string, _ bool, _ catalog.AccessFilter) ([]AuthorSummary, int, error) {
return s.authors, len(s.authors), nil
}
// TestLibraryAuthors_EnvelopeBranchesOnPagination guards the real ABS
// LibraryController.getAuthors shape: bare { authors: [...] } when NOT
// paginated, paged { results, total, ... } when limit+page are present.
func TestLibraryAuthors_EnvelopeBranchesOnPagination(t *testing.T) {
media := &libAuthorsStub{authors: []AuthorSummary{
{ID: "1", Name: "Alpha", NumBooks: 2},
{ID: "2", Name: "Beta", NumBooks: 1},
}}
h := New(Dependencies{MediaStore: media})
params := map[string]string{"libraryId": VirtualLibraryID}
// Non-paginated → { authors: [...] }
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/"+VirtualLibraryID+"/authors", params, nil, "1", "", h.handleLibraryAuthors)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
authors, ok := got["authors"].([]any)
if !ok {
t.Fatalf("non-paginated response missing 'authors' key; got keys %v", keysOf(got))
}
if _, isPaged := got["results"]; isPaged {
t.Errorf("non-paginated response must NOT carry paged 'results'")
}
if len(authors) != 2 {
t.Errorf("authors len = %d, want 2", len(authors))
}
if a0, _ := authors[0].(map[string]any); a0 != nil {
if _, ok := a0["asin"]; !ok {
t.Errorf("author object missing 'asin' (thin shape regression)")
}
}
// limit present but NO page → still bare { authors: [...] }. Both limit and
// page are required to trigger the paged envelope; a limit-only request
// (e.g. Prologue's ?limit=100) must decode via `authors`, not `results`.
recLimitOnly := dispatchABSWithParams(http.MethodGet, "/api/libraries/"+VirtualLibraryID+"/authors?limit=100", params, nil, "1", "", h.handleLibraryAuthors)
if recLimitOnly.Code != http.StatusOK {
t.Fatalf("limit-only status = %d; body=%s", recLimitOnly.Code, recLimitOnly.Body.String())
}
var gotLimitOnly map[string]any
if err := json.Unmarshal(recLimitOnly.Body.Bytes(), &gotLimitOnly); err != nil {
t.Fatalf("decode limit-only: %v", err)
}
if _, ok := gotLimitOnly["authors"].([]any); !ok {
t.Errorf("limit-only (no page) response missing bare 'authors' key; got keys %v", keysOf(gotLimitOnly))
}
if _, isPaged := gotLimitOnly["results"]; isPaged {
t.Errorf("limit-only (no page) response must NOT carry paged 'results'")
}
// Paginated → paged envelope
rec2 := dispatchABSWithParams(http.MethodGet, "/api/libraries/"+VirtualLibraryID+"/authors?limit=10&page=0", params, nil, "1", "", h.handleLibraryAuthors)
var got2 map[string]any
if err := json.Unmarshal(rec2.Body.Bytes(), &got2); err != nil {
t.Fatalf("decode paged: %v", err)
}
if _, ok := got2["results"]; !ok {
t.Errorf("paginated response missing 'results'; got keys %v", keysOf(got2))
}
if _, ok := got2["total"]; !ok {
t.Errorf("paginated response missing 'total'")
}
}
func keysOf(m map[string]any) []string {
ks := make([]string, 0, len(m))
for k := range m {
ks = append(ks, k)
}
return ks
}
func TestAuthor_Detail_Unknown_404(t *testing.T) {
@@ -86,10 +184,22 @@ func TestSeries_Detail_ReturnsBooks(t *testing.T) {
if got["name"] != "Mistborn" {
t.Errorf("name = %v", got["name"])
}
// Real ABS Series.toOldJSON key set (+ numBooks/books).
for _, k := range []string{"id", "name", "nameIgnorePrefix", "description", "addedAt", "updatedAt", "libraryId", "numBooks"} {
if _, ok := got[k]; !ok {
t.Errorf("series object missing key %q", k)
}
}
books, _ := got["books"].([]any)
if len(books) != 2 {
t.Errorf("books len = %d, want 2", len(books))
}
if len(books) > 0 {
b0, _ := books[0].(map[string]any)
if _, ok := b0["ino"]; !ok {
t.Errorf("series book missing minified key 'ino' (thin stub regression)")
}
}
}
func TestSeries_Detail_Unknown_404(t *testing.T) {
internal/audiobooks/abs/collections.go
+3 -2
View File
@@ -7,7 +7,7 @@ import (
// CollectionStore is the narrow slice of user_personal_collections
// (collection_type='manual') and user_personal_collection_items
// (sub_item_id='') the collections handlers need. Implemented by
// (sub_item_id=”) the collections handlers need. Implemented by
// ABSCollectionStore in internal/audiobooks/abs_collection_store.go;
// post-migration-156 it reads the unified canonical tables.
type CollectionStore interface {
@@ -53,7 +53,7 @@ type Collection struct {
// CollectionItem is the in-memory representation of a
// user_personal_collection_items row scoped to a manual collection
// (sub_item_id='').
// (sub_item_id=”).
type CollectionItem struct {
CollectionID string
LibraryItemID string
@@ -69,6 +69,7 @@ type CollectionItem struct {
func collectionToABS(c Collection, books []map[string]any) map[string]any {
out := map[string]any{
"id": c.ID,
"libraryId": VirtualLibraryID, // real ABS Collection.toOldJSON has libraryId; silo collections are cross-library user-personal
"userId": c.UserID,
"name": c.Name,
"description": c.Description,
internal/audiobooks/abs/collections_envelope_test.go
+1 -1
View File
@@ -25,7 +25,7 @@ func TestCollectionEnvelope_HasRequiredKeys(t *testing.T) {
body, _ := json.Marshal(out)
js := string(body)
for _, key := range []string{
`"id":`, `"userId":`, `"name":`, `"description":`,
`"id":`, `"libraryId":`, `"userId":`, `"name":`, `"description":`,
`"isPublic":`, `"lastUpdate":`, `"createdAt":`, `"books":`,
} {
if !strings.Contains(js, key) {
internal/audiobooks/abs/handler.go
+42 -19
View File
@@ -46,7 +46,9 @@ type MediaStore interface {
GetAudiobooksByIDs(ctx context.Context, contentIDs []string, access catalog.AccessFilter) (map[string]*models.MediaItem, error)
// ListAudiobooks returns a page of audiobooks. When libraryID is non-zero
// it filters to items in that media_folder; 0 means all audiobook items.
ListAudiobooks(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter) ([]*models.MediaItem, int, error)
// filter optionally pushes an authors/series/narrators predicate into the
// query (Filter{} for none) so per-author syncs avoid a full-library scan.
ListAudiobooks(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter, filter Filter) ([]*models.MediaItem, int, error)
GetMediaFiles(ctx context.Context, contentID string, access catalog.AccessFilter) ([]*models.MediaFile, error)
// GetMediaFileByID fetches a single media file by its integer PK.
// Used by the ABS file-streaming handler when a caller supplies a
@@ -67,12 +69,14 @@ type MediaStore interface {
// ListDiscover returns a randomized sampling of audiobooks for the
// Home tab's discover shelf (helps new users browse the library).
ListDiscover(ctx context.Context, libraryID int64, limit int, access catalog.AccessFilter) ([]*models.MediaItem, error)
// ListLibraryAuthors returns distinct authors of audiobooks in the
// library along with each author's book count.
ListLibraryAuthors(ctx context.Context, libraryID int64, limit int, access catalog.AccessFilter) ([]AuthorSummary, error)
// ListLibrarySeries returns distinct series (from audiobook_series)
// represented in the library, ordered by name.
ListLibrarySeries(ctx context.Context, libraryID int64, limit int, access catalog.AccessFilter) ([]SeriesSummary, error)
// ListLibraryAuthors returns one page of distinct audiobook authors (from a
// precomputed materialized view) plus the total author count. sortBy is one
// of "name" (default), "addedAt", or "numBooks"; limit<=0 returns all.
ListLibraryAuthors(ctx context.Context, libraryID int64, limit, offset int, sortBy string, sortDesc bool, access catalog.AccessFilter) ([]AuthorSummary, int, error)
// ListLibrarySeries returns one SQL-paginated page of distinct series (from
// audiobook_series) in the library plus the total series count. limit<=0
// returns all.
ListLibrarySeries(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter) ([]SeriesSummary, int, error)
// GetAuthorByID returns the author with the given people.id plus
// their audiobook list, sorted by title. Returns ErrNotFound when
// no people row matches.
@@ -223,10 +227,16 @@ type Dependencies struct {
TokenStore TokenStore
CredValidator ProfileCredentialValidator
AccessResolver AccessResolver
Config ConfigProvider
Publisher EventPublisher // may be nil
Recommender Recommender // may be nil
LoginLimiter *LoginLimiter // may be nil — one is created if absent
// UsernameResolver returns the display username for an ABS principal
// (userID, profileID) without re-authenticating. Optional; GET /me falls
// back to the userID when this is nil or returns "". Login gets the
// display name from the credential validator, but /me only has the token
// claims, so it needs this to show the real username instead of the id.
UsernameResolver func(ctx context.Context, userID, profileID string) string
Config ConfigProvider
Publisher EventPublisher // may be nil
Recommender Recommender // may be nil
LoginLimiter *LoginLimiter // may be nil — one is created if absent
// InstallID returns the current plugin install ID for building
// host-proxy-routable URLs. Defaults to "silo.audiobooks" when nil.
InstallID func() string
@@ -329,13 +339,17 @@ func (h *Handler) mountRoutes(r chi.Router) {
r.Get(prefix+"/status", h.handleABSStatus)
}
// Stage 2: login (body credentials).
r.Post("/login", h.handleLogin)
r.Post("/abs/api/login", h.handleLogin)
// Token rotation — mobile clients call this every ~22h to avoid the
// 24h access-token interactive re-login trap.
r.Post("/auth/refresh", h.handleRefresh)
r.Post("/abs/api/auth/refresh", h.handleRefresh)
// Stage 2: login (body credentials). Real ABS serves /login at root, but
// clients differ on the prefix — some POST /api/login or /abs/api/login.
// The rest of the authenticated surface is mounted under both /api and
// /abs/api, so mount login+refresh under the same set; a client posting
// /api/login otherwise 404s and surfaces a generic "unknown error".
for _, prefix := range []string{"", "/api", "/abs/api"} {
r.Post(prefix+"/login", h.handleLogin)
// Token rotation — mobile clients call this every ~22h to avoid the
// 24h access-token interactive re-login trap.
r.Post(prefix+"/auth/refresh", h.handleRefresh)
}
// Logout is mounted OUTSIDE bearerAuth so an expired-access client can
// still sign out (the primary "sign out" UX moment). The handler parses
// the bearer locally, revokes the JTI if parseable, and always returns
@@ -414,10 +428,19 @@ func (h *Handler) mountRoutes(r chi.Router) {
// PATCH /me/progress/{id}/{episodeId} — podcast episode
// progress; audiobook-only catalog, so this is a stub.
r.Patch(prefix+"/me/progress/{libraryItemId}/{episodeId}", h.handleSetEpisodeProgress)
// PATCH /session/{sid} — heartbeat: position + time_listening
// POST /session/{sid}/sync — real ABS heartbeat path
// (SessionController.sync). The official ABS mobile/web clients
// POST here; missing it means playback progress never syncs.
r.Post(prefix+"/session/{sid}/sync", h.handleSessionSync)
// PATCH /session/{sid} — silo-native heartbeat alias
// (kept additive for silo's own clients).
r.Patch(prefix+"/session/{sid}", h.handleSessionSync)
// POST /session/{sid}/close — finalise the play session
r.Post(prefix+"/session/{sid}/close", h.handleSessionClose)
// POST /session/local — sync one offline-recorded session
r.Post(prefix+"/session/local", h.handleSyncLocalSession)
// POST /session/local-all — batch-sync offline-recorded sessions
r.Post(prefix+"/session/local-all", h.handleSyncLocalSessions)
// Bookmarks — POST/PATCH both upsert; DELETE is idempotent.
r.Post(prefix+"/me/item/{itemId}/bookmark", h.handleUpsertBookmark("bookmark_created"))
r.Patch(prefix+"/me/item/{itemId}/bookmark", h.handleUpsertBookmark("bookmark_updated"))
internal/audiobooks/abs/items_detail_test.go
+87
View File
@@ -0,0 +1,87 @@
package abs
import (
"encoding/json"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// TestSiloItemToLibraryItemDetail_ExpandedShape guards that GET /items/{id}
// matches real ABS LibraryItem.toOldJSONExpanded + Book.toOldJSONExpanded +
// oldMetadataToJSONExpanded: expanded outer keys, media.size + tracks, and the
// expanded metadata keys (authorName, descriptionPlain, ...).
func TestSiloItemToLibraryItemDetail_ExpandedShape(t *testing.T) {
item := &models.MediaItem{
ContentID: "book-7",
Title: "The Test",
Overview: "<p>Hello <b>world</b></p>",
People: []models.ItemPerson{
{Person: models.Person{ID: 5, Name: "Jane Roe"}, Kind: models.PersonKindAuthor},
{Person: models.Person{ID: 6, Name: "Ann Reader"}, Kind: models.PersonKindNarrator},
},
}
files := []*models.MediaFile{
{FilePath: "/x/part1.mp3", Duration: 120, FileSize: 4096},
}
detail := siloItemToLibraryItemDetail(item, files, AudiobookLibrary{ID: 1, Name: "Audiobooks"}, "http://x")
body, err := json.Marshal(detail)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var m map[string]any
if err := json.Unmarshal(body, &m); err != nil {
t.Fatalf("decode: %v", err)
}
outer := []string{
"id", "ino", "oldLibraryItemId", "libraryId", "folderId", "path",
"relPath", "isFile", "mtimeMs", "ctimeMs", "birthtimeMs", "addedAt",
"updatedAt", "lastScan", "scanVersion", "isMissing", "isInvalid",
"mediaType", "media", "libraryFiles", "size",
}
for _, k := range outer {
if _, ok := m[k]; !ok {
t.Errorf("expanded item missing outer key %q", k)
}
}
if lf, ok := m["libraryFiles"].([]any); !ok || len(lf) != 1 {
t.Errorf("libraryFiles = %v, want 1 entry", m["libraryFiles"])
}
if sz, _ := m["size"].(float64); sz != 4096 {
t.Errorf("size = %v, want 4096", m["size"])
}
media, _ := m["media"].(map[string]any)
for _, k := range []string{"id", "libraryItemId", "metadata", "coverPath", "tags", "audioFiles", "chapters", "duration", "size", "tracks"} {
if _, ok := media[k]; !ok {
t.Errorf("expanded media missing key %q", k)
}
}
if media["id"] != "book-7" {
t.Errorf("media.id = %v, want book-7", media["id"])
}
meta, _ := media["metadata"].(map[string]any)
for _, k := range []string{
"title", "titleIgnorePrefix", "subtitle", "authors", "authorName",
"authorNameLF", "narrators", "narratorName", "series", "seriesName",
"genres", "publishedYear", "publishedDate", "publisher", "description",
"descriptionPlain", "isbn", "asin", "language", "explicit", "abridged",
} {
if _, ok := meta[k]; !ok {
t.Errorf("expanded metadata missing key %q", k)
}
}
if meta["authorName"] != "Jane Roe" {
t.Errorf("authorName = %v, want Jane Roe", meta["authorName"])
}
if meta["narratorName"] != "Ann Reader" {
t.Errorf("narratorName = %v, want Ann Reader", meta["narratorName"])
}
// descriptionPlain strips HTML tags.
if dp, _ := meta["descriptionPlain"].(string); dp != "Hello world" {
t.Errorf("descriptionPlain = %q, want %q", dp, "Hello world")
}
}
internal/audiobooks/abs/items_handler.go
+24 -22
View File
@@ -2,6 +2,7 @@ package abs
import (
"context"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
@@ -126,9 +127,14 @@ func (h *Handler) handleSimilarItems(w http.ResponseWriter, r *http.Request) {
// handleItemsInProgress — GET /abs/api/me/items-in-progress
//
// Returns the Continue Listening shelf. Queries the ProgressStore for in-
// progress rows, then hydrates each with a summary LibraryItem from the
// catalog. Items without a matching catalog entry are skipped silently.
// Matches server/controllers/MeController.js `getAllLibraryItemsInProgress`:
// the envelope is `{ libraryItems: [...] }` and each entry is the item's
// `toOldJSONMinified()` shape spread with a flat `progressLastUpdate` (ms)
// field — real ABS does NOT wrap progress in a nested `userMediaProgress`
// object for this endpoint (that shape belongs to other responses, e.g.
// item-detail). Queries the ProgressStore for in-progress rows, then
// hydrates each with a minified LibraryItem from the catalog. Items
// without a matching catalog entry are skipped silently.
func (h *Handler) handleItemsInProgress(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
@@ -174,25 +180,21 @@ func (h *Handler) handleItemsInProgress(w http.ResponseWriter, r *http.Request)
if si == nil {
continue
}
li := siloItemToLibraryItem(si, lib, baseURL)
items = append(items, map[string]any{
"id": li.ID,
"libraryId": li.LibraryID,
"folderId": li.FolderID,
"mediaType": li.MediaType,
"media": li.Media,
"numTracks": li.NumTracks,
"addedAt": li.AddedAt,
"updatedAt": li.UpdatedAt,
"userMediaProgress": map[string]any{
"id": a.UserID + "-" + p.ContentID,
"libraryItemId": p.ContentID,
"currentTime": p.CurrentSeconds,
"progress": p.ProgressPct,
"isFinished": p.IsFinished,
"lastUpdate": p.UpdatedAt.UnixMilli(),
},
})
mli := Minify(siloItemToLibraryItem(si, lib, baseURL))
wire := minifiedItemToWireMap(mli)
wire["progressLastUpdate"] = p.UpdatedAt.UnixMilli()
items = append(items, wire)
}
writeJSON(w, http.StatusOK, map[string]any{"libraryItems": items})
}
// minifiedItemToWireMap reuses the json tags on MinifiedLibraryItem so a
// caller can merge extra keys (e.g. progressLastUpdate) into it inside a
// heterogeneous map[string]any envelope, mirroring the spread-operator
// pattern real ABS uses (`{ ...libraryItem.toOldJSONMinified(), ... }`).
func minifiedItemToWireMap(mli MinifiedLibraryItem) map[string]any {
b, _ := json.Marshal(mli)
var m map[string]any
_ = json.Unmarshal(b, &m)
return m
}
internal/audiobooks/abs/items_in_progress_test.go
+136
View File
@@ -0,0 +1,136 @@
package abs
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
// inProgressStubMediaStore backs the /me/items-in-progress shape tests.
type inProgressStubMediaStore struct {
noopMediaStore
libs []AudiobookLibrary
byID map[string]*models.MediaItem
}
func (s *inProgressStubMediaStore) ListAudiobookLibraries(context.Context, catalog.AccessFilter) ([]AudiobookLibrary, error) {
return s.libs, nil
}
func (s *inProgressStubMediaStore) GetAudiobooksByIDs(_ context.Context, ids []string, _ catalog.AccessFilter) (map[string]*models.MediaItem, error) {
out := make(map[string]*models.MediaItem, len(ids))
for _, id := range ids {
if it, ok := s.byID[id]; ok {
out[id] = it
}
}
return out, nil
}
// inProgressFakeProgressStore returns a fixed set of progress rows.
type inProgressFakeProgressStore struct {
fakeProgressStore
rows []ProgressRow
}
func (f *inProgressFakeProgressStore) ListProgressForAudiobooks(context.Context, string, string, int) ([]ProgressRow, error) {
return f.rows, nil
}
// TestItemsInProgress_EnvelopeAndItemShape asserts the response matches
// real ABS MeController.getAllLibraryItemsInProgress: envelope key
// "libraryItems", each entry is the minified library item spread with a
// flat "progressLastUpdate" field — no nested "userMediaProgress" object.
func TestItemsInProgress_EnvelopeAndItemShape(t *testing.T) {
updatedAt := time.Now()
media := &inProgressStubMediaStore{
libs: []AudiobookLibrary{{ID: 1, Name: "Audiobooks", Type: "audiobooks"}},
byID: map[string]*models.MediaItem{
"book-1": {ContentID: "book-1", Title: "In Progress Book"},
},
}
progress := &inProgressFakeProgressStore{
rows: []ProgressRow{
{
UserID: "1",
ContentID: "book-1",
CurrentSeconds: 120,
ProgressPct: 0.25,
IsFinished: false,
UpdatedAt: updatedAt,
},
},
}
h := New(Dependencies{MediaStore: media, ProgressStore: progress})
rec := dispatchABSWithParams(http.MethodGet, "/api/me/items-in-progress", nil, nil, "1", "", h.handleItemsInProgress)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
items, ok := got["libraryItems"].([]any)
if !ok || len(items) != 1 {
t.Fatalf("libraryItems = %v, want 1 entry", got["libraryItems"])
}
entry, ok := items[0].(map[string]any)
if !ok {
t.Fatalf("entry not an object: %v", items[0])
}
if entry["id"] != "book-1" {
t.Errorf("id = %v, want book-1", entry["id"])
}
if _, hasMedia := entry["media"]; !hasMedia {
t.Errorf("entry missing minified 'media' key: %v", entry)
}
lastUpdate, ok := entry["progressLastUpdate"].(float64)
if !ok {
t.Fatalf("entry missing progressLastUpdate: %v", entry)
}
if int64(lastUpdate) != updatedAt.UnixMilli() {
t.Errorf("progressLastUpdate = %v, want %v", int64(lastUpdate), updatedAt.UnixMilli())
}
if _, hasWrapper := entry["userMediaProgress"]; hasWrapper {
t.Errorf("entry has userMediaProgress wrapper, real ABS flattens progress instead: %v", entry)
}
}
// TestItemsInProgress_NoProgressStore_ReturnsEmptyEnvelope covers the
// no-store-configured fallback.
func TestItemsInProgress_NoProgressStore_ReturnsEmptyEnvelope(t *testing.T) {
h := New(Dependencies{MediaStore: &inProgressStubMediaStore{}})
rec := dispatchABSWithParams(http.MethodGet, "/api/me/items-in-progress", nil, nil, "1", "", h.handleItemsInProgress)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d", rec.Code)
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
items, ok := got["libraryItems"].([]any)
if !ok || len(items) != 0 {
t.Fatalf("libraryItems = %v, want empty array", got["libraryItems"])
}
}
// TestItemsInProgress_Unauthenticated_401 covers the auth guard: no
// ctxAuth in the request context (bearerAuth middleware never ran).
func TestItemsInProgress_Unauthenticated_401(t *testing.T) {
h := New(Dependencies{MediaStore: &inProgressStubMediaStore{}})
req := httptest.NewRequest(http.MethodGet, "/api/me/items-in-progress", nil)
rec := httptest.NewRecorder()
h.handleItemsInProgress(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Errorf("status = %d, want 401", rec.Code)
}
}
internal/audiobooks/abs/libraries_handler.go
+269 -145
View File
@@ -44,21 +44,25 @@ func (h *Handler) handleLibraryDetail(w http.ResponseWriter, r *http.Request) {
if !ok {
return
}
resp := map[string]any{
"library": audiobookLibraryMap(lib),
library := audiobookLibraryMap(lib)
// Real ABS LibraryController.findOne returns the library object DIRECTLY
// when there is no ?include=filterdata; only the filterdata request wraps
// it in { filterdata, issues, numUserPlaylists, customMetadataProviders,
// library }. Returning the wrapped shape unconditionally breaks clients
// that read library fields off the top level.
if !includeHas(r.URL.Query().Get("include"), "filterdata") {
writeJSON(w, http.StatusOK, library)
return
}
if includeHas(r.URL.Query().Get("include"), "filterdata") {
resp["filterdata"] = h.buildFilterData(r, lib)
resp["issues"] = 0
// numUserPlaylists drives the bottom-nav "Playlists" tab
// visibility on the ABS mobile client (BookshelfNavBar.vue:25
// gates the tab on `numUserPlaylists` being truthy). Comment
// in plugins/server.js:129 confirms "precise number is not
// necessary" — we just need a non-zero count when the caller
// has any playlists, so the ListUserPlaylists len suffices.
resp["numUserPlaylists"] = h.countUserPlaylists(r)
}
writeJSON(w, http.StatusOK, resp)
// numUserPlaylists drives the bottom-nav "Playlists" tab visibility on the
// ABS mobile client (BookshelfNavBar.vue gates the tab on it being truthy).
writeJSON(w, http.StatusOK, map[string]any{
"filterdata": h.buildFilterData(r, lib),
"issues": 0,
"numUserPlaylists": h.countUserPlaylists(r),
"customMetadataProviders": []any{},
"library": library,
})
}
// countUserPlaylists returns the playlist count for the authenticated
@@ -99,14 +103,14 @@ func (h *Handler) buildFilterData(r *http.Request, lib AudiobookLibrary) map[str
access, _, _ := h.accessFilterFromRequest(r)
authorObjs := []AuthorObj{}
if rows, err := h.deps.MediaStore.ListLibraryAuthors(ctx, lib.ID, fetchCap, access); err == nil {
if rows, _, err := h.deps.MediaStore.ListLibraryAuthors(ctx, lib.ID, fetchCap, 0, "name", false, access); err == nil {
for _, a := range rows {
authorObjs = append(authorObjs, AuthorObj{ID: a.ID, Name: a.Name})
}
}
seriesObjs := []SeriesObj{}
if rows, err := h.deps.MediaStore.ListLibrarySeries(ctx, lib.ID, fetchCap, access); err == nil {
if rows, _, err := h.deps.MediaStore.ListLibrarySeries(ctx, lib.ID, fetchCap, 0, access); err == nil {
for _, s := range rows {
seriesObjs = append(seriesObjs, SeriesObj{ID: s.ID, Name: s.Name})
}
@@ -152,7 +156,11 @@ func (h *Handler) handleLibraryItems(w http.ResponseWriter, r *http.Request) {
sortBy := q.Get("sort")
sortDesc := q.Get("desc") == "1"
filterBy := q.Get("filter")
minified := q.Get("minified") == "1"
// Real ABS getByFilterAndSort ALWAYS serializes list items minified
// (LibraryItem.toOldJSONMinified); the non-minified hybrid is a shape no
// real client requests. Default to minified; only an explicit minified=0
// opts into the full shape.
minified := q.Get("minified") != "0"
collapseSeries := q.Get("collapseseries") == "1"
include := q.Get("include")
@@ -164,18 +172,31 @@ func (h *Handler) handleLibraryItems(w http.ResponseWriter, r *http.Request) {
return
}
// Fetch the full visible library when filtering so local post-filter
// cannot truncate candidates before applying the predicate.
fetchLimit := limit
if hasFilter || collapseSeries || limit == 0 {
fetchLimit = 0
// authors/series/narrators filters push down into SQL (indexed) so we never
// load + hydrate the whole library. This applies even with collapseseries=1
// (the client's per-artist album sync): the SQL filter reduces to a handful
// of rows, then collapse + paging run in Go over that small set. Only
// progress/genre/tag/language filters still need the Go post-filter and the
// full fetch.
pushDown := hasFilter &&
(filter.Kind == FilterAuthors || filter.Kind == FilterSeries || filter.Kind == FilterNarrators)
sqlFilter := Filter{}
if pushDown {
sqlFilter = filter
}
fetchOffset := 0
if !hasFilter && !collapseSeries && limit > 0 {
goFilter := hasFilter && !pushDown
// SQL paginates only when nothing is post-processed in Go (no Go filter, no
// collapse) and the limit is positive; otherwise fetch the (now
// SQL-filtered, hence small) candidate set in full and slice in Go.
sqlPaginated := !goFilter && !collapseSeries && limit > 0
fetchLimit, fetchOffset := 0, 0
if sqlPaginated {
fetchLimit = limit
fetchOffset = page * limit
}
items, total, err := h.deps.MediaStore.ListAudiobooks(r.Context(), lib.ID, fetchLimit, fetchOffset, access)
items, total, err := h.deps.MediaStore.ListAudiobooks(r.Context(), lib.ID, fetchLimit, fetchOffset, access, sqlFilter)
if err != nil {
http.Error(w, "list audiobooks: "+err.Error(), http.StatusInternalServerError)
return
@@ -188,8 +209,8 @@ func (h *Handler) handleLibraryItems(w http.ResponseWriter, r *http.Request) {
all = append(all, siloItemToLibraryItem(item, lib, baseURL))
}
// Local filter (post-fetch).
if hasFilter {
// Local filter (post-fetch) — only for filters not pushed into SQL.
if goFilter {
filtered := make([]LibraryItem, 0, len(all))
for _, it := range all {
if filter.Matches(it, false, false, false) {
@@ -207,9 +228,9 @@ func (h *Handler) handleLibraryItems(w http.ResponseWriter, r *http.Request) {
total = len(collapsed)
}
// Slice for page/limit.
// Slice for page/limit. When SQL already paginated we serve the rows as-is.
pageStart, pageEnd := 0, len(collapsed)
if limit > 0 && (hasFilter || collapseSeries) {
if limit > 0 && !sqlPaginated {
pageStart = page * limit
if pageStart > len(collapsed) {
pageStart = len(collapsed)
@@ -313,48 +334,40 @@ func (h *Handler) handleLibraryAuthors(w http.ResponseWriter, r *http.Request) {
return
}
limit, page := readPagedQuery(r, 50)
// Fetch the full list (capped at 5000) and paginate locally so the
// envelope's total reflects real DB count, not the page slice length.
// ABS clients use total to decide whether to fetch page 2.
const fetchCap = 5000
sortBy := r.URL.Query().Get("sort")
sortDesc := r.URL.Query().Get("desc") == "1"
access, _, err := h.accessFilterFromRequest(r)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
authors, err := h.deps.MediaStore.ListLibraryAuthors(r.Context(), lib.ID, fetchCap, access)
// Reads the precomputed author materialized view: indexed paginated read +
// trivial count, so large libraries aren't capped and full syncs don't blow
// the client's background-task window. limit=0 means "return all".
offset := 0
if limit > 0 {
offset = page * limit
}
pageAuthors, total, err := h.deps.MediaStore.ListLibraryAuthors(r.Context(), lib.ID, limit, offset, sortBy, sortDesc, access)
if err != nil {
http.Error(w, "list authors: "+err.Error(), http.StatusInternalServerError)
return
}
libID := audiobookLibraryID(lib)
total := len(authors)
// Local slice for the requested page.
// ABS contract: limit=0 means "return all".
var pageAuthors []AuthorSummary
if limit == 0 {
pageAuthors = authors
} else {
start := page * limit
end := start + limit
if start > total {
start = total
}
if end > total {
end = total
}
pageAuthors = authors[start:end]
}
results := make([]map[string]any, 0, len(pageAuthors))
for _, a := range pageAuthors {
results = append(results, map[string]any{
"id": a.ID,
"name": a.Name,
"numBooks": a.NumBooks,
"libraryId": libID,
})
results = append(results, authorObjectABS(a.ID, a.Name, libID, a.NumBooks))
}
writeJSON(w, http.StatusOK, pagedEnvelope(results, total, limit, page, "name", false, "", false, ""))
// Real ABS LibraryController.getAuthors branches on isPaginated =
// (limit present & numeric) && (page present & numeric): paged envelope
// when true, else a bare { authors: [...] }. Emitting the paged shape for
// the non-paginated request crashes clients that key on `authors`.
q := r.URL.Query()
if q.Get("limit") != "" && q.Get("page") != "" {
writeJSON(w, http.StatusOK, pagedEnvelope(results, total, limit, page, "name", false, "", false, ""))
return
}
writeJSON(w, http.StatusOK, map[string]any{"authors": results})
}
// handleLibrarySeries — GET /abs/api/libraries/{id}/series
@@ -368,78 +381,56 @@ func (h *Handler) handleLibrarySeries(w http.ResponseWriter, r *http.Request) {
return
}
limit, page := readPagedQuery(r, 25)
const fetchCap = 5000
access, _, err := h.accessFilterFromRequest(r)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
series, err := h.deps.MediaStore.ListLibrarySeries(r.Context(), lib.ID, fetchCap, access)
// Paginate in SQL with a separate COUNT so large libraries aren't truncated
// at a fixed cap. limit=0 means "return all" (ABS contract).
offset := 0
if limit > 0 {
offset = page * limit
}
pageSeries, total, err := h.deps.MediaStore.ListLibrarySeries(r.Context(), lib.ID, limit, offset, access)
if err != nil {
http.Error(w, "list series: "+err.Error(), http.StatusInternalServerError)
return
}
libID := audiobookLibraryID(lib)
baseURL := h.absBaseURL(r)
total := len(series)
// ABS contract: limit=0 means "return all".
var pageSeries []SeriesSummary
if limit == 0 {
pageSeries = series
} else {
start := page * limit
end := start + limit
if start > total {
start = total
}
if end > total {
end = total
}
pageSeries = series[start:end]
}
results := make([]map[string]any, 0, len(pageSeries))
for _, s := range pageSeries {
// books[] is what LazySeriesCard reads to populate the
// GroupCover stack. Each entry is a minified LibraryItem with
// the cover URL on media.coverPath; the mobile client's
// globals/getLibraryItemCoverSrc getter requires this field
// to render any cover image, otherwise the card falls back
// to a name-only placeholder.
books := make([]map[string]any, 0, len(s.Books))
// books[] is what LazySeriesCard reads to populate the GroupCover
// stack. Real ABS emits FULL minified library items here; a thin stub
// crashes strict clients (Plappa) on the first missing required key.
books := make([]MinifiedLibraryItem, 0, len(s.Books))
for _, bp := range s.Books {
updatedMs := int64(0)
if !bp.UpdatedAt.IsZero() {
updatedMs = bp.UpdatedAt.UnixMilli()
}
books = append(books, map[string]any{
"id": bp.ContentID,
"libraryId": libID,
"mediaType": LibraryMediaType,
"updatedAt": updatedMs,
"media": map[string]any{
"coverPath": baseURL + "/api/items/" + bp.ContentID + "/cover",
"metadata": map[string]any{"title": bp.Title},
},
})
books = append(books, seriesBookMinified(bp.ContentID, bp.Title, libID, baseURL, updatedMs))
}
results = append(results, map[string]any{
"id": s.ID,
"name": s.Name,
"numBooks": s.NumBooks,
"libraryId": libID,
"addedAt": 0,
"books": books,
})
obj := seriesObjectABS(s.ID, s.Name, libID, s.NumBooks)
obj["books"] = books
results = append(results, obj)
}
writeJSON(w, http.StatusOK, pagedEnvelope(results, total, limit, page, "name", false, "", false, ""))
}
// handleLibrarySearch — GET /abs/api/libraries/{id}/search?q=…&limit=…
// Returns matching books grouped under "book", with empty arrays for the
// other ABS-standard buckets (podcast, series, authors, tags). Bucket
// names match continuum-plugin-audiobooks exactly: note "authors" plural,
// not "author" — ABS mobile clients key off the plural form and a
// singular bucket is silently dropped.
//
// Matches server/utils/queries/libraryItemsBookFilters.js `search()` (real
// ABS branches to the book-filter search for a non-podcast library, which
// is all Silo ever serves). That function returns exactly these keys:
// book, narrators, tags, genres, series, authors — there is NO "podcast"
// key for a book-library search (that only appears from the separate
// podcast-filter branch). Each book entry is `{ libraryItem }` — real ABS
// does not include matchKey/matchText on book entries (those only exist
// on the interactive-search HTML autocomplete, not this JSON endpoint).
// We keep an extra empty "podcast" bucket anyway since an extra key never
// crashes a strict client, only a missing one does.
func (h *Handler) handleLibrarySearch(w http.ResponseWriter, r *http.Request) {
lib, ok := h.resolveLibrary(w, r)
if !ok {
@@ -451,11 +442,13 @@ func (h *Handler) handleLibrarySearch(w http.ResponseWriter, r *http.Request) {
limit = n
}
empty := map[string]any{
"book": []any{},
"podcast": []any{},
"series": []any{},
"authors": []any{},
"tags": []any{},
"book": []any{},
"podcast": []any{},
"narrators": []any{},
"tags": []any{},
"genres": []any{},
"series": []any{},
"authors": []any{},
}
if q == "" {
writeJSON(w, http.StatusOK, empty)
@@ -472,16 +465,89 @@ func (h *Handler) handleLibrarySearch(w http.ResponseWriter, r *http.Request) {
return
}
baseURL := h.absBaseURL(r)
libID := audiobookLibraryID(lib)
books := make([]map[string]any, 0, len(items))
for _, it := range items {
books = append(books, map[string]any{
"libraryItem": siloItemToLibraryItem(it, lib, baseURL),
"matchKey": "title",
"matchText": it.Title,
})
}
// Best-effort author/series buckets: silo has no dedicated search-scoped
// store query for these yet, so we reuse the existing aggregate listers
// (capped, same pattern as buildFilterData/handleLibraryAuthors/
// handleLibrarySeries) and filter client-side on a case-insensitive
// substring match. narrators/tags/genres have no backing aggregation
// query at all in silo's catalog today and stay empty-but-present.
qLower := strings.ToLower(q)
const fetchCap = 5000
authorsOut := []any{}
if rows, _, err := h.deps.MediaStore.ListLibraryAuthors(r.Context(), lib.ID, fetchCap, 0, "name", false, access); err == nil {
for _, a := range rows {
if !strings.Contains(strings.ToLower(a.Name), qLower) {
continue
}
authorsOut = append(authorsOut, map[string]any{
"id": a.ID,
"name": a.Name,
"numBooks": a.NumBooks,
"libraryId": libID,
})
if len(authorsOut) >= limit {
break
}
}
}
seriesOut := []any{}
if rows, _, err := h.deps.MediaStore.ListLibrarySeries(r.Context(), lib.ID, fetchCap, 0, access); err == nil {
for _, s := range rows {
if !strings.Contains(strings.ToLower(s.Name), qLower) {
continue
}
// Real ABS wraps series search hits as { series, books } — the
// series sub-object is the plain Series.toOldJSON() shape (no
// numBooks field there; we add it anyway since an extra key is
// harmless), matched here with the same per-book thin map
// handleLibrarySeries above uses for its books[] entries.
seriesBooks := make([]map[string]any, 0, len(s.Books))
for _, bp := range s.Books {
updatedMs := int64(0)
if !bp.UpdatedAt.IsZero() {
updatedMs = bp.UpdatedAt.UnixMilli()
}
seriesBooks = append(seriesBooks, map[string]any{
"id": bp.ContentID,
"libraryId": libID,
"mediaType": LibraryMediaType,
"updatedAt": updatedMs,
"media": map[string]any{
"coverPath": baseURL + "/api/items/" + bp.ContentID + "/cover",
"metadata": map[string]any{"title": bp.Title},
},
})
}
seriesOut = append(seriesOut, map[string]any{
"series": map[string]any{
"id": s.ID,
"name": s.Name,
"numBooks": s.NumBooks,
"libraryId": libID,
"addedAt": 0,
},
"books": seriesBooks,
})
if len(seriesOut) >= limit {
break
}
}
}
out := empty
out["book"] = books
out["authors"] = authorsOut
out["series"] = seriesOut
writeJSON(w, http.StatusOK, out)
}
@@ -535,16 +601,23 @@ func (h *Handler) handlePersonalized(w http.ResponseWriter, r *http.Request) {
}
libID := audiobookLibraryID(lib)
if series, err := h.deps.MediaStore.ListLibrarySeries(r.Context(), lib.ID, shelfLimit, access); err == nil && len(series) > 0 {
if series, _, err := h.deps.MediaStore.ListLibrarySeries(r.Context(), lib.ID, shelfLimit, 0, access); err == nil && len(series) > 0 {
recent := make([]map[string]any, 0, len(series))
for _, s := range series {
recent = append(recent, map[string]any{
"id": s.ID,
"name": s.Name,
"numBooks": s.NumBooks,
"libraryId": libID,
"books": []any{},
})
// Full real-ABS series object + minified books (same shape as
// /libraries/{id}/series) so the recent-series shelf card decodes
// identically and its cover stack has real items.
obj := seriesObjectABS(s.ID, s.Name, libID, s.NumBooks)
books := make([]MinifiedLibraryItem, 0, len(s.Books))
for _, bp := range s.Books {
updatedMs := int64(0)
if !bp.UpdatedAt.IsZero() {
updatedMs = bp.UpdatedAt.UnixMilli()
}
books = append(books, seriesBookMinified(bp.ContentID, bp.Title, libID, baseURL, updatedMs))
}
obj["books"] = books
recent = append(recent, obj)
}
shelves[3]["entities"] = recent
shelves[3]["total"] = len(recent)
@@ -652,22 +725,28 @@ func siloItemToLibraryItem(item *models.MediaItem, lib AudiobookLibrary, baseURL
FolderID: VirtualFolderID,
Path: "",
RelPath: "",
IsFile: true,
MtimeMs: addedAtMs,
CtimeMs: addedAtMs,
BirthtimeMs: addedAtMs,
MediaType: LibraryMediaType,
Media: LibraryItemMedia{
Metadata: meta,
Duration: duration,
CoverPath: coverPath,
AudioFiles: []AudioTrack{},
Tracks: []AudioTrack{},
Chapters: []ChapterABS{},
NumTracks: 0, // populated by item-detail handler
Tags: []string{},
ID: item.ContentID,
LibraryItemID: item.ContentID,
Metadata: meta,
Duration: duration,
CoverPath: coverPath,
AudioFiles: []AudioTrack{},
Tracks: []AudioTrack{},
Chapters: []ChapterABS{},
NumTracks: 0, // populated by item-detail handler
Tags: []string{},
},
AddedAt: addedAtMs,
UpdatedAt: updatedAtMs,
LibraryFiles: []map[string]any{}, // populated by item-detail handler
LastScan: addedAtMs,
ScanVersion: ServerVersion,
AddedAt: addedAtMs,
UpdatedAt: updatedAtMs,
}
}
@@ -681,6 +760,8 @@ func siloItemToLibraryItem(item *models.MediaItem, lib AudiobookLibrary, baseURL
func siloItemToMetadata(item *models.MediaItem) Metadata {
authors := make([]AuthorObj, 0)
narrators := make([]string, 0)
authorNames := make([]string, 0)
lfNames := make([]string, 0)
for _, p := range item.People {
switch p.Kind {
@@ -689,12 +770,15 @@ func siloItemToMetadata(item *models.MediaItem) Metadata {
ID: strconv.FormatInt(p.ID, 10),
Name: p.Name,
})
authorNames = append(authorNames, p.Name)
lfNames = append(lfNames, lastFirst(p.Name))
case models.PersonKindNarrator:
narrators = append(narrators, p.Name)
}
}
series := make([]SeriesObj, 0, len(item.AudiobookSeries))
seriesName := ""
for _, membership := range item.AudiobookSeries {
name := strings.TrimSpace(membership.Name)
if name == "" {
@@ -705,6 +789,12 @@ func siloItemToMetadata(item *models.MediaItem) Metadata {
obj.Sequence = strconv.FormatFloat(*membership.Index, 'f', -1, 64)
}
series = append(series, obj)
if seriesName == "" {
seriesName = name
if obj.Sequence != "" {
seriesName += " #" + obj.Sequence
}
}
}
publishedYear := ""
@@ -727,15 +817,22 @@ func siloItemToMetadata(item *models.MediaItem) Metadata {
}
return Metadata{
Title: item.Title,
Authors: authors,
Narrators: narrators,
Series: series,
Description: item.Overview,
PublishedYear: publishedYear,
Publisher: publisher,
Genres: genres,
Tags: tags,
Title: item.Title,
TitleIgnorePrefix: titleIgnorePrefix(item.Title),
Authors: authors,
AuthorName: strings.Join(authorNames, ", "),
AuthorNameLF: strings.Join(lfNames, ", "),
Narrators: narrators,
NarratorName: strings.Join(narrators, ", "),
Series: series,
SeriesName: seriesName,
Description: item.Overview,
DescriptionPlain: stripHTML(item.Overview),
PublishedYear: publishedYear,
Publisher: publisher,
Genres: genres,
Language: "en",
Tags: tags,
}
}
@@ -747,12 +844,16 @@ func siloItemToLibraryItemDetail(item *models.MediaItem, files []*models.MediaFi
tracks := siloFilesToAudioTracks(item.ContentID, files, baseURL, "")
// Recompute duration from files if the item's Runtime is zero.
totalDuration := base.Media.Duration
// media.duration is the summed track duration (real ABS: sum of audio file
// durations), NOT the item's Runtime — Runtime is often stale/mis-scanned
// (e.g. 222s for a 3.7h book), which desyncs the player's scrubber. Fall
// back to Runtime only when there are no tracks to sum.
totalDuration := float64(0)
for _, t := range tracks {
totalDuration += t.Duration
}
if totalDuration == 0 {
for _, t := range tracks {
totalDuration += t.Duration
}
totalDuration = base.Media.Duration
}
// Chapters from the first file that has them.
@@ -771,12 +872,34 @@ func siloItemToLibraryItemDetail(item *models.MediaItem, files []*models.MediaFi
}
}
// libraryFiles + summed size mirror real ABS toOldJSONExpanded. Each entry
// is the real-ABS library file shape (ino + file metadata + fileType).
nowMs := time.Now().UnixMilli()
libraryFiles := make([]map[string]any, 0, len(tracks))
var totalSize int64
for _, t := range tracks {
if t.Metadata != nil {
totalSize += t.Metadata.Size
}
libraryFiles = append(libraryFiles, map[string]any{
"ino": t.Ino,
"metadata": t.Metadata,
"isSupplementary": false,
"addedAt": nowMs,
"updatedAt": nowMs,
"fileType": "audio",
})
}
base.Media.AudioFiles = tracks
base.Media.Tracks = tracks
base.Media.Chapters = chapters
base.Media.NumTracks = len(tracks)
base.Media.Duration = totalDuration
base.Media.Size = totalSize
base.NumTracks = len(tracks)
base.LibraryFiles = libraryFiles
base.Size = totalSize
return base
}
@@ -843,6 +966,7 @@ func siloFilesToAudioTracks(contentID string, files []*models.MediaFile, baseURL
TimeBase: "1/14112000",
Channels: channels,
ChannelLayout: channelLayout,
Chapters: []ChapterABS{},
EmbeddedCoverArt: nil,
MetaTags: map[string]string{},
MimeType: mimeType,
internal/audiobooks/abs/libraries_shape_test.go
+113
View File
@@ -0,0 +1,113 @@
package abs
import (
"context"
"encoding/json"
"net/http"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
)
type libStub struct {
noopMediaStore
libs []AudiobookLibrary
}
func (s *libStub) ListAudiobookLibraries(_ context.Context, _ catalog.AccessFilter) ([]AudiobookLibrary, error) {
return s.libs, nil
}
// TestAudiobookLibraryMap_FullShape asserts the library object matches real
// ABS Library.toOldJSON (12 keys) — a strict client decodes the library model
// and crashes on any missing key.
func TestAudiobookLibraryMap_FullShape(t *testing.T) {
body, _ := json.Marshal(audiobookLibraryMap(AudiobookLibrary{ID: 18, Name: "Audiobooks"}))
var m map[string]any
if err := json.Unmarshal(body, &m); err != nil {
t.Fatalf("decode: %v", err)
}
for _, k := range []string{
"id", "name", "folders", "displayOrder", "icon", "mediaType",
"provider", "settings", "lastScan", "lastScanVersion", "createdAt", "lastUpdate",
} {
if _, ok := m[k]; !ok {
t.Errorf("library object missing key %q", k)
}
}
folders, _ := m["folders"].([]any)
if len(folders) != 1 {
t.Fatalf("folders len = %d, want 1", len(folders))
}
f0, _ := folders[0].(map[string]any)
for _, k := range []string{"id", "fullPath", "libraryId", "addedAt"} {
if _, ok := f0[k]; !ok {
t.Errorf("folder missing key %q", k)
}
}
}
// TestLibraryDetail_UnwrappedWithoutInclude: real ABS findOne returns the
// library object directly (not { library: ... }) when no include=filterdata.
func TestLibraryDetail_UnwrappedWithoutInclude(t *testing.T) {
media := &libStub{libs: []AudiobookLibrary{{ID: 18, Name: "Audiobooks", Type: "audiobooks"}}}
h := New(Dependencies{MediaStore: media})
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/18", map[string]string{"libraryId": "18"}, nil, "1", "", h.handleLibraryDetail)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
if _, wrapped := got["library"]; wrapped {
t.Errorf("response must be unwrapped (no 'library' key) without include")
}
if got["id"] != "18" {
t.Errorf("id = %v, want 18 (unwrapped library object)", got["id"])
}
if _, ok := got["folders"]; !ok {
t.Errorf("unwrapped library missing 'folders'")
}
}
// TestLibraryDetail_WrappedWithInclude: with include=filterdata the response
// wraps in { filterdata, issues, numUserPlaylists, customMetadataProviders, library }.
func TestLibraryDetail_WrappedWithInclude(t *testing.T) {
media := &libStub{libs: []AudiobookLibrary{{ID: 18, Name: "Audiobooks", Type: "audiobooks"}}}
h := New(Dependencies{MediaStore: media})
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/18?include=filterdata", map[string]string{"libraryId": "18"}, nil, "1", "", h.handleLibraryDetail)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
for _, k := range []string{"filterdata", "issues", "numUserPlaylists", "customMetadataProviders", "library"} {
if _, ok := got[k]; !ok {
t.Errorf("include=filterdata response missing key %q", k)
}
}
}
// TestLibraries_WrappedEnvelope: GET /libraries returns { libraries: [...] }.
func TestLibraries_WrappedEnvelope(t *testing.T) {
media := &libStub{libs: []AudiobookLibrary{{ID: 18, Name: "Audiobooks", Type: "audiobooks"}}}
h := New(Dependencies{MediaStore: media})
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries", nil, nil, "1", "", h.handleLibraries)
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
libs, ok := got["libraries"].([]any)
if !ok {
t.Fatalf("missing 'libraries' array; keys %v", keysOf(got))
}
if len(libs) != 1 {
t.Errorf("libraries len = %d, want 1", len(libs))
}
}
internal/audiobooks/abs/listening_stats_handler.go
+150 -9
View File
@@ -6,6 +6,8 @@ import (
"strconv"
"github.com/go-chi/chi/v5"
"github.com/Silo-Server/silo-server/internal/models"
)
func (h *Handler) handleListeningStats(w http.ResponseWriter, r *http.Request) {
@@ -27,30 +29,67 @@ func (h *Handler) handleListeningStats(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, statsToABS(stats))
}
// handleListeningSessions — GET /me/listening-sessions
//
// Real ABS (MeController.getListeningSessions) returns
// { total, numPages, page, itemsPerPage, sessions } — NOT the generic
// pagedEnvelope shape used by browse endpoints (results/sortBy/filterBy).
// Each entry in `sessions` must match PlaybackSession.toJSON() key-for-key
// (server/objects/PlaybackSession.js upstream) or strict decoders
// (Flutter/Swift) throw keyNotFound on the first missing field.
func (h *Handler) handleListeningSessions(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
itemsPerPage, page := readPagedQuery(r, 10)
if h.deps.PlaybackSessionStore == nil {
writeJSON(w, http.StatusOK, pagedEnvelope([]any{}, 0, 30, 0, "started_at", true, "", false, ""))
writeJSON(w, http.StatusOK, listeningSessionsEnvelope([]map[string]any{}, 0, itemsPerPage, page))
return
}
limit, page := readPagedQuery(r, 30)
sessions, total, err := h.deps.PlaybackSessionStore.ListClosedSessions(r.Context(), a.UserID, a.ProfileID, limit, page*limit)
sessions, total, err := h.deps.PlaybackSessionStore.ListClosedSessions(r.Context(), a.UserID, a.ProfileID, itemsPerPage, page*itemsPerPage)
if err != nil {
slog.Error("abs listening sessions failed", "err", err, "user", a.UserID)
http.Error(w, "sessions unavailable", http.StatusInternalServerError)
return
}
// Best-effort batch hydration of mediaMetadata/displayTitle/displayAuthor
// for every session's content item. A lookup failure (deleted item,
// access revoked, store error) must never crash the response — the
// session just falls back to a placeholder mediaMetadata shape built
// from a stub item so every key strict clients expect is still present.
items := map[string]*models.MediaItem{}
if h.deps.MediaStore != nil && len(sessions) > 0 {
access, aerr := h.accessFilterForAuth(r.Context(), a)
if aerr != nil {
slog.Debug("abs listening sessions: resolve access failed", "user", a.UserID, "err", aerr)
} else {
contentIDs := make([]string, 0, len(sessions))
for _, s := range sessions {
contentIDs = append(contentIDs, s.ContentID)
}
hydrated, herr := h.deps.MediaStore.GetAudiobooksByIDs(r.Context(), contentIDs, access)
if herr != nil {
slog.Debug("abs listening sessions: hydrate media items failed", "user", a.UserID, "err", herr)
} else {
items = hydrated
}
}
}
baseURL := h.absBaseURL(r)
out := make([]map[string]any, 0, len(sessions))
for _, s := range sessions {
out = append(out, sessionToABS(s))
out = append(out, sessionToABS(s, items[s.ContentID], baseURL))
}
writeJSON(w, http.StatusOK, pagedEnvelope(out, total, limit, page, "started_at", true, "", false, ""))
writeJSON(w, http.StatusOK, listeningSessionsEnvelope(out, total, itemsPerPage, page))
}
// handleListeningSessionDetail — GET /me/listening-sessions/{sid}
// Returns a single PlaybackSession.toJSON()-shaped object, not the thin
// 5-field object the prior implementation emitted.
func (h *Handler) handleListeningSessionDetail(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
@@ -67,7 +106,20 @@ func (h *Handler) handleListeningSessionDetail(w http.ResponseWriter, r *http.Re
http.Error(w, "session not found", http.StatusNotFound)
return
}
writeJSON(w, http.StatusOK, sessionToABS(sess))
var item *models.MediaItem
if h.deps.MediaStore != nil {
access, aerr := h.accessFilterForAuth(r.Context(), a)
if aerr != nil {
slog.Debug("abs listening session detail: resolve access failed", "user", a.UserID, "err", aerr)
} else if fetched, ferr := h.deps.MediaStore.GetAudiobookByID(r.Context(), sess.ContentID, access); ferr != nil {
slog.Debug("abs listening session detail: hydrate media item failed", "user", a.UserID, "content_id", sess.ContentID, "err", ferr)
} else {
item = fetched
}
}
writeJSON(w, http.StatusOK, sessionToABS(sess, item, h.absBaseURL(r)))
}
func statsToABS(s Stats) map[string]any {
@@ -92,14 +144,103 @@ func statsToABS(s Stats) map[string]any {
}
}
func sessionToABS(s ABSPlaybackSession) map[string]any {
// listeningSessionsEnvelope builds the exact envelope shape
// MeController.getListeningSessions returns upstream. It intentionally does
// NOT reuse pagedEnvelope: that helper's {results,sortBy,filterBy,minified}
// shape is for browse/list endpoints, while real ABS listening-sessions
// responses only ever carry {total,numPages,page,itemsPerPage,sessions}.
func listeningSessionsEnvelope(sessions []map[string]any, total, itemsPerPage, page int) map[string]any {
numPages := 0
if itemsPerPage > 0 {
numPages = (total + itemsPerPage - 1) / itemsPerPage
}
return map[string]any{
"total": total,
"numPages": numPages,
"page": page,
"itemsPerPage": itemsPerPage,
"sessions": sessions,
}
}
// sessionToABS converts a silo ABSPlaybackSession row into the exact key set
// of PlaybackSession.toJSON() upstream (server/objects/PlaybackSession.js).
// item may be nil (deleted item, access revoked, lookup error) — in that
// case a stub MediaItem is fed through the same mediaMetadata builder the
// /play endpoint uses (buildSiloPlayMediaMetadata), so every key is still
// present with empty/zero values rather than being omitted.
func sessionToABS(s ABSPlaybackSession, item *models.MediaItem, baseURL string) map[string]any {
if item == nil {
item = &models.MediaItem{ContentID: s.ContentID}
}
mediaMetadata := buildSiloPlayMediaMetadata(item)
displayAuthor := ""
if v, ok := mediaMetadata["authorName"].(string); ok {
displayAuthor = v
}
startedAt := s.StartedAt
updatedAt := s.LastSyncAt
if updatedAt.IsZero() {
updatedAt = startedAt
}
dateStr := ""
dayOfWeek := ""
var startedAtMs, updatedAtMs int64
if !startedAt.IsZero() {
dateStr = startedAt.UTC().Format("2006-01-02")
dayOfWeek = startedAt.UTC().Weekday().String()
startedAtMs = startedAt.UnixMilli()
}
if !updatedAt.IsZero() {
updatedAtMs = updatedAt.UnixMilli()
}
out := map[string]any{
"id": s.ID,
"libraryItemId": s.ContentID,
"userId": s.UserID,
"libraryId": VirtualLibraryID,
"libraryItemId": s.ContentID,
"bookId": s.ContentID,
"episodeId": nil, // silo is audiobook-only; podcasts are out of scope
"mediaType": LibraryMediaType,
"mediaMetadata": mediaMetadata,
// Chapters are not loaded for session list/detail responses (would
// require an extra media-files fetch per session); real ABS clients
// read chapters from the /play or /sync payload for in-player
// rendering, so an empty list here is a safe placeholder rather than
// the full per-file chapter set.
"chapters": []map[string]any{},
"displayTitle": item.Title,
"displayAuthor": displayAuthor,
"coverPath": baseURL + "/api/items/" + s.ContentID + "/cover",
// duration: the total book duration isn't tracked on the session row
// itself; 0 is a safe placeholder (never crashes, only affects the
// progress-bar denominator on this historical-session view).
"duration": 0,
"playMethod": 0, // DIRECTPLAY
"mediaPlayer": "exo-player",
"deviceInfo": map[string]any{
"deviceId": "unknown",
"manufacturer": "Unknown",
"model": "Unknown",
"sdkVersion": 0,
"clientVersion": "0.0.0",
},
"serverVersion": ServerVersion,
"date": dateStr,
"dayOfWeek": dayOfWeek,
"timeListening": s.TimeListeningSeconds,
"currentTime": s.CurrentPositionSeconds,
// startTime: media position when this session began. Not persisted
// separately from currentTime on ABSPlaybackSession; 0 is safe.
"startTime": 0,
"currentTime": s.CurrentPositionSeconds,
"startedAt": startedAtMs,
"updatedAt": updatedAtMs,
}
// Additive extra field (not part of upstream toJSON) kept for backward
// compatibility with any existing silo-side consumers.
if s.ClosedAt != nil {
out["closedAt"] = s.ClosedAt.UnixMilli()
}
internal/audiobooks/abs/listening_stats_handler_test.go
+53 -3
View File
@@ -70,9 +70,59 @@ func TestStats_Sessions_List_Paginated(t *testing.T) {
if env["total"] != float64(3) {
t.Errorf("total = %v, want 3", env["total"])
}
results, _ := env["results"].([]any)
if len(results) != 3 {
t.Errorf("results len = %d, want 3", len(results))
sessions, _ := env["sessions"].([]any)
if len(sessions) != 3 {
t.Errorf("sessions len = %d, want 3", len(sessions))
}
}
// TestStats_Sessions_List_EnvelopeShape asserts the response matches the
// exact envelope real audiobookshelf's MeController.getListeningSessions
// returns ({total, numPages, page, itemsPerPage, sessions}), and that each
// session object carries the PlaybackSession.toJSON() keys strict clients
// (Flutter/Swift decoders) require: mediaType, mediaMetadata, displayTitle.
func TestStats_Sessions_List_EnvelopeShape(t *testing.T) {
fake := &statsFakeStore{closed: []ABSPlaybackSession{
{ID: "s1", UserID: "1", ContentID: "book-1", TimeListeningSeconds: 120, CurrentPositionSeconds: 45.5},
}}
h := New(Dependencies{MediaStore: noopMediaStore{}, PlaybackSessionStore: fake})
rec := dispatchABSWithParams(http.MethodGet, "/api/me/listening-sessions", nil, nil, "1", "", h.handleListeningSessions)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d; body=%s", rec.Code, rec.Body.String())
}
var env map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &env)
for _, key := range []string{"total", "numPages", "page", "itemsPerPage", "sessions"} {
if _, ok := env[key]; !ok {
t.Errorf("envelope missing key %q; body=%s", key, rec.Body.String())
}
}
if _, ok := env["results"]; ok {
t.Errorf("envelope must not carry the pagedEnvelope 'results' key")
}
sessions, _ := env["sessions"].([]any)
if len(sessions) != 1 {
t.Fatalf("sessions len = %d, want 1", len(sessions))
}
sess, _ := sessions[0].(map[string]any)
if sess["mediaType"] != "book" {
t.Errorf("mediaType = %v, want book", sess["mediaType"])
}
if _, ok := sess["mediaMetadata"].(map[string]any); !ok {
t.Errorf("mediaMetadata missing or wrong type: %v", sess["mediaMetadata"])
}
if _, ok := sess["displayTitle"]; !ok {
t.Errorf("displayTitle missing")
}
for _, key := range []string{"id", "userId", "libraryId", "libraryItemId", "bookId", "episodeId",
"chapters", "displayAuthor", "coverPath", "duration", "playMethod", "mediaPlayer",
"deviceInfo", "serverVersion", "date", "dayOfWeek", "timeListening", "startTime",
"currentTime", "startedAt", "updatedAt"} {
if _, ok := sess[key]; !ok {
t.Errorf("session missing key %q; body=%s", key, rec.Body.String())
}
}
}
internal/audiobooks/abs/login.go
+188 -64
View File
@@ -6,6 +6,7 @@ import (
"io"
"log/slog"
"net/http"
"net/url"
"strings"
"time"
@@ -54,13 +55,27 @@ func (h *Handler) handleStandaloneLogin(w http.ResponseWriter, r *http.Request)
return
}
// Real ABS (express body-parser + passport local) accepts BOTH JSON and
// application/x-www-form-urlencoded credential bodies; different clients
// send different encodings. Buffer the body once, try JSON, then fall back
// to form-encoded — a JSON-only parse 400s a form-encoded client, which
// the app surfaces as a generic "unknown error" on sign-in.
raw, err := io.ReadAll(io.LimitReader(r.Body, 1<<20))
if err != nil {
http.Error(w, "invalid request body", http.StatusBadRequest)
return
}
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil {
http.Error(w, "invalid request body", http.StatusBadRequest)
return
if jsonErr := json.Unmarshal(raw, &body); jsonErr != nil || strings.TrimSpace(body.Username) == "" {
if vals, formErr := url.ParseQuery(string(raw)); formErr == nil {
if u := vals.Get("username"); u != "" {
body.Username = u
body.Password = vals.Get("password")
}
}
}
if strings.TrimSpace(body.Username) == "" || body.Password == "" {
http.Error(w, "username and password are required", http.StatusUnauthorized)
@@ -163,7 +178,15 @@ func (h *Handler) completeLogin(w http.ResponseWriter, r *http.Request, userID,
slog.Debug("abs completeLogin: tokens persisted",
"user_id", userID, "access_jti", accessJTI, "refresh_jti", refreshJTI)
writeJSON(w, http.StatusOK, h.loginEnvelope(r, now, userID, displayName, access, refresh))
// Real ABS delivers the refresh token in the body when the client opts in
// via x-return-tokens, otherwise as an HttpOnly refresh_token cookie.
// Mirrors server/Auth.js: setRefreshTokenCookie when !returnTokens.
returnRefreshInBody := strings.EqualFold(r.Header.Get("x-return-tokens"), "true")
if !returnRefreshInBody {
setRefreshCookie(w, r, refresh, refreshTTL)
}
writeJSON(w, http.StatusOK, h.loginEnvelope(r, now, userID, displayName, access, refresh, returnRefreshInBody))
}
// handleABSPing — GET /ping (mounted also as /healthcheck). Wire shape
@@ -171,7 +194,12 @@ func (h *Handler) completeLogin(w http.ResponseWriter, r *http.Request, userID,
// use this to validate the URL before showing the login form, and any
// other shape causes the official mobile app to reject the server.
func (h *Handler) handleABSPing(w http.ResponseWriter, _ *http.Request) {
// Real ABS /ping returns {"success": true} (server/Server.js). The ABS
// apps validate a server address by reading response.success — without it
// they report "unable to reach". The pong/server/version keys are kept as
// harmless extras for plugin-shape clients.
writeJSON(w, http.StatusOK, map[string]any{
"success": true,
"server": "audiobookshelf",
"version": ServerVersion,
"pong": true,
@@ -184,17 +212,70 @@ func (h *Handler) handleABSInit(w http.ResponseWriter, _ *http.Request) {
}
// handleABSStatus — GET /status. Mobile clients call this on every
// connection to confirm the server is an ABS install and pull a few
// global flags. Matches the plugin shape exactly (key order intentional).
// connection to confirm the server is an ABS install and to learn which
// auth methods to render on the login form. Mirrors real ABS Server.js
// /status: {app, serverVersion, isInit, language, authMethods, authFormData}.
// authMethods drives the login UI — omitting it can leave the app unable to
// present a usable login flow.
func (h *Handler) handleABSStatus(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{
"isInit": true,
"language": "en-us",
"app": "audiobookshelf",
"serverVersion": ServerVersion,
"isInit": true,
"language": "en-us",
"authMethods": []string{"local"},
"authFormData": map[string]any{},
})
}
// absUserObject builds the ABS user object shared by the login/authorize
// envelope and GET /me. It mirrors the key set of audiobookshelf
// User.toOldJSONForBrowser (server/models/User.js) so every endpoint that
// emits a user decodes with one client model — a missing key crashes strict
// clients, so we emit the full set even where silo has no analog (email is
// "", the flags are constant).
//
// `token` is the caller's current access token — real ABS's deprecated
// non-expiring `token` slot. The login/authorize caller additionally sets
// user.accessToken / user.refreshToken; GET /me never carries those.
func absUserObject(userID, displayName, token, defaultLibraryID string, now time.Time) map[string]any {
name := displayName
if name == "" {
name = userID
}
nowMs := now.UnixMilli()
return map[string]any{
"id": userID,
"username": name,
"email": "",
"type": "user",
"defaultLibraryId": defaultLibraryID,
"librariesAccessible": []any{},
"itemTagsAccessible": []any{},
"itemTagsSelected": []any{},
"mediaProgress": []any{},
"bookmarks": []any{},
"seriesHideFromContinueListening": []any{},
"isOldToken": false,
"isActive": true,
"isLocked": false,
"hasOpenIDLink": false,
"token": token,
"lastSeen": nowMs,
"createdAt": nowMs,
"permissions": map[string]any{
"download": true,
"update": true,
"delete": true,
"upload": true,
"accessAllLibraries": true,
"accessAllTags": true,
"accessExplicitContent": true,
"selectedTagsNotAccessible": false,
},
}
}
// loginEnvelope builds the response body shared by /login and /authorize.
// Both endpoints must return the identical shape so the iOS client's
// resume-on-launch flow validates the same way as fresh login.
@@ -209,14 +290,9 @@ func (h *Handler) loginEnvelope(
r *http.Request,
now time.Time,
userID, displayName, accessToken, refreshToken string,
returnRefreshInBody bool,
) map[string]any {
// displayName falls back to userID when the validator didn't supply one.
// ABS clients require a non-empty username on the user envelope.
name := displayName
if name == "" {
name = userID
}
libraryMaps := make([]map[string]any, 0)
defaultLibraryID := VirtualLibraryID
access, _, _ := h.accessFilterFromRequest(r)
@@ -228,40 +304,18 @@ func (h *Handler) loginEnvelope(
libraryMaps = append(libraryMaps, audiobookLibraryMap(lib))
}
nowMs := now.UnixMilli()
user := absUserObject(userID, displayName, accessToken, defaultLibraryID, now)
user := map[string]any{
"id": userID,
"username": name,
"type": "user",
"defaultLibraryId": defaultLibraryID,
"librariesAccessible": []any{},
"itemTagsAccessible": []any{},
"itemTagsSelected": []any{},
"mediaProgress": []any{},
"bookmarks": []any{},
"seriesHideFromContinueListening": []any{},
"isOldToken": false,
"token": accessToken,
"lastSeen": nowMs,
"createdAt": nowMs,
"permissions": map[string]any{
"download": true,
"update": true,
"delete": true,
"upload": true,
"accessAllLibraries": true,
"accessAllTags": true,
"accessExplicitContent": true,
"selectedTagsNotAccessible": false,
},
}
// x-return-tokens opt-in: when set, embed token pair on user object too
// (some clients read from the user envelope, others from the top level).
if strings.EqualFold(r.Header.Get("x-return-tokens"), "true") {
user["accessToken"] = accessToken
// Real ABS (v2.26+) ALWAYS sets user.accessToken; modern clients read it
// from exactly res.user.accessToken. The x-return-tokens header gates ONLY
// the refresh token: present in the body when the client opts in, otherwise
// null (and delivered as the refresh_token cookie by the caller). See
// audiobookshelf server/Auth.js handleLoginSuccess.
user["accessToken"] = accessToken
if returnRefreshInBody {
user["refreshToken"] = refreshToken
} else {
user["refreshToken"] = nil
}
serverSettings := map[string]any{
@@ -300,7 +354,31 @@ func (h *Handler) loginEnvelope(
"podcastEpisodeSchedule": "0 * * * *",
"sortingIgnorePrefixesValue": "",
"allowIframe": false,
"authActiveAuthMethods": []string{"local"},
// Auth / rate-limit / OpenID fields from real ABS
// ServerSettings.toJSONForBrowser. OIDC-aware clients (Prologue)
// decode serverSettings into a strict model that includes these keys;
// omitting them throws keyNotFound and the whole login response fails
// to decode ("unknown error" on the login screen). Emit real ABS's
// defaults for an OIDC-disabled server — authActiveAuthMethods still
// advertises only "local", so no client tries the OpenID flow.
"rateLimitLoginRequests": 10,
"rateLimitLoginWindow": 600000,
"backupPath": "/metadata/backups",
"allowedOrigins": []string{},
"authActiveAuthMethods": []string{"local"},
"authLoginCustomMessage": nil,
"authOpenIDIssuerURL": nil,
"authOpenIDAuthorizationURL": nil,
"authOpenIDTokenURL": nil,
"authOpenIDUserInfoURL": nil,
"authOpenIDJwksURL": nil,
"authOpenIDLogoutURL": nil,
"authOpenIDTokenSigningAlgorithm": "RS256",
"authOpenIDButtonText": "Login with OpenID",
"authOpenIDAutoLaunch": false,
"authOpenIDAutoRegister": false,
"authOpenIDMatchExistingBy": nil,
"authOpenIDSubfolderForRedirectURLs": "",
}
return map[string]any{
@@ -360,7 +438,24 @@ func (h *Handler) handleABSAuthorize(w http.ResponseWriter, r *http.Request) {
}
}
}
writeJSON(w, http.StatusOK, h.loginEnvelope(r, time.Now(), a.UserID, a.UserID, access, ""))
// /authorize never issues a refresh token (the client already holds one),
// so there is nothing to return in the body.
writeJSON(w, http.StatusOK, h.loginEnvelope(r, time.Now(), a.UserID, a.UserID, access, "", false))
}
// setRefreshCookie writes the ABS refresh_token cookie exactly as real ABS
// does when a client did not opt into body-token delivery. HttpOnly + Lax,
// Secure only under TLS so plain-HTTP LAN deployments still receive it.
func setRefreshCookie(w http.ResponseWriter, r *http.Request, refresh string, ttl time.Duration) {
http.SetCookie(w, &http.Cookie{
Name: "refresh_token",
Value: refresh,
Path: "/",
HttpOnly: true,
Secure: r.TLS != nil,
SameSite: http.SameSiteLaxMode,
MaxAge: int(ttl / time.Second),
})
}
// handleRefresh — POST /auth/refresh
@@ -387,6 +482,16 @@ func (h *Handler) handleRefresh(w http.ResponseWriter, r *http.Request) {
refreshTok = p.RefreshToken
}
}
if refreshTok == "" {
// Cookie-flow clients (those that omit x-return-tokens at login) hold the
// refresh token only in the HttpOnly refresh_token cookie the server set —
// they send neither header nor body. Read it here or they get a spurious
// 400 once the access token expires. Mirrors real ABS Auth.js, which
// checks req.cookies.refresh_token.
if c, err := r.Cookie("refresh_token"); err == nil {
refreshTok = strings.TrimSpace(c.Value)
}
}
if refreshTok == "" {
http.Error(w, "refreshToken required", http.StatusBadRequest)
return
@@ -481,31 +586,50 @@ func (h *Handler) handleRefresh(w http.ResponseWriter, r *http.Request) {
slog.Debug("abs refresh: rotated", "user", claims.UserID,
"old_jti", claims.JTI, "new_access_jti", newAccessJTI, "new_refresh_jti", newRefreshJTI)
writeJSON(w, http.StatusOK, map[string]any{
"user": map[string]any{
"id": claims.UserID,
"accessToken": access,
"refreshToken": refresh,
},
"accessToken": access,
"refreshToken": refresh,
})
// Real ABS /auth/refresh returns the SAME payload as /login. Return the
// full envelope (not a thin token map) so a strict client can decode it
// with the same model it uses for login. The rotated refresh token goes
// in the body when the client sent x-refresh-token (mobile), otherwise as
// the refresh_token cookie. No displayName is available at refresh time —
// loginEnvelope falls back to the userID for username.
returnRefreshInBody := strings.TrimSpace(r.Header.Get("x-refresh-token")) != ""
if !returnRefreshInBody {
setRefreshCookie(w, r, refresh, refreshTTL)
}
writeJSON(w, http.StatusOK, h.loginEnvelope(r, now, claims.UserID, "", access, refresh, returnRefreshInBody))
}
// handleLogout — POST /logout (and /api/logout, /abs/api/logout, /abs/api/auth/logout)
//
// Mounted OUTSIDE the bearerAuth group so a client whose access token has
// expired — the most common "I want to sign out" moment — can still revoke
// their JTI without first re-authenticating. The handler parses the bearer
// itself, attempts JTI revoke if parseable, and ALWAYS returns 204. Mirrors
// continuum-plugin-audiobooks/internal/abs/handler.go:handleLogout.
// their JTI without first re-authenticating.
//
// Logout invalidates every active ABS access/refresh token for the presented
// user profile so a signed-out client cannot silently mint a new access token
// with its saved refresh token.
// Real ABS returns HTTP 200 with { redirect_url } (null for local auth), NOT
// an empty 204 — a strict client decodes the body and would fail on 204. It
// also clears the refresh_token cookie. Token/session revocation is
// best-effort: a failure there must not turn logout into an error the client
// can't recover from.
func (h *Handler) handleLogout(w http.ResponseWriter, r *http.Request) {
defer w.WriteHeader(http.StatusNoContent)
h.revokeLogoutPrincipal(r)
// Clear the refresh_token cookie set at login (MaxAge<0 deletes it).
http.SetCookie(w, &http.Cookie{
Name: "refresh_token",
Value: "",
Path: "/",
HttpOnly: true,
MaxAge: -1,
})
// redirect_url is null for local auth (only OpenID logout returns a URL).
writeJSON(w, http.StatusOK, map[string]any{"redirect_url": nil})
}
// revokeLogoutPrincipal best-effort revokes every active token for the bearer's
// principal and closes its open playback sessions. All failures are logged and
// swallowed — the caller always responds 200.
func (h *Handler) revokeLogoutPrincipal(r *http.Request) {
raw := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer ")
if raw == "" {
raw = r.URL.Query().Get("token")
internal/audiobooks/abs/login_body_test.go
+65
View File
@@ -0,0 +1,65 @@
package abs
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// recordingValidator captures the credentials handleStandaloneLogin extracted
// from the request body, and reports success so completeLogin runs.
type recordingValidator struct{ gotUser, gotPass string }
func (v *recordingValidator) Validate(_ context.Context, u, p string) (string, string, string, error) {
v.gotUser, v.gotPass = u, p
return "1", "", "Alice", nil
}
func newLoginBodyHandler(v *recordingValidator) *Handler {
return New(Dependencies{
Config: &staticConfig{secret: []byte("test-secret-32-bytes-aaaaaaaaaaaaa")},
TokenStore: newMemTokenStore(),
MediaStore: noopMediaStore{},
CredValidator: v,
})
}
// TestLogin_AcceptsFormEncoded is the regression guard for the real bug:
// real ABS accepts application/x-www-form-urlencoded credentials; a JSON-only
// parse 400s a form-encoded client, surfacing as "unknown error" on sign-in.
func TestLogin_AcceptsFormEncoded(t *testing.T) {
v := &recordingValidator{}
h := newLoginBodyHandler(v)
req := httptest.NewRequest(http.MethodPost, "/api/login", strings.NewReader("username=alice&password=s3cret%21"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
rec := httptest.NewRecorder()
h.handleLogin(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if v.gotUser != "alice" || v.gotPass != "s3cret!" {
t.Errorf("validator got user=%q pass=%q, want alice/s3cret!", v.gotUser, v.gotPass)
}
}
// TestLogin_AcceptsJSON confirms the JSON path still works after the change.
func TestLogin_AcceptsJSON(t *testing.T) {
v := &recordingValidator{}
h := newLoginBodyHandler(v)
req := httptest.NewRequest(http.MethodPost, "/api/login", strings.NewReader(`{"username":"bob","password":"pw"}`))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.handleLogin(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if v.gotUser != "bob" || v.gotPass != "pw" {
t.Errorf("validator got user=%q pass=%q, want bob/pw", v.gotUser, v.gotPass)
}
}
internal/audiobooks/abs/login_envelope_test.go
+14 -10
View File
@@ -17,7 +17,7 @@ func TestLoginEnvelope_HasRequiredKeys(t *testing.T) {
h, _, _ := newRefreshTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/login", nil)
env := h.loginEnvelope(req, time.Now(),"u1", "Display Name", "access.jwt", "refresh.jwt")
env := h.loginEnvelope(req, time.Now(), "u1", "Display Name", "access.jwt", "refresh.jwt", true)
body, err := json.Marshal(env)
if err != nil {
t.Fatalf("marshal: %v", err)
@@ -105,7 +105,7 @@ func TestLoginEnvelope_XReturnTokens_SurfacesOnUser(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/login", nil)
req.Header.Set("x-return-tokens", "true")
env := h.loginEnvelope(req, time.Now(),"u1", "Display Name", "access.jwt", "refresh.jwt")
env := h.loginEnvelope(req, time.Now(), "u1", "Display Name", "access.jwt", "refresh.jwt", true)
user, ok := env["user"].(map[string]any)
if !ok {
t.Fatalf("user is not a map: %T", env["user"])
@@ -118,16 +118,21 @@ func TestLoginEnvelope_XReturnTokens_SurfacesOnUser(t *testing.T) {
}
}
// TestLoginEnvelope_NoXReturnTokens_OmitsFromUser is the inverse: without the
// header, user object should NOT carry the duplicated tokens (top-level only).
func TestLoginEnvelope_NoXReturnTokens_OmitsFromUser(t *testing.T) {
// TestLoginEnvelope_NoXReturnTokens covers the default flow: real ABS ALWAYS
// puts accessToken on the user object, and sets refreshToken to null when the
// client did not opt into body delivery via x-return-tokens (the caller then
// ships the refresh token as a cookie).
func TestLoginEnvelope_NoXReturnTokens(t *testing.T) {
h, _, _ := newRefreshTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/login", nil)
env := h.loginEnvelope(req, time.Now(),"u1", "Display Name", "access.jwt", "refresh.jwt")
env := h.loginEnvelope(req, time.Now(), "u1", "Display Name", "access.jwt", "refresh.jwt", false)
user, _ := env["user"].(map[string]any)
if _, present := user["accessToken"]; present {
t.Errorf("user.accessToken should not be set without x-return-tokens header")
if user["accessToken"] != "access.jwt" {
t.Errorf("user.accessToken = %v, want access.jwt (must always be present)", user["accessToken"])
}
if rt, present := user["refreshToken"]; !present || rt != nil {
t.Errorf("user.refreshToken = %v (present=%v), want nil without x-return-tokens", rt, present)
}
}
@@ -138,10 +143,9 @@ func TestLoginEnvelope_DisplayNameFallsBackToUserID(t *testing.T) {
h, _, _ := newRefreshTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/login", nil)
env := h.loginEnvelope(req, time.Now(),"user-42", "", "a", "r")
env := h.loginEnvelope(req, time.Now(), "user-42", "", "a", "r", false)
user, _ := env["user"].(map[string]any)
if user["username"] != "user-42" {
t.Errorf("username = %v, want user-42 (fallback)", user["username"])
}
}
internal/audiobooks/abs/login_logout_test.go
+26 -17
View File
@@ -2,6 +2,7 @@ package abs
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
@@ -39,7 +40,7 @@ func mintAndPersistAccess(t *testing.T, store *memTokenStore, cfg *staticConfig,
return access
}
func TestHandleLogout_RevokesJTIAndReturns204(t *testing.T) {
func TestHandleLogout_RevokesJTIAndReturns200(t *testing.T) {
h, store, cfg := newLogoutTestHandler(t)
jti := "logout-test-jti"
access := mintAndPersistAccess(t, store, cfg, "1", jti)
@@ -50,8 +51,16 @@ func TestHandleLogout_RevokesJTIAndReturns204(t *testing.T) {
rec := httptest.NewRecorder()
h.handleLogout(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want 204; body=%s", rec.Code, rec.Body.String())
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
// Real ABS logout body: { redirect_url: null } for local auth.
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode: %v", err)
}
if v, ok := body["redirect_url"]; !ok || v != nil {
t.Errorf("redirect_url = %v (present=%v), want null", v, ok)
}
tok, _ := store.GetTokenByJTI(context.Background(), jti)
if tok.RevokedAt == nil {
@@ -59,35 +68,35 @@ func TestHandleLogout_RevokesJTIAndReturns204(t *testing.T) {
}
}
// TestHandleLogout_NoBearer_204 covers the "client called sign-out with no
// TestHandleLogout_NoBearer_200 covers the "client called sign-out with no
// token" path — must still 204 (logout is idempotent / fire-and-forget).
func TestHandleLogout_NoBearer_204(t *testing.T) {
func TestHandleLogout_NoBearer_200(t *testing.T) {
h, _, _ := newLogoutTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/logout", nil)
rec := httptest.NewRecorder()
h.handleLogout(rec, req)
if rec.Code != http.StatusNoContent {
t.Errorf("status = %d, want 204", rec.Code)
if rec.Code != http.StatusOK {
t.Errorf("status = %d, want 200", rec.Code)
}
}
// TestHandleLogout_GarbageBearer_204 covers an unparseable token: must still
// TestHandleLogout_GarbageBearer_200 covers an unparseable token: must still
// 204 (we never want sign-out to error the client out).
func TestHandleLogout_GarbageBearer_204(t *testing.T) {
func TestHandleLogout_GarbageBearer_200(t *testing.T) {
h, _, _ := newLogoutTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/logout", nil)
req.Header.Set("Authorization", "Bearer not.a.real.jwt")
rec := httptest.NewRecorder()
h.handleLogout(rec, req)
if rec.Code != http.StatusNoContent {
t.Errorf("status = %d, want 204", rec.Code)
if rec.Code != http.StatusOK {
t.Errorf("status = %d, want 200", rec.Code)
}
}
// TestHandleLogout_WrongSignature_204 covers a token signed by a different
// TestHandleLogout_WrongSignature_200 covers a token signed by a different
// secret (attacker token, restored from backup, etc.): must 204 and NOT
// revoke (signature mismatch means we can't trust the JTI claim).
func TestHandleLogout_WrongSignature_204(t *testing.T) {
func TestHandleLogout_WrongSignature_200(t *testing.T) {
h, store, _ := newLogoutTestHandler(t)
jti := "victim-jti"
_ = store.InsertToken(context.Background(), ABSToken{ID: jti, UserID: "victim", JTI: jti})
@@ -100,8 +109,8 @@ func TestHandleLogout_WrongSignature_204(t *testing.T) {
req.Header.Set("Authorization", "Bearer "+bogus)
rec := httptest.NewRecorder()
h.handleLogout(rec, req)
if rec.Code != http.StatusNoContent {
t.Errorf("status = %d, want 204", rec.Code)
if rec.Code != http.StatusOK {
t.Errorf("status = %d, want 200", rec.Code)
}
tok, _ := store.GetTokenByJTI(context.Background(), jti)
if tok.RevokedAt != nil {
@@ -121,8 +130,8 @@ func TestHandleLogout_IsIdempotent(t *testing.T) {
req.Header.Set("Authorization", "Bearer "+access)
rec := httptest.NewRecorder()
h.handleLogout(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("iter %d: status = %d, want 204", i, rec.Code)
if rec.Code != http.StatusOK {
t.Fatalf("iter %d: status = %d, want 200", i, rec.Code)
}
}
}
internal/audiobooks/abs/login_refresh_test.go
+69 -5
View File
@@ -26,7 +26,7 @@ func (noopMediaStore) GetAudiobookByID(context.Context, string, catalog.AccessFi
func (noopMediaStore) GetAudiobooksByIDs(context.Context, []string, catalog.AccessFilter) (map[string]*models.MediaItem, error) {
return map[string]*models.MediaItem{}, nil
}
func (noopMediaStore) ListAudiobooks(context.Context, int64, int, int, catalog.AccessFilter) ([]*models.MediaItem, int, error) {
func (noopMediaStore) ListAudiobooks(context.Context, int64, int, int, catalog.AccessFilter, Filter) ([]*models.MediaItem, int, error) {
return nil, 0, nil
}
func (noopMediaStore) GetMediaFiles(context.Context, string, catalog.AccessFilter) ([]*models.MediaFile, error) {
@@ -50,11 +50,11 @@ func (noopMediaStore) ListRecentlyAdded(context.Context, int64, int, catalog.Acc
func (noopMediaStore) ListDiscover(context.Context, int64, int, catalog.AccessFilter) ([]*models.MediaItem, error) {
return nil, nil
}
func (noopMediaStore) ListLibraryAuthors(context.Context, int64, int, catalog.AccessFilter) ([]AuthorSummary, error) {
return nil, nil
func (noopMediaStore) ListLibraryAuthors(context.Context, int64, int, int, string, bool, catalog.AccessFilter) ([]AuthorSummary, int, error) {
return nil, 0, nil
}
func (noopMediaStore) ListLibrarySeries(context.Context, int64, int, catalog.AccessFilter) ([]SeriesSummary, error) {
return nil, nil
func (noopMediaStore) ListLibrarySeries(context.Context, int64, int, int, catalog.AccessFilter) ([]SeriesSummary, int, error) {
return nil, 0, nil
}
func (noopMediaStore) GetAuthorByID(context.Context, string, catalog.AccessFilter) (Author, error) {
return Author{}, ErrNotFound
@@ -210,6 +210,70 @@ func TestHandleRefresh_BodyToken_Works(t *testing.T) {
}
}
// TestHandleRefresh_ReturnsFullLoginEnvelope guards that /auth/refresh returns
// the SAME payload shape as /login (real ABS behavior) — not a thin token map —
// so strict clients can decode it with their login model.
func TestHandleRefresh_ReturnsFullLoginEnvelope(t *testing.T) {
h, store, cfg := newRefreshTestHandler(t)
refresh, _ := mintAndPersistRefresh(t, store, cfg, "5")
req := httptest.NewRequest(http.MethodPost, "/auth/refresh", nil)
req.Header.Set("x-refresh-token", refresh)
rec := httptest.NewRecorder()
h.handleRefresh(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
for _, k := range []string{"user", "userDefaultLibraryId", "serverSettings", "Source", "ereaderDevices"} {
if _, ok := resp[k]; !ok {
t.Errorf("refresh envelope missing top-level %q", k)
}
}
// x-refresh-token present → token in body, not a cookie.
if user, _ := resp["user"].(map[string]any); user["refreshToken"] == nil {
t.Errorf("user.refreshToken should be in body when x-refresh-token sent")
}
}
// TestHandleRefresh_NoHeader_SetsCookie: without x-refresh-token the rotated
// refresh token is delivered as the refresh_token cookie and nulled in the body.
func TestHandleRefresh_NoHeader_SetsCookie(t *testing.T) {
h, store, cfg := newRefreshTestHandler(t)
refresh, _ := mintAndPersistRefresh(t, store, cfg, "6")
body := bytes.NewBufferString(`{"refreshToken":"` + refresh + `"}`)
req := httptest.NewRequest(http.MethodPost, "/auth/refresh", body)
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.handleRefresh(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var got *http.Cookie
for _, c := range rec.Result().Cookies() {
if c.Name == "refresh_token" {
got = c
}
}
if got == nil || got.Value == "" {
t.Fatalf("refresh_token cookie not set")
}
if !got.HttpOnly {
t.Errorf("refresh_token cookie should be HttpOnly")
}
var resp map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
if user, _ := resp["user"].(map[string]any); user["refreshToken"] != nil {
t.Errorf("user.refreshToken should be null when delivered via cookie")
}
}
func TestHandleRefresh_NoToken_400(t *testing.T) {
h, _, _ := newRefreshTestHandler(t)
req := httptest.NewRequest(http.MethodPost, "/auth/refresh", nil)
internal/audiobooks/abs/me_handler.go
+55 -14
View File
@@ -4,16 +4,17 @@ import (
"net/http"
"strconv"
"strings"
"time"
)
// handleMe — GET /abs/api/me (and /api/me)
//
// Minimal {id, username, defaultLibraryId} envelope — matches the
// continuum-plugin-audiobooks shape exactly. ABS clients already
// have the full user object from /login and /authorize; /me is just
// a session-resume probe in real-ABS, so returning the rich user
// envelope here is unnecessary and can confuse clients that pattern-
// match on the minimal shape.
// Real ABS returns req.user.toOldJSONForBrowser() — the FULL user object, the
// same shape carried on the login/authorize envelope's `user`. A strict client
// decodes /me with its User model, so a thin {id,username,...} map crashes it
// on the first missing required key. Emit the shared absUserObject (minus the
// login-only accessToken/refreshToken; /me still carries the `token` slot set
// to the caller's presented bearer).
func (h *Handler) handleMe(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
@@ -32,11 +33,16 @@ func (h *Handler) handleMe(w http.ResponseWriter, r *http.Request) {
defaultLibID = audiobookLibraryID(libs[0])
}
writeJSON(w, http.StatusOK, map[string]any{
"id": a.UserID,
"username": a.UserID,
"defaultLibraryId": defaultLibID,
})
// Resolve the real display username; the token only carries the userID, so
// without this /me would report the numeric id as the username.
name := a.UserID
if h.deps.UsernameResolver != nil {
if resolved := h.deps.UsernameResolver(r.Context(), a.UserID, a.ProfileID); resolved != "" {
name = resolved
}
}
writeJSON(w, http.StatusOK, absUserObject(a.UserID, name, a.Token, defaultLibID, time.Now()))
}
// audiobookLibraryID returns the ABS-wire library ID string for an
@@ -59,9 +65,44 @@ func audiobookLibraryMap(lib AudiobookLibrary) map[string]any {
if name == "" {
name = VirtualLibraryName
}
id := audiobookLibraryID(lib)
return map[string]any{
"id": audiobookLibraryID(lib),
"name": name,
"mediaType": LibraryMediaType,
"id": id,
"name": name,
// folders mirror real ABS LibraryFolder.toOldJSON {id,fullPath,libraryId,addedAt}.
// silo serves a single virtual folder per library.
"folders": []map[string]any{
{"id": VirtualFolderID, "fullPath": "/" + name, "libraryId": id, "addedAt": 0},
},
"displayOrder": 1,
"icon": "audiobookshelf",
"mediaType": LibraryMediaType,
"provider": "audible",
"settings": audiobookLibrarySettings(),
"lastScan": nil,
"lastScanVersion": ServerVersion,
"createdAt": 0,
"lastUpdate": 0,
}
}
// audiobookLibrarySettings emits the real ABS library `settings` object.
// It's a loose object clients read defensively; silo has no per-library
// settings storage, so these are sensible audiobook defaults. coverAspectRatio
// 1 = square (audiobook covers).
func audiobookLibrarySettings() map[string]any {
return map[string]any{
"coverAspectRatio": 1,
"disableWatcher": true,
"skipMatchingMediaWithAsin": false,
"skipMatchingMediaWithIsbn": false,
"autoScanCronExpression": nil,
"audiobooksOnly": false,
"hideSingleBookSeries": false,
"onlyShowLaterBooksInContinueSeries": false,
"metadataPrecedence": []string{},
"epubsAllowScriptedContent": false,
"markAsFinishedPercentComplete": nil,
"markAsFinishedTimeRemaining": 10,
}
}
internal/audiobooks/abs/me_handler_test.go
+56
View File
@@ -0,0 +1,56 @@
package abs
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// TestHandleMe_ReturnsFullUserObject guards that GET /me emits the full ABS
// user object (audiobookshelf User.toOldJSONForBrowser), not a thin
// {id,username,defaultLibraryId} map — a strict client decodes /me with its
// User model and crashes on any missing required key.
func TestHandleMe_ReturnsFullUserObject(t *testing.T) {
h := New(Dependencies{MediaStore: noopMediaStore{}})
req := httptest.NewRequest(http.MethodGet, "/api/me", nil)
req = req.WithContext(context.WithValue(req.Context(), ctxKey{}, ctxAuth{
UserID: "42",
Token: "bearer.jwt",
}))
rec := httptest.NewRecorder()
h.handleMe(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var user map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &user); err != nil {
t.Fatalf("decode: %v", err)
}
// Full toOldJSONForBrowser key set.
want := []string{
"id", "username", "email", "type", "token", "isOldToken",
"mediaProgress", "seriesHideFromContinueListening", "bookmarks",
"isActive", "isLocked", "lastSeen", "createdAt", "permissions",
"librariesAccessible", "itemTagsSelected", "hasOpenIDLink",
}
for _, k := range want {
if _, ok := user[k]; !ok {
t.Errorf("/me user missing key %q", k)
}
}
if user["token"] != "bearer.jwt" {
t.Errorf("token = %v, want bearer.jwt (presented bearer)", user["token"])
}
// /me must NOT carry login-only token pair.
if _, ok := user["accessToken"]; ok {
t.Errorf("/me should not carry accessToken")
}
if _, ok := user["refreshToken"]; ok {
t.Errorf("/me should not carry refreshToken")
}
}
internal/audiobooks/abs/minified.go
+114 -38
View File
@@ -12,31 +12,69 @@ import "strings"
// for a minified request works (clients ignore extra fields) but is wasteful
// on the wire when a client is paging through hundreds of items. Emitting
// the minified shape for a full request silently breaks detail pages.
// Keys mirror real ABS LibraryItem.toOldJSONMinified. numFiles/size are
// minified-only; libraryFiles/lastScan/scanVersion (full-only) are absent.
type MinifiedLibraryItem struct {
ID string `json:"id"`
LibraryID string `json:"libraryId"`
FolderID string `json:"folderId"`
MediaType string `json:"mediaType"`
Media minifiedMedia `json:"media"`
NumTracks int `json:"numTracks,omitempty"`
AddedAt int64 `json:"addedAt"`
UpdatedAt int64 `json:"updatedAt"`
ID string `json:"id"`
Ino string `json:"ino"`
OldLibraryItemID *string `json:"oldLibraryItemId"`
LibraryID string `json:"libraryId"`
FolderID string `json:"folderId"`
Path string `json:"path"`
RelPath string `json:"relPath"`
IsFile bool `json:"isFile"`
MtimeMs int64 `json:"mtimeMs"`
CtimeMs int64 `json:"ctimeMs"`
BirthtimeMs int64 `json:"birthtimeMs"`
AddedAt int64 `json:"addedAt"`
UpdatedAt int64 `json:"updatedAt"`
IsMissing bool `json:"isMissing"`
IsInvalid bool `json:"isInvalid"`
MediaType string `json:"mediaType"`
Media minifiedMedia `json:"media"`
NumFiles int `json:"numFiles"`
Size int64 `json:"size"`
CollapsedSeries *CollapsedSeriesV1 `json:"collapsedSeries,omitempty"`
}
// Keys mirror real ABS Book.toOldJSONMinified: numeric summaries instead of
// audioFiles/chapters/tracks. numTracks/numAudioFiles drive whether strict
// clients (Plappa) show the item at all — an item reporting 0 audio files is
// dropped, so they are always >= 1 in the browse projection.
type minifiedMedia struct {
Metadata minifiedMetadata `json:"metadata"`
Duration float64 `json:"duration"`
CoverPath string `json:"coverPath"`
ID string `json:"id"`
Metadata minifiedMetadata `json:"metadata"`
CoverPath string `json:"coverPath"`
Tags []string `json:"tags"`
NumTracks int `json:"numTracks"`
NumAudioFiles int `json:"numAudioFiles"`
NumChapters int `json:"numChapters"`
Duration float64 `json:"duration"`
Size int64 `json:"size"`
EbookFormat *string `json:"ebookFormat"`
}
// Keys mirror real ABS Book.oldMetadataToJSONMinified — flat author/series
// strings, no authors[]/series[] arrays. Nullable-in-ABS string fields are
// emitted as "" (safe: the clients decode them as String?), never dropped.
type minifiedMetadata struct {
Title string `json:"title"`
AuthorName string `json:"authorName"`
AuthorNameLF string `json:"authorNameLF"`
SeriesName string `json:"seriesName,omitempty"`
SeriesSequence string `json:"seriesSequence,omitempty"`
Narrators []string `json:"narrators,omitempty"`
PublishedYear string `json:"publishedYear,omitempty"`
Title string `json:"title"`
TitleIgnorePrefix string `json:"titleIgnorePrefix"`
Subtitle string `json:"subtitle"`
AuthorName string `json:"authorName"`
AuthorNameLF string `json:"authorNameLF"`
NarratorName string `json:"narratorName"`
SeriesName string `json:"seriesName"`
Genres []string `json:"genres"`
PublishedYear string `json:"publishedYear"`
PublishedDate string `json:"publishedDate"`
Publisher string `json:"publisher"`
Description string `json:"description"`
ISBN string `json:"isbn"`
ASIN string `json:"asin"`
Language string `json:"language"`
Explicit bool `json:"explicit"`
Abridged bool `json:"abridged"`
}
// Minify projects a LibraryItem onto the minified shape. The original item
@@ -55,33 +93,71 @@ func Minify(item LibraryItem) MinifiedLibraryItem {
names = append(names, a.Name)
lfNames = append(lfNames, lastFirst(a.Name))
}
seriesName, seriesSeq := "", ""
seriesName := ""
if len(m.Series) > 0 {
s := m.Series[0]
seriesName = s.Name
seriesSeq = s.Sequence
seriesName = m.Series[0].Name
if seq := m.Series[0].Sequence; seq != "" {
seriesName += " #" + seq
}
}
genres := m.Genres
if genres == nil {
genres = []string{}
}
tags := item.Media.Tags
if tags == nil {
tags = []string{}
}
// Every audiobook has at least one audio file; report >= 1 so strict
// clients (Plappa) don't drop the item. Exact counts come from item-detail.
numTracks := item.Media.NumTracks
if numTracks < 1 {
numTracks = 1
}
return MinifiedLibraryItem{
ID: item.ID,
LibraryID: item.LibraryID,
FolderID: item.FolderID,
MediaType: item.MediaType,
ID: item.ID,
Ino: item.Ino,
LibraryID: item.LibraryID,
FolderID: item.FolderID,
Path: item.Path,
RelPath: item.RelPath,
IsFile: item.IsFile,
MtimeMs: item.MtimeMs,
CtimeMs: item.CtimeMs,
BirthtimeMs: item.BirthtimeMs,
AddedAt: item.AddedAt,
UpdatedAt: item.UpdatedAt,
IsMissing: item.IsMissing,
IsInvalid: item.IsInvalid,
MediaType: item.MediaType,
Media: minifiedMedia{
ID: item.Media.ID,
Metadata: minifiedMetadata{
Title: m.Title,
AuthorName: strings.Join(names, ", "),
AuthorNameLF: strings.Join(lfNames, " & "),
SeriesName: seriesName,
SeriesSequence: seriesSeq,
Narrators: m.Narrators,
PublishedYear: m.PublishedYear,
Title: m.Title,
TitleIgnorePrefix: titleIgnorePrefix(m.Title),
AuthorName: strings.Join(names, ", "),
AuthorNameLF: strings.Join(lfNames, ", "),
NarratorName: strings.Join(m.Narrators, ", "),
SeriesName: seriesName,
Genres: genres,
PublishedYear: m.PublishedYear,
Publisher: m.Publisher,
Description: m.Description,
ISBN: m.ISBN,
Language: "en",
Explicit: m.Explicit,
},
Duration: item.Media.Duration,
CoverPath: item.Media.CoverPath,
CoverPath: item.Media.CoverPath,
Tags: tags,
NumTracks: numTracks,
NumAudioFiles: numTracks,
NumChapters: len(item.Media.Chapters),
Duration: item.Media.Duration,
Size: 0,
},
NumTracks: item.NumTracks,
AddedAt: item.AddedAt,
UpdatedAt: item.UpdatedAt,
NumFiles: numTracks,
Size: 0,
CollapsedSeries: item.CollapsedSeries,
}
}
internal/audiobooks/abs/minified_test.go
+92
View File
@@ -0,0 +1,92 @@
package abs
import (
"encoding/json"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// TestMinify_ConformsToRealABSKeys asserts the minified list item matches real
// ABS LibraryItem.toOldJSONMinified + Book.toOldJSONMinified key sets — a
// missing key crashes strict clients, and media.numTracks/numAudioFiles must
// be >= 1 or Plappa drops the item.
func TestMinify_ConformsToRealABSKeys(t *testing.T) {
item := &models.MediaItem{
ContentID: "book-1",
Title: "Test Book",
People: []models.ItemPerson{
{Person: models.Person{ID: 42, Name: "Stephen King"}, Kind: models.PersonKindAuthor},
},
}
lib := AudiobookLibrary{ID: 18, Name: "Audiobooks", Type: "audiobooks"}
full := siloItemToLibraryItem(item, lib, "http://x")
min := Minify(full)
body, err := json.Marshal(min)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var m map[string]any
if err := json.Unmarshal(body, &m); err != nil {
t.Fatalf("decode: %v", err)
}
outer := []string{
"id", "ino", "oldLibraryItemId", "libraryId", "folderId", "path",
"relPath", "isFile", "mtimeMs", "ctimeMs", "birthtimeMs", "addedAt",
"updatedAt", "isMissing", "isInvalid", "mediaType", "media", "numFiles", "size",
}
for _, k := range outer {
if _, ok := m[k]; !ok {
t.Errorf("minified item missing outer key %q", k)
}
}
media, _ := m["media"].(map[string]any)
mediaKeys := []string{
"id", "metadata", "coverPath", "tags", "numTracks", "numAudioFiles",
"numChapters", "duration", "size", "ebookFormat",
}
for _, k := range mediaKeys {
if _, ok := media[k]; !ok {
t.Errorf("minified media missing key %q", k)
}
}
if media["id"] != "book-1" {
t.Errorf("media.id = %v, want book-1", media["id"])
}
// Plappa guard: never advertise 0 audio files.
if n, _ := media["numTracks"].(float64); n < 1 {
t.Errorf("media.numTracks = %v, want >= 1", media["numTracks"])
}
if n, _ := media["numAudioFiles"].(float64); n < 1 {
t.Errorf("media.numAudioFiles = %v, want >= 1", media["numAudioFiles"])
}
meta, _ := media["metadata"].(map[string]any)
metaKeys := []string{
"title", "titleIgnorePrefix", "subtitle", "authorName", "authorNameLF",
"narratorName", "seriesName", "genres", "publishedYear", "publishedDate",
"publisher", "description", "isbn", "asin", "language", "explicit", "abridged",
}
for _, k := range metaKeys {
if _, ok := meta[k]; !ok {
t.Errorf("minified metadata missing key %q", k)
}
}
}
// TestSiloItemToLibraryItem_MediaHasID guards the yaabsa BookMedia.id crash:
// the non-minified media object must carry id + libraryItemId = ContentID.
func TestSiloItemToLibraryItem_MediaHasID(t *testing.T) {
item := &models.MediaItem{ContentID: "book-9", Title: "T"}
full := siloItemToLibraryItem(item, AudiobookLibrary{ID: 1}, "http://x")
if full.Media.ID != "book-9" {
t.Errorf("media.id = %q, want book-9", full.Media.ID)
}
if full.Media.LibraryItemID != "book-9" {
t.Errorf("media.libraryItemId = %q, want book-9", full.Media.LibraryItemID)
}
}
internal/audiobooks/abs/ping_status_test.go
+47
View File
@@ -0,0 +1,47 @@
package abs
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// TestABSPing_HasSuccess guards the reachability probe: real ABS /ping returns
// {"success": true} and the ABS apps validate a server address by reading that
// field. Without it the app reports "unable to reach".
func TestABSPing_HasSuccess(t *testing.T) {
h := &Handler{}
rec := httptest.NewRecorder()
h.handleABSPing(rec, httptest.NewRequest(http.MethodGet, "/ping", nil))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
var m map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &m); err != nil {
t.Fatalf("decode: %v", err)
}
if m["success"] != true {
t.Errorf("/ping success = %v, want true", m["success"])
}
}
// TestABSStatus_HasAuthMethods guards that /status carries authMethods (drives
// the login form) — real ABS Server.js /status shape.
func TestABSStatus_HasAuthMethods(t *testing.T) {
h := &Handler{}
rec := httptest.NewRecorder()
h.handleABSStatus(rec, httptest.NewRequest(http.MethodGet, "/status", nil))
var m map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &m); err != nil {
t.Fatalf("decode: %v", err)
}
for _, k := range []string{"app", "serverVersion", "isInit", "language", "authMethods", "authFormData"} {
if _, ok := m[k]; !ok {
t.Errorf("/status missing key %q", k)
}
}
if methods, ok := m["authMethods"].([]any); !ok || len(methods) == 0 {
t.Errorf("authMethods = %v, want non-empty array", m["authMethods"])
}
}
internal/audiobooks/abs/play_response.go
+1 -1
View File
@@ -254,7 +254,7 @@ func buildSiloAudioTracks(
TimeBase: "1/14112000",
Channels: channels,
ChannelLayout: channelLayout,
Chapters: nil,
Chapters: []ChapterABS{},
EmbeddedCoverArt: nil,
MetaTags: map[string]string{},
MimeType: mimeType,
internal/audiobooks/abs/playlists.go
+1
View File
@@ -55,6 +55,7 @@ type PlaylistItem struct {
func playlistToABS(p Playlist, items []map[string]any) map[string]any {
out := map[string]any{
"id": p.ID,
"libraryId": VirtualLibraryID, // real ABS Playlist.toOldJSON has libraryId; silo playlists are cross-library user-personal
"userId": p.UserID,
"name": p.Name,
"description": p.Description,
internal/audiobooks/abs/playlists_envelope_test.go
+1 -1
View File
@@ -25,7 +25,7 @@ func TestPlaylistEnvelope_HasRequiredKeys(t *testing.T) {
body, _ := json.Marshal(out)
js := string(body)
for _, key := range []string{
`"id":`, `"userId":`, `"name":`, `"description":`,
`"id":`, `"libraryId":`, `"userId":`, `"name":`, `"description":`,
`"isPublic":`, `"coverPath":`, `"createdAt":`, `"lastUpdate":`, `"items":`,
} {
if !strings.Contains(js, key) {
internal/audiobooks/abs/search_shape_test.go
+166
View File
@@ -0,0 +1,166 @@
package abs
import (
"context"
"encoding/json"
"net/http"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
// searchStubMediaStore backs the /libraries/{id}/search shape tests. It
// embeds noopMediaStore so only the methods the handler exercises need
// overriding.
type searchStubMediaStore struct {
noopMediaStore
libs []AudiobookLibrary
results []*models.MediaItem
authors []AuthorSummary
series []SeriesSummary
}
func (s *searchStubMediaStore) ListAudiobookLibraries(context.Context, catalog.AccessFilter) ([]AudiobookLibrary, error) {
return s.libs, nil
}
func (s *searchStubMediaStore) SearchAudiobooks(_ context.Context, _ int64, _ string, _ int, _ catalog.AccessFilter) ([]*models.MediaItem, error) {
return s.results, nil
}
func (s *searchStubMediaStore) ListLibraryAuthors(context.Context, int64, int, int, string, bool, catalog.AccessFilter) ([]AuthorSummary, int, error) {
return s.authors, len(s.authors), nil
}
func (s *searchStubMediaStore) ListLibrarySeries(context.Context, int64, int, int, catalog.AccessFilter) ([]SeriesSummary, int, error) {
return s.series, len(s.series), nil
}
func newSearchHarness() *Handler {
store := &searchStubMediaStore{
libs: []AudiobookLibrary{{ID: 1, Name: "Audiobooks", Type: "audiobooks"}},
results: []*models.MediaItem{
{ContentID: "book-1", Title: "The Search Result"},
},
authors: []AuthorSummary{{ID: "a1", Name: "Search Author", NumBooks: 3}},
series: []SeriesSummary{{
ID: "s1", Name: "Search Series", NumBooks: 2,
Books: []SeriesBookPreview{{ContentID: "book-1", Title: "The Search Result"}},
}},
}
return New(Dependencies{MediaStore: store})
}
// TestLibrarySearch_BucketKeysPresent asserts the response has every bucket
// key real ABS's libraryItemsBookFilters.search() returns (book, narrators,
// tags, genres, series, authors), plus our extra podcast bucket. A missing
// key crashes strict ABS clients that decode the whole envelope up front.
func TestLibrarySearch_BucketKeysPresent(t *testing.T) {
h := newSearchHarness()
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/1/search?q=search&limit=10",
map[string]string{"libraryId": "1"}, nil, "1", "", h.handleLibrarySearch)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
for _, key := range []string{"book", "podcast", "narrators", "tags", "genres", "series", "authors"} {
if _, ok := got[key]; !ok {
t.Errorf("missing bucket key %q in response: %v", key, got)
}
}
}
// TestLibrarySearch_BookEntryHasLibraryItem asserts each "book" bucket entry
// is `{ libraryItem: <expanded LibraryItem> }`, matching real ABS
// (itemMatches.push({ libraryItem: libraryItem.toOldJSONExpanded() })).
func TestLibrarySearch_BookEntryHasLibraryItem(t *testing.T) {
h := newSearchHarness()
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/1/search?q=search",
map[string]string{"libraryId": "1"}, nil, "1", "", h.handleLibrarySearch)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
books, ok := got["book"].([]any)
if !ok || len(books) != 1 {
t.Fatalf("book bucket = %v, want 1 entry", got["book"])
}
entry, ok := books[0].(map[string]any)
if !ok {
t.Fatalf("book entry not an object: %v", books[0])
}
li, ok := entry["libraryItem"].(map[string]any)
if !ok {
t.Fatalf("book entry missing libraryItem sub-object: %v", entry)
}
if li["id"] != "book-1" {
t.Errorf("libraryItem.id = %v, want book-1", li["id"])
}
if _, hasMatchKey := entry["matchKey"]; hasMatchKey {
t.Errorf("book entry has matchKey, real ABS does not emit it here: %v", entry)
}
}
// TestLibrarySearch_EmptyQuery_ReturnsEmptyBuckets covers the q="" short
// circuit — real ABS 400s on a missing q, but Silo has historically
// returned the empty-bucket envelope for an empty query; keep that
// behavior and just assert the keys survive.
func TestLibrarySearch_EmptyQuery_ReturnsEmptyBuckets(t *testing.T) {
h := newSearchHarness()
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/1/search?q=",
map[string]string{"libraryId": "1"}, nil, "1", "", h.handleLibrarySearch)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body=%s", rec.Code, rec.Body.String())
}
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
for _, key := range []string{"book", "podcast", "narrators", "tags", "genres", "series", "authors"} {
if _, ok := got[key]; !ok {
t.Errorf("missing bucket key %q on empty query: %v", key, got)
}
}
}
// TestLibrarySearch_AuthorsAndSeriesMatched asserts a query matching the
// stubbed author/series name populates those buckets with real-ABS-shaped
// entries (series wrapped as { series, books }).
func TestLibrarySearch_AuthorsAndSeriesMatched(t *testing.T) {
h := newSearchHarness()
rec := dispatchABSWithParams(http.MethodGet, "/api/libraries/1/search?q=search",
map[string]string{"libraryId": "1"}, nil, "1", "", h.handleLibrarySearch)
var got map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
authors, _ := got["authors"].([]any)
if len(authors) != 1 {
t.Fatalf("authors bucket = %v, want 1 entry", got["authors"])
}
series, _ := got["series"].([]any)
if len(series) != 1 {
t.Fatalf("series bucket = %v, want 1 entry", got["series"])
}
seriesEntry, ok := series[0].(map[string]any)
if !ok {
t.Fatalf("series entry not an object: %v", series[0])
}
if _, ok := seriesEntry["series"]; !ok {
t.Errorf("series entry missing nested 'series' key: %v", seriesEntry)
}
if _, ok := seriesEntry["books"]; !ok {
t.Errorf("series entry missing 'books' key: %v", seriesEntry)
}
}
internal/audiobooks/abs/session_local.go
+213
View File
@@ -0,0 +1,213 @@
package abs
import (
"context"
"encoding/json"
"io"
"log/slog"
"net/http"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
)
// ---------------------------------------------------------------------------
// Offline session sync (ABS SessionController.syncLocal / syncLocalSessions)
// ---------------------------------------------------------------------------
//
// The official ABS mobile app records playback while offline into local
// PlaybackSession objects, then POSTs them back on reconnect so the server's
// media progress catches up. Two endpoints implement this:
//
// POST /session/local — one session (SessionController.syncLocal)
// POST /session/local-all — many sessions (SessionController.syncLocalSessions)
//
// Real ABS (server/managers/PlaybackSessionManager.js):
// - syncLocalSessionRequest: syncLocalSession(one) → 200 on success,
// 500 + error text on failure.
// - syncLocalSessionsRequest: reads req.body.sessions, loops each through
// syncLocalSession, replies { results: [ {id, success, error?,
// progressSynced} ] } with HTTP 200 regardless of per-session outcome.
// - syncLocalSession returns {id, success:false, error} when the library
// item can't be found, else {id, success:true, progressSynced}.
//
// silo does not persist arbitrary client-supplied sessions, so we do not
// create a server-side playback-session row here. We mirror the *effect* that
// matters — the caller's resume position — into user_watch_progress and emit
// the same user_item_progress_updated realtime event handleSessionSync does.
// An existing progress row is advanced monotonically (UpdateProgressPosition);
// a book listened to entirely offline has no row yet, so one is created
// (UpsertProgress) rather than silently dropping the position. Accumulated
// offline listening time is not persisted: there is no store method to add
// standalone listening time without an existing session row. Position sync is
// the client-visible behaviour offline sync exists to restore.
// localPlaybackSession is the subset of the ABS PlaybackSession payload the
// client POSTs for offline sync that silo acts on. EpisodeID is a pointer so a
// present-but-null value (audiobook) is distinguishable from a podcast episode.
type localPlaybackSession struct {
ID string `json:"id"`
LibraryItemID string `json:"libraryItemId"`
EpisodeID *string `json:"episodeId"`
CurrentTime float64 `json:"currentTime"`
TimeListening float64 `json:"timeListening"`
DisplayTitle string `json:"displayTitle"`
}
// localSyncResult mirrors the per-session object ABS returns from
// syncLocalSession: {id, success, error?, progressSynced}.
type localSyncResult struct {
ID string `json:"id"`
Success bool `json:"success"`
Error string `json:"error,omitempty"`
ProgressSynced bool `json:"progressSynced"`
}
// handleSyncLocalSession — POST /session/local
// Syncs a single offline-recorded session. Matches ABS syncLocalSessionRequest:
// 200 on success, 500 + error text when the item can't be resolved.
func (h *Handler) handleSyncLocalSession(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
var sess localPlaybackSession
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&sess); err != nil {
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
res := h.syncOneLocalSession(r.Context(), a, access, sess)
if !res.Success {
// Real ABS: res.status(500).send(result.error).
http.Error(w, res.Error, http.StatusInternalServerError)
return
}
// Real ABS: res.sendStatus(200) — plain 200, no JSON body.
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("OK"))
}
// handleSyncLocalSessions — POST /session/local-all
// Batch-syncs offline-recorded sessions. Matches ABS syncLocalSessionsRequest:
// reads {sessions: [...]}, loops each, always replies 200 with
// {results: [...]}. A bad session never fails the whole batch.
func (h *Handler) handleSyncLocalSessions(w http.ResponseWriter, r *http.Request) {
a, ok := absAuthFrom(r)
if !ok || a.UserID == "" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
// Decode into raw messages so one malformed session doesn't sink the batch.
var body struct {
Sessions []json.RawMessage `json:"sessions"`
}
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil {
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
access, err := h.accessFilterForAuth(r.Context(), a)
if err != nil {
http.Error(w, "resolve access: "+err.Error(), http.StatusForbidden)
return
}
results := make([]localSyncResult, 0, len(body.Sessions))
for _, raw := range body.Sessions {
var sess localPlaybackSession
if err := json.Unmarshal(raw, &sess); err != nil {
slog.Warn("abs local session sync: skipping malformed session", "error", err)
results = append(results, localSyncResult{Success: false, Error: "invalid session"})
continue
}
results = append(results, h.syncOneLocalSession(r.Context(), a, access, sess))
}
writeJSON(w, http.StatusOK, map[string]any{"results": results})
}
// syncOneLocalSession applies one offline session's position to the caller's
// media progress and returns the ABS-shaped per-session result. It never
// panics and never propagates an error to fail a batch.
func (h *Handler) syncOneLocalSession(ctx context.Context, a ctxAuth, access catalog.AccessFilter, sess localPlaybackSession) localSyncResult {
res := localSyncResult{ID: sess.ID}
// Podcast episode sessions are out of scope for silo's audiobook-only
// catalog. Accept them as a no-op success so the client clears its queue.
if sess.EpisodeID != nil && *sess.EpisodeID != "" {
res.Success = true
return res
}
if sess.LibraryItemID == "" {
res.Error = "Media item not found"
return res
}
// Ownership / existence / access gate — the item must be visible to the
// caller. Mirrors handleSessionSync's access check.
if h.deps.MediaStore == nil {
res.Error = "Media item not found"
return res
}
item, err := h.deps.MediaStore.GetAudiobookByID(ctx, sess.LibraryItemID, access)
if err != nil || item == nil {
if err != nil {
slog.Warn("abs local session sync: media lookup failed",
"library_item_id", sess.LibraryItemID, "error", err)
}
res.Error = "Media item not found"
return res
}
res.Success = true
// Persist the offline resume position into user_watch_progress.
//
// For an existing row we use UpdateProgressPosition (not a full upsert) so a
// stale offline tick can't overwrite is_finished / progress_pct the user set
// explicitly — it advances position monotonically and skips completed rows.
//
// But UpdateProgressPosition is UPDATE-only: when a book was listened to
// entirely offline no row exists yet, so it would affect zero rows, drop the
// position, and still report ProgressSynced=true — the client then clears its
// local session with nothing saved. Create the row in that case so the resume
// point survives.
if h.deps.ProgressStore != nil {
var syncErr error
existing, getErr := h.deps.ProgressStore.GetProgress(ctx, a.UserID, a.ProfileID, sess.LibraryItemID)
if getErr == nil && existing == nil {
syncErr = h.deps.ProgressStore.UpsertProgress(ctx, ProgressRow{
UserID: a.UserID,
ProfileID: a.ProfileID,
ContentID: sess.LibraryItemID,
CurrentSeconds: sess.CurrentTime,
// For audiobooks MediaItem.Runtime holds total seconds (set by the
// scanner), matching what the ABS libraries handler reads.
DurationSeconds: float64(item.Runtime),
UpdatedAt: time.Now(),
})
} else {
syncErr = h.deps.ProgressStore.UpdateProgressPosition(
ctx, a.UserID, a.ProfileID, sess.LibraryItemID, sess.CurrentTime,
)
}
if syncErr != nil {
slog.Warn("abs local session sync: persist progress position failed",
"library_item_id", sess.LibraryItemID, "error", syncErr)
} else {
res.ProgressSynced = true
// Realtime push so other connected clients see the caught-up
// position — same event handleSessionSync emits.
h.publish(a.UserID, "user_item_progress_updated", map[string]any{
"data": map[string]any{
"libraryItemId": sess.LibraryItemID,
"currentTime": sess.CurrentTime,
},
})
}
}
return res
}
internal/audiobooks/abs/session_local_test.go
+213
View File
@@ -0,0 +1,213 @@
package abs
import (
"context"
"encoding/json"
"net/http"
"sync"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
// positionRecordingProgressFake models just enough of the progress store for the
// offline-sync tests: GetProgress returns any pre-seeded row, and both the
// create (UpsertProgress) and update (UpdateProgressPosition) paths record the
// persisted position plus which path handled each item.
type positionRecordingProgressFake struct {
fakeProgressStore
mu sync.Mutex
seeded map[string]*ProgressRow // contentID → existing row GetProgress returns
calls map[string]float64 // contentID → last persisted position (either path)
viaPath map[string]string // contentID → "create" | "update"
}
func (f *positionRecordingProgressFake) GetProgress(_ context.Context, _, _, contentID string) (*ProgressRow, error) {
f.mu.Lock()
defer f.mu.Unlock()
if r, ok := f.seeded[contentID]; ok {
return r, nil
}
return nil, nil
}
func (f *positionRecordingProgressFake) UpsertProgress(_ context.Context, row ProgressRow) error {
f.mu.Lock()
defer f.mu.Unlock()
f.recordLocked(row.ContentID, row.CurrentSeconds, "create")
return nil
}
func (f *positionRecordingProgressFake) UpdateProgressPosition(_ context.Context, _, _, contentID string, pos float64) error {
f.mu.Lock()
defer f.mu.Unlock()
f.recordLocked(contentID, pos, "update")
return nil
}
func (f *positionRecordingProgressFake) recordLocked(contentID string, pos float64, path string) {
if f.calls == nil {
f.calls = map[string]float64{}
f.viaPath = map[string]string{}
}
f.calls[contentID] = pos
f.viaPath[contentID] = path
}
func (f *positionRecordingProgressFake) pos(contentID string) (float64, bool) {
f.mu.Lock()
defer f.mu.Unlock()
v, ok := f.calls[contentID]
return v, ok
}
func (f *positionRecordingProgressFake) path(contentID string) string {
f.mu.Lock()
defer f.mu.Unlock()
return f.viaPath[contentID]
}
func TestSyncLocalSession_UpdatesPosition(t *testing.T) {
prog := &positionRecordingProgressFake{}
media := &stubMediaStore{known: map[string]*models.MediaItem{"book-1": nil}}
pub := &recordingPublisher{}
h := New(Dependencies{MediaStore: media, ProgressStore: prog, Publisher: pub})
body := []byte(`{"id":"sess-1","libraryItemId":"book-1","currentTime":123.5,"timeListening":60}`)
rec := dispatchABSWithParams(http.MethodPost, "/api/session/local", nil, body, "1", "", h.handleSyncLocalSession)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
got, ok := prog.pos("book-1")
if !ok {
t.Fatalf("progress not persisted for book-1")
}
if got != 123.5 {
t.Errorf("position = %v, want 123.5", got)
}
// No pre-existing row: the position must be persisted via a create (upsert),
// not the UPDATE-only path that would silently drop an offline-only book.
if p := prog.path("book-1"); p != "create" {
t.Errorf("persist path = %q, want create", p)
}
// Realtime event should fire so other clients catch up.
found := false
for _, ev := range pub.snapshot() {
if ev.Event == "user_item_progress_updated" {
found = true
}
}
if !found {
t.Errorf("expected user_item_progress_updated event")
}
}
func TestSyncLocalSession_ExistingRow_UsesUpdate(t *testing.T) {
// A row already exists (book was played online before), so the offline sync
// must advance it via the monotonic UPDATE path — not recreate it — to avoid
// clobbering is_finished / progress_pct the user set explicitly.
prog := &positionRecordingProgressFake{
seeded: map[string]*ProgressRow{"book-1": {ContentID: "book-1", CurrentSeconds: 5}},
}
media := &stubMediaStore{known: map[string]*models.MediaItem{"book-1": nil}}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
body := []byte(`{"id":"sess-1","libraryItemId":"book-1","currentTime":200}`)
rec := dispatchABSWithParams(http.MethodPost, "/api/session/local", nil, body, "1", "", h.handleSyncLocalSession)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
if p := prog.path("book-1"); p != "update" {
t.Errorf("persist path = %q, want update", p)
}
if got, _ := prog.pos("book-1"); got != 200 {
t.Errorf("position = %v, want 200", got)
}
}
func TestSyncLocalSession_UnknownItem_500(t *testing.T) {
prog := &positionRecordingProgressFake{}
media := &stubMediaStore{known: map[string]*models.MediaItem{}}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
body := []byte(`{"id":"sess-1","libraryItemId":"ghost","currentTime":10}`)
rec := dispatchABSWithParams(http.MethodPost, "/api/session/local", nil, body, "1", "", h.handleSyncLocalSession)
if rec.Code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500; body=%s", rec.Code, rec.Body.String())
}
if _, ok := prog.pos("ghost"); ok {
t.Errorf("position should not be written for unknown item")
}
}
func TestSyncLocalSessions_Batch_ResultsShape(t *testing.T) {
prog := &positionRecordingProgressFake{}
media := &stubMediaStore{known: map[string]*models.MediaItem{"book-1": nil, "book-2": nil}}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
body := []byte(`{"sessions":[
{"id":"s1","libraryItemId":"book-1","currentTime":11},
{"id":"s2","libraryItemId":"ghost","currentTime":22},
{"id":"s3","libraryItemId":"book-2","currentTime":33}
]}`)
rec := dispatchABSWithParams(http.MethodPost, "/api/session/local-all", nil, body, "1", "", h.handleSyncLocalSessions)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp struct {
Results []localSyncResult `json:"results"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v; body=%s", err, rec.Body.String())
}
if len(resp.Results) != 3 {
t.Fatalf("results len = %d, want 3", len(resp.Results))
}
// s1 and s3 succeed; the unknown ghost item fails but does not sink the batch.
if !resp.Results[0].Success || !resp.Results[0].ProgressSynced {
t.Errorf("s1 result = %+v, want success+synced", resp.Results[0])
}
if resp.Results[1].Success {
t.Errorf("s2 (ghost) should not succeed: %+v", resp.Results[1])
}
if !resp.Results[2].Success {
t.Errorf("s3 result = %+v, want success", resp.Results[2])
}
if p, _ := prog.pos("book-1"); p != 11 {
t.Errorf("book-1 position = %v, want 11", p)
}
if p, _ := prog.pos("book-2"); p != 33 {
t.Errorf("book-2 position = %v, want 33", p)
}
}
func TestSyncLocalSessions_MalformedSessionSkipped(t *testing.T) {
prog := &positionRecordingProgressFake{}
media := &stubMediaStore{known: map[string]*models.MediaItem{"book-1": nil}}
h := New(Dependencies{MediaStore: media, ProgressStore: prog})
// Second session is not an object — must be skipped, not fatal.
body := []byte(`{"sessions":[{"id":"s1","libraryItemId":"book-1","currentTime":5}, 42]}`)
rec := dispatchABSWithParams(http.MethodPost, "/api/session/local-all", nil, body, "1", "", h.handleSyncLocalSessions)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
}
var resp struct {
Results []localSyncResult `json:"results"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
if len(resp.Results) != 2 {
t.Fatalf("results len = %d, want 2", len(resp.Results))
}
if !resp.Results[0].Success {
t.Errorf("s1 should succeed: %+v", resp.Results[0])
}
if resp.Results[1].Success {
t.Errorf("malformed session should be marked failure: %+v", resp.Results[1])
}
}
internal/audiobooks/abs/smart_collections_handler.go
+1 -1
View File
@@ -325,7 +325,7 @@ func (h *Handler) handleSmartCollectionItems(w http.ResponseWriter, r *http.Requ
candidates := make([]smartcoll.Candidate, 0, 256)
for _, lib := range targetLibs {
items, _, lerr := h.deps.MediaStore.ListAudiobooks(r.Context(), lib.ID, 0, 0, access)
items, _, lerr := h.deps.MediaStore.ListAudiobooks(r.Context(), lib.ID, 0, 0, access, Filter{})
if lerr != nil {
slog.Warn("abs smart collection list-audiobooks failed", "err", lerr, "library", lib.ID)
continue
internal/audiobooks/abs/smart_collections_handler_test.go
+1 -1
View File
@@ -110,7 +110,7 @@ type itemListStubMediaStore struct {
items []*models.MediaItem
}
func (s *itemListStubMediaStore) ListAudiobooks(_ context.Context, _ int64, _, _ int, _ catalog.AccessFilter) ([]*models.MediaItem, int, error) {
func (s *itemListStubMediaStore) ListAudiobooks(_ context.Context, _ int64, _, _ int, _ catalog.AccessFilter, _ Filter) ([]*models.MediaItem, int, error) {
return s.items, len(s.items), nil
}
internal/audiobooks/abs/types.go
+96 -53
View File
@@ -61,9 +61,9 @@ type AudioTrackMetadata struct {
type AudioTrack struct {
Index int `json:"index"`
Ino string `json:"ino"`
Metadata *AudioTrackMetadata `json:"metadata,omitempty"`
AddedAt int64 `json:"addedAt,omitempty"`
UpdatedAt int64 `json:"updatedAt,omitempty"`
Metadata *AudioTrackMetadata `json:"metadata"`
AddedAt int64 `json:"addedAt"`
UpdatedAt int64 `json:"updatedAt"`
TrackNumFromMeta *int `json:"trackNumFromMeta"`
DiscNumFromMeta *int `json:"discNumFromMeta"`
TrackNumFromFilename *int `json:"trackNumFromFilename"`
@@ -71,21 +71,26 @@ type AudioTrack struct {
ManuallyVerified bool `json:"manuallyVerified"`
Exclude bool `json:"exclude"`
Error *string `json:"error"`
Format string `json:"format,omitempty"`
Duration float64 `json:"duration"`
BitRate int `json:"bitRate,omitempty"`
Language *string `json:"language"`
Codec string `json:"codec,omitempty"`
TimeBase string `json:"timeBase,omitempty"`
Channels int `json:"channels,omitempty"`
ChannelLayout string `json:"channelLayout,omitempty"`
Chapters []ChapterABS `json:"chapters,omitempty"`
EmbeddedCoverArt any `json:"embeddedCoverArt"`
MetaTags map[string]string `json:"metaTags,omitempty"`
MimeType string `json:"mimeType"`
Title string `json:"title,omitempty"`
StartOffset float64 `json:"startOffset"`
ContentURL string `json:"contentUrl"`
// Real ABS AudioFile/AudioTrack ALWAYS emit every key below. Strict
// clients (Prologue, yaabsa) decode these into a model with required
// fields, so a key omitted by omitempty on an empty value throws
// keyNotFound and the player reports "Unable to load book contents".
// chapters/metaTags must be [] / {} (non-nil), never dropped, never null.
Format string `json:"format"`
Duration float64 `json:"duration"`
BitRate int `json:"bitRate"`
Language *string `json:"language"`
Codec string `json:"codec"`
TimeBase string `json:"timeBase"`
Channels int `json:"channels"`
ChannelLayout string `json:"channelLayout"`
Chapters []ChapterABS `json:"chapters"`
EmbeddedCoverArt any `json:"embeddedCoverArt"`
MetaTags map[string]string `json:"metaTags"`
MimeType string `json:"mimeType"`
Title string `json:"title"`
StartOffset float64 `json:"startOffset"`
ContentURL string `json:"contentUrl"`
}
// Metadata is the book-level metadata block. Authors / Narrators / Series
@@ -94,21 +99,35 @@ type AudioTrack struct {
// clients (Plappa, AudioBookShelfFully) branch on these keys being present
// (even if empty), and dropping the key sends them into degraded mode.
type Metadata struct {
Title string `json:"title"`
Authors []AuthorObj `json:"authors"`
Narrators []string `json:"narrators"`
Series []SeriesObj `json:"series"`
Description string `json:"description,omitempty"`
PublishedYear string `json:"publishedYear,omitempty"`
ISBN string `json:"isbn,omitempty"`
Publisher string `json:"publisher,omitempty"`
Genres []string `json:"genres"`
Tags []string `json:"tags"`
Title string `json:"title"`
TitleIgnorePrefix string `json:"titleIgnorePrefix"`
Subtitle string `json:"subtitle"`
Authors []AuthorObj `json:"authors"`
AuthorName string `json:"authorName"`
AuthorNameLF string `json:"authorNameLF"`
Narrators []string `json:"narrators"`
NarratorName string `json:"narratorName"`
Series []SeriesObj `json:"series"`
SeriesName string `json:"seriesName"`
Genres []string `json:"genres"`
// Nullable-in-ABS string fields are emitted as "" (never dropped) —
// real ABS oldMetadataToJSON always includes the key; a MISSING key is
// what crashes strict clients, an empty string is safe.
PublishedYear string `json:"publishedYear"`
PublishedDate string `json:"publishedDate"`
Publisher string `json:"publisher"`
Description string `json:"description"`
DescriptionPlain string `json:"descriptionPlain"`
ISBN string `json:"isbn"`
ASIN string `json:"asin"`
Language string `json:"language"`
// Explicit is a content-warning flag the Kotlin BookMetadata declares
// as non-nullable Boolean. Always emit (default false). silo does not
// track per-item explicit metadata today; surface it when scanner-side
// support lands.
Explicit bool `json:"explicit"`
Explicit bool `json:"explicit"`
Abridged bool `json:"abridged"`
Tags []string `json:"tags"`
}
// LibraryItemMedia carries the bulk of the audiobook metadata.
@@ -119,13 +138,24 @@ type Metadata struct {
// media.tracks.length to decide whether to render the play button, while
// card/list views read media.numTracks.
type LibraryItemMedia struct {
Metadata Metadata `json:"metadata"`
Duration float64 `json:"duration"`
CoverPath string `json:"coverPath"`
AudioFiles []AudioTrack `json:"audioFiles"`
Tracks []AudioTrack `json:"tracks"`
Chapters []ChapterABS `json:"chapters"`
NumTracks int `json:"numTracks"`
// ID and LibraryItemID are BOTH the ContentID. Real ABS Book.toOldJSON
// sets media.id (= book id) and media.libraryItemId; yaabsa's BookMedia.id
// is required non-null, so omitting id throws
// "type 'Null' is not a subtype of type 'String'" and the whole item fails
// to parse. Every non-minified media object (list-full + detail) carries both.
ID string `json:"id"`
LibraryItemID string `json:"libraryItemId"`
Metadata Metadata `json:"metadata"`
Duration float64 `json:"duration"`
CoverPath string `json:"coverPath"`
AudioFiles []AudioTrack `json:"audioFiles"`
Tracks []AudioTrack `json:"tracks"`
Chapters []ChapterABS `json:"chapters"`
NumTracks int `json:"numTracks"`
// Size is the summed byte size of the media (real ABS
// Book.toOldJSONExpanded media.size). 0 when the catalog has no per-file
// byte sizes; populated by the detail builder.
Size int64 `json:"size"`
// Tags is a book-level tag list. NEVER null on the wire — the ABS
// Android client's Kotlin `Book.tags: List<String>` is non-nullable,
// so Jackson throws MissingKotlinParameterException when the field
@@ -187,26 +217,39 @@ type CollapsedSeriesV1 struct {
// echoed across the three time fields). Costs almost nothing on the wire
// and never causes a parser failure that silently breaks downloads.
type LibraryItem struct {
ID string `json:"id"`
Ino string `json:"ino"`
LibraryID string `json:"libraryId"`
FolderID string `json:"folderId"`
Path string `json:"path"`
RelPath string `json:"relPath"`
MtimeMs int64 `json:"mtimeMs"`
CtimeMs int64 `json:"ctimeMs"`
BirthtimeMs int64 `json:"birthtimeMs"`
MediaType string `json:"mediaType"`
ID string `json:"id"`
Ino string `json:"ino"`
LibraryID string `json:"libraryId"`
FolderID string `json:"folderId"`
Path string `json:"path"`
RelPath string `json:"relPath"`
// IsFile mirrors real ABS LibraryItem.isFile (single-file vs folder item).
// silo serves file-backed items, so this is always true; the minified
// projection carries it through and some clients gate download UI on it.
IsFile bool `json:"isFile"`
MtimeMs int64 `json:"mtimeMs"`
CtimeMs int64 `json:"ctimeMs"`
BirthtimeMs int64 `json:"birthtimeMs"`
MediaType string `json:"mediaType"`
// IsMissing / IsInvalid are gating fields the ABS mobile client checks
// before rendering the play affordance. We always emit them (no omitempty)
// so the client never sees them as undefined; the catalog we serve is by
// definition present and valid.
// Ref: /opt/audiobookshelf-app/pages/item/_id/index.vue:445
IsMissing bool `json:"isMissing"`
IsInvalid bool `json:"isInvalid"`
Media LibraryItemMedia `json:"media"`
NumTracks int `json:"numTracks,omitempty"`
AddedAt int64 `json:"addedAt"`
UpdatedAt int64 `json:"updatedAt"`
CollapsedSeries *CollapsedSeriesV1 `json:"collapsedSeries,omitempty"`
IsMissing bool `json:"isMissing"`
IsInvalid bool `json:"isInvalid"`
// OldLibraryItemID / LastScan / ScanVersion / Size / LibraryFiles mirror
// real ABS LibraryItem.toOldJSONExpanded. The detail builder populates
// LibraryFiles + Size; on the (rare) non-minified list path they default
// to empty/zero. OldLibraryItemID is always null (silo has no legacy IDs).
OldLibraryItemID *string `json:"oldLibraryItemId"`
LastScan int64 `json:"lastScan"`
ScanVersion string `json:"scanVersion"`
Media LibraryItemMedia `json:"media"`
LibraryFiles []map[string]any `json:"libraryFiles"`
Size int64 `json:"size"`
NumTracks int `json:"numTracks,omitempty"`
AddedAt int64 `json:"addedAt"`
UpdatedAt int64 `json:"updatedAt"`
CollapsedSeries *CollapsedSeriesV1 `json:"collapsedSeries,omitempty"`
}
internal/audiobooks/cred_validator.go
+37 -6
View File
@@ -27,13 +27,13 @@ type SiloCredValidator struct {
// Accepts two username formats:
//
// - "alice" — primary profile for user alice. profileID will be the
// alice user's primary profile.
// alice user's primary profile.
// - "alice#kids" — the "kids" household profile under user alice.
// Authentication still uses alice's password; only the
// profile selector differs. If alice has no profile
// named "kids" (case-insensitive), returns
// ErrInvalidCredentials so we don't leak which arm of
// the user#profile pair was wrong.
// Authentication still uses alice's password; only the
// profile selector differs. If alice has no profile
// named "kids" (case-insensitive), returns
// ErrInvalidCredentials so we don't leak which arm of
// the user#profile pair was wrong.
//
// The ABS compat layer represents user/profile IDs as strings; we format the
// integer user ID and UUID profile ID accordingly. Users with no profiles at
@@ -92,6 +92,37 @@ func (v *SiloCredValidator) Validate(
return userIDStr, profileID, displayName, nil
}
// ResolveUsername returns the ABS display name for (userID, profileID) without
// re-authenticating — used by GET /me, which only has the token claims. It
// mirrors Validate's display-name logic: the profile name when a profile is
// set and named, otherwise the account username. Returns "" when unresolvable
// so the caller falls back to the userID.
func (v *SiloCredValidator) ResolveUsername(ctx context.Context, userID, profileID string) string {
if v.Pool == nil {
return ""
}
if strings.TrimSpace(profileID) != "" {
var name string
err := v.Pool.QueryRow(ctx,
`SELECT name FROM user_profiles WHERE id = $1`, profileID,
).Scan(&name)
if err == nil && strings.TrimSpace(name) != "" {
return name
}
}
uid, err := strconv.Atoi(userID)
if err != nil {
return ""
}
var username string
if err := v.Pool.QueryRow(ctx,
`SELECT username FROM users WHERE id = $1`, uid,
).Scan(&username); err == nil {
return username
}
return ""
}
// splitUserProfile separates a username like "alice#kids" into the
// authentication username ("alice") and the profile selector ("kids").
// Plain "alice" returns ("alice", ""). Whitespace is trimmed on both sides.
internal/audiobooks/enrichment.go
+23 -9
View File
@@ -15,6 +15,7 @@ package audiobooks
import (
"context"
"errors"
"fmt"
"log/slog"
"os"
@@ -382,19 +383,18 @@ func (e *Enricher) enrichItem(ctx context.Context, item enrichmentItemRow) error
// Phase 1: Search — collect provider IDs.
searchQuery := metadata.SearchQuery{
Title: item.Title,
Author: item.Author,
Year: item.Year,
ContentType: "audiobook",
ProviderIDs: accumulatedIDs,
Language: item.Language,
}
// Include author in the search query title hint when present.
// The audnexus plugin uses title+author for ASIN lookup.
if item.Author != "" {
searchQuery.Title = item.Title
// Some plugins accept author via the generic extras pathway; others rely
// on the title field only. We pass it as a secondary field (no standard
// slot exists in SearchQuery yet).
}
// Track whether any provider errored during this item's enrichment. When
// nothing is accumulated AND at least one provider errored, we return an
// error WITHOUT stamping last_refreshed so the sweep retries the item later
// rather than burning it terminally on a transient provider failure.
var providerErrs []error
for _, p := range providers {
sp, ok := p.(metadata.SearchProvider)
@@ -408,6 +408,7 @@ func (e *Enricher) enrichItem(ctx context.Context, item enrichmentItemRow) error
"content_id", item.ContentID,
"error", searchErr,
)
providerErrs = append(providerErrs, fmt.Errorf("%s search: %w", p.Slug(), searchErr))
continue
}
if len(results) == 0 {
@@ -449,6 +450,7 @@ func (e *Enricher) enrichItem(ctx context.Context, item enrichmentItemRow) error
"content_id", item.ContentID,
"error", getErr,
)
providerErrs = append(providerErrs, fmt.Errorf("%s metadata: %w", p.Slug(), getErr))
continue
}
if result == nil || !result.HasMetadata {
@@ -467,8 +469,20 @@ func (e *Enricher) enrichItem(ctx context.Context, item enrichmentItemRow) error
)
}
// Nothing found — stamp last_refreshed so we skip on the next sweep.
// Nothing found.
if !accumulator.HasMetadata && accumulator.PosterPath == "" && accumulator.Overview == "" {
if err := ctx.Err(); err != nil {
// A cancelled sweep says nothing about the item or the providers.
return err
}
if len(providerErrs) > 0 {
// Transient provider trouble must not stamp the item terminally;
// surfacing an error lets the sweep retry it later instead.
return fmt.Errorf("no metadata obtained, %d provider error(s): %w",
len(providerErrs), errors.Join(providerErrs...))
}
// Providers ran cleanly but nothing matched — stamp last_refreshed so we
// skip on the next sweep.
slog.Info("audiobook enrichment: no metadata found",
"content_id", item.ContentID,
"title", item.Title,
internal/audiobooks/media_store.go
+328 -59
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"strconv"
"strings"
"sync"
"time"
"github.com/jackc/pgx/v5"
@@ -23,6 +24,59 @@ type ABSMediaStore struct {
Items *catalog.ItemRepository
Files *scanner.FileRepository
Pool *pgxpool.Pool
// countCache memoizes the per-(library, filter, access) COUNT(*) that
// ListAudiobooks would otherwise recompute on every page. A full client
// library sync pages through thousands of requests reading the same total;
// the count only shifts when the library changes, so a short TTL is safe.
countMu sync.Mutex
countCache map[string]absCountEntry
}
type absCountEntry struct {
n int
exp time.Time
}
// absCountCacheTTL bounds how stale the paginated total may be. During an active
// scan the count can lag by up to this window; clients re-sync, so that is fine.
const absCountCacheTTL = 60 * time.Second
// cachedAudiobookCount returns a memoized COUNT(*) for the given (countSQL, args)
// pair, running the query on a miss/expiry. The key is derived from the fully
// rendered SQL plus its bound args, so it automatically covers every input the
// count WHERE depends on (library, pushed-down filter, and all access
// predicates) — it can't drift as access logic evolves. The DB query runs
// outside the lock so concurrent syncs don't serialize on it.
func (s *ABSMediaStore) cachedAudiobookCount(ctx context.Context, countSQL string, args []any) (int, error) {
key := countSQL + "\x1f" + fmt.Sprintf("%v", args)
now := time.Now()
s.countMu.Lock()
if e, ok := s.countCache[key]; ok && now.Before(e.exp) {
s.countMu.Unlock()
return e.n, nil
}
s.countMu.Unlock()
var n int
if err := s.Pool.QueryRow(ctx, countSQL, args...).Scan(&n); err != nil {
return 0, err
}
s.countMu.Lock()
if s.countCache == nil {
s.countCache = make(map[string]absCountEntry)
}
// Sweep expired entries on write so per-filter keys (e.g. one per author
// during a sync) don't accumulate for the process lifetime.
for k, e := range s.countCache {
if now.After(e.exp) {
delete(s.countCache, k)
}
}
s.countCache[key] = absCountEntry{n: n, exp: now.Add(absCountCacheTTL)}
s.countMu.Unlock()
return n, nil
}
var _ abs.MediaStore = (*ABSMediaStore)(nil)
@@ -98,7 +152,7 @@ func (s *ABSMediaStore) GetAudiobooksByIDs(ctx context.Context, contentIDs []str
// path that bails out when the query string is empty. We page content_ids
// here via SQL, then load full rows via GetByIDs so the scan logic stays
// in the catalog package.
func (s *ABSMediaStore) ListAudiobooks(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter) ([]*models.MediaItem, int, error) {
func (s *ABSMediaStore) ListAudiobooks(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter, filter abs.Filter) ([]*models.MediaItem, int, error) {
if s.Pool == nil {
return nil, 0, fmt.Errorf("abs_media_store: no pgx pool")
}
@@ -119,10 +173,14 @@ func (s *ABSMediaStore) ListAudiobooks(ctx context.Context, libraryID int64, lim
argIdx++
}
appendAudiobookAccessConditions("mi", access, &conditions, &args, &argIdx)
// Push author/series/narrator filters into SQL so per-author album syncs
// don't load and hydrate the entire library on every request. Uses the
// idx_item_people_content_kind_person and audiobook_series indexes.
appendAudiobookFilterConditions(filter, &conditions, &args, &argIdx)
where := strings.Join(conditions, " AND ")
countSQL := `SELECT COUNT(*) FROM media_items mi WHERE ` + where
if err := s.Pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil {
total, err := s.cachedAudiobookCount(ctx, `SELECT COUNT(*) FROM media_items mi WHERE `+where, args)
if err != nil {
return nil, 0, fmt.Errorf("abs_media_store: count audiobooks: %w", err)
}
if total == 0 {
@@ -130,7 +188,12 @@ func (s *ABSMediaStore) ListAudiobooks(ctx context.Context, libraryID int64, lim
}
dataArgs := append([]any(nil), args...)
dataSQL := `SELECT mi.content_id FROM media_items mi WHERE ` + where + ` ORDER BY LOWER(mi.sort_title), LOWER(mi.title)`
// Order by the same expression idx_media_items_sort_key is built on so the
// page can be served by an ordered index scan instead of sorting the whole
// library on every request; content_id is a stable tiebreaker so sequential
// pages don't skip or repeat rows when sort keys collide.
dataSQL := `SELECT mi.content_id FROM media_items mi WHERE ` + where +
` ORDER BY lower(coalesce(nullif(btrim(mi.sort_title), ''), mi.title)), mi.content_id`
if limit > 0 {
argIdx = len(dataArgs) + 1
dataSQL += fmt.Sprintf(` LIMIT $%d OFFSET $%d`, argIdx, argIdx+1)
@@ -187,6 +250,46 @@ func (s *ABSMediaStore) ListAudiobooks(ctx context.Context, libraryID int64, lim
return ordered, total, nil
}
// appendAudiobookFilterConditions pushes an ABS authors/series/narrators
// filter down into the SQL WHERE clause. Author values are the numeric
// person_id (as returned by the /authors endpoint), with a name fallback for
// non-numeric values; narrator and series values are names. This mirrors the
// case-sensitive exact-match identity semantics of abs.Filter.Matches.
// Progress/genre/tag/language filters are not pushable here (they need
// per-user or detail data) and are handled by the caller in Go.
func appendAudiobookFilterConditions(filter abs.Filter, conditions *[]string, args *[]any, argIdx *int) {
switch filter.Kind {
case abs.FilterAuthors:
if id, err := strconv.ParseInt(filter.Value, 10, 64); err == nil {
*conditions = append(*conditions, fmt.Sprintf(
`EXISTS (SELECT 1 FROM item_people ip WHERE ip.content_id = mi.content_id AND ip.kind = %d AND ip.person_id = $%d)`,
models.PersonKindAuthor, *argIdx))
*args = append(*args, id)
} else {
*conditions = append(*conditions, fmt.Sprintf(
`EXISTS (SELECT 1 FROM item_people ip JOIN people p ON p.id = ip.person_id WHERE ip.content_id = mi.content_id AND ip.kind = %d AND p.name = $%d)`,
models.PersonKindAuthor, *argIdx))
*args = append(*args, filter.Value)
}
*argIdx = *argIdx + 1
case abs.FilterNarrators:
*conditions = append(*conditions, fmt.Sprintf(
`EXISTS (SELECT 1 FROM item_people ip JOIN people p ON p.id = ip.person_id WHERE ip.content_id = mi.content_id AND ip.kind = %d AND p.name = $%d)`,
models.PersonKindNarrator, *argIdx))
*args = append(*args, filter.Value)
*argIdx = *argIdx + 1
case abs.FilterSeries:
if filter.Value == abs.SentinelNoSeries {
*conditions = append(*conditions, `NOT EXISTS (SELECT 1 FROM audiobook_series abs WHERE abs.content_id = mi.content_id)`)
} else {
*conditions = append(*conditions, fmt.Sprintf(
`EXISTS (SELECT 1 FROM audiobook_series abs WHERE abs.content_id = mi.content_id AND abs.series_name = $%d)`, *argIdx))
*args = append(*args, filter.Value)
*argIdx = *argIdx + 1
}
}
}
func appendAudiobookAccessConditions(alias string, filter catalog.AccessFilter, conditions *[]string, args *[]any, argIdx *int) {
if filter.AllowedLibraryIDs != nil {
if len(filter.AllowedLibraryIDs) == 0 {
@@ -440,42 +543,67 @@ func (s *ABSMediaStore) listAudiobookIDs(ctx context.Context, sql string, args [
return ordered, nil
}
// SearchAudiobooks matches the query against title (case-insensitive
// substring) plus author/narrator name. Capped by limit; ordered by
// title-prefix match first then alphabetical.
// SearchAudiobooks matches the query against title plus author/narrator name.
// Capped by limit; ordered by title-prefix match first, then title substring,
// then author/narrator matches, then alphabetical.
//
// Both arms are shaped to hit the existing pg_trgm GIN indexes rather than
// seq-scanning the whole library: the title arm matches media_items.
// title_normalized (idx_media_items_title_normalized_trgm) using the same
// normalize_search_text() the rest of catalog search uses, and the people arm
// matches people.name (idx_people_name_trgm). Running them as a UNION lets the
// planner drive each arm from its own index; GROUP BY content_id keeps the best
// rank when an item matches both. normalize_search_text($2) <> ” guards a
// punctuation-only query from degenerating into ILIKE '%%' over everything.
func (s *ABSMediaStore) SearchAudiobooks(ctx context.Context, libraryID int64, query string, limit int, access catalog.AccessFilter) ([]*models.MediaItem, error) {
if limit <= 0 {
limit = 12
}
conditions := []string{`mi.type = 'audiobook'`, `(
mi.title ILIKE $1
OR EXISTS (
SELECT 1 FROM item_people ip
JOIN people p ON p.id = ip.person_id
WHERE ip.content_id = mi.content_id
AND ip.kind IN (7, 8)
AND p.name ILIKE $1
)
)`}
// $1 = raw-query substring pattern for people.name; $2 = raw query, normalized
// in-SQL for the title arm.
args := []any{"%" + query + "%", query}
argIdx := 3
// Library + access predicates are identical in both UNION arms and reuse the
// same positional placeholders, so append their args only once.
scopeConds := []string{}
if libraryID != 0 {
conditions = append(conditions, fmt.Sprintf(`EXISTS (
scopeConds = append(scopeConds, fmt.Sprintf(`EXISTS (
SELECT 1 FROM media_item_libraries mil
WHERE mil.content_id = mi.content_id AND mil.media_folder_id = $%d
)`, argIdx))
args = append(args, int(libraryID))
argIdx++
}
appendAudiobookAccessConditions("mi", access, &conditions, &args, &argIdx)
appendAudiobookAccessConditions("mi", access, &scopeConds, &args, &argIdx)
scope := ""
if len(scopeConds) > 0 {
scope = " AND " + strings.Join(scopeConds, " AND ")
}
args = append(args, limit)
sql := `
SELECT mi.content_id FROM media_items mi
WHERE ` + strings.Join(conditions, " AND ") + `
ORDER BY
CASE WHEN LOWER(mi.title) LIKE LOWER($2) || '%' THEN 0 ELSE 1 END,
LOWER(mi.sort_title),
LOWER(mi.title)
SELECT content_id FROM (
SELECT mi.content_id AS content_id, mi.sort_title AS sort_title, mi.title AS title,
CASE WHEN mi.title_normalized LIKE normalize_search_text($2) || '%' THEN 0 ELSE 1 END AS rank
FROM media_items mi
WHERE mi.type = 'audiobook'
AND normalize_search_text($2) <> ''
AND mi.title_normalized ILIKE '%' || normalize_search_text($2) || '%'` + scope + `
UNION ALL
SELECT mi.content_id AS content_id, mi.sort_title AS sort_title, mi.title AS title, 2 AS rank
FROM media_items mi
WHERE mi.type = 'audiobook'
AND EXISTS (
SELECT 1 FROM item_people ip
JOIN people p ON p.id = ip.person_id
WHERE ip.content_id = mi.content_id
AND ip.kind IN (7, 8)
AND p.name ILIKE $1
)` + scope + `
) m
GROUP BY content_id, sort_title, title
ORDER BY MIN(rank), LOWER(sort_title), LOWER(title)
LIMIT $` + strconv.Itoa(argIdx) + `
`
return s.listAudiobookIDs(ctx, sql, args)
@@ -579,18 +707,104 @@ func (s *ABSMediaStore) ListDiscover(ctx context.Context, libraryID int64, limit
return s.listAudiobookIDs(ctx, sql, args)
}
// ListLibraryAuthors aggregates audiobook authors (item_people kind=7)
// for the library, returning distinct (person_id, name, book_count).
func (s *ABSMediaStore) ListLibraryAuthors(ctx context.Context, libraryID int64, limit int, access catalog.AccessFilter) ([]abs.AuthorSummary, error) {
// RefreshAuthorCounts rebuilds the abs_audiobook_author_counts materialized
// view that ListLibraryAuthors reads. CONCURRENTLY keeps it readable during the
// refresh (requires the unique index). Driven by a periodic ticker in the
// audiobooks service.
func (s *ABSMediaStore) RefreshAuthorCounts(ctx context.Context) error {
if s.Pool == nil {
return nil, nil
return nil
}
if limit <= 0 {
limit = 100
_, err := s.Pool.Exec(ctx, `REFRESH MATERIALIZED VIEW CONCURRENTLY abs_audiobook_author_counts`)
if err != nil {
return fmt.Errorf("abs_media_store: refresh author counts: %w", err)
}
args := []any{limit}
return nil
}
// ListLibraryAuthors returns one page of distinct audiobook authors plus the
// total author count for the library. It reads the precomputed
// abs_audiobook_author_counts materialized view keyed by media_folder_id.
//
// The MV is keyed by library_id only and carries no per-item access predicate,
// so it is safe only when the caller has no item-level restriction. When the
// access filter carries an item-level predicate (a content-rating cap or
// excluded media types), reading the MV would leak authors of books the caller
// can't see, so we take the access-aware live path instead. Library-level
// access is already enforced by scoping to a single resolved library.
//
// When the MV read returns zero rows (stale, empty, or not-yet-refreshed view)
// we also fall back to the live GROUP BY so /authors never blanks out.
func (s *ABSMediaStore) ListLibraryAuthors(ctx context.Context, libraryID int64, limit, offset int, sortBy string, sortDesc bool, access catalog.AccessFilter) ([]abs.AuthorSummary, int, error) {
if s.Pool == nil {
return nil, 0, nil
}
if offset < 0 {
offset = 0
}
if access.MaxContentRating != "" || len(access.ExcludedMediaTypes) > 0 {
return s.listLibraryAuthorsLive(ctx, libraryID, limit, offset, sortBy, sortDesc, access)
}
var total int
if err := s.Pool.QueryRow(ctx,
`SELECT COUNT(*) FROM abs_audiobook_author_counts WHERE library_id = $1`, int(libraryID),
).Scan(&total); err != nil {
return nil, 0, fmt.Errorf("abs_media_store: count authors: %w", err)
}
if total == 0 {
// Stale/empty/unrefreshed MV: fall back to the live query so a fresh
// deploy (before the first REFRESH) still serves authors.
return s.listLibraryAuthorsLive(ctx, libraryID, limit, offset, sortBy, sortDesc, access)
}
dir := "ASC"
if sortDesc {
dir = "DESC"
}
var orderBy string
switch sortBy {
case "addedAt":
orderBy = "added_at " + dir + ", person_id"
case "numBooks":
orderBy = "num_books " + dir + ", LOWER(name)"
default: // name
orderBy = "LOWER(name) " + dir
}
dataSQL := `SELECT person_id, name, num_books, added_at FROM abs_audiobook_author_counts
WHERE library_id = $1 ORDER BY ` + orderBy
args := []any{int(libraryID)}
if limit > 0 {
dataSQL += ` LIMIT $2 OFFSET $3`
args = append(args, limit, offset)
}
rows, err := s.Pool.Query(ctx, dataSQL, args...)
if err != nil {
return nil, 0, fmt.Errorf("abs_media_store: list authors: %w", err)
}
defer rows.Close()
out := make([]abs.AuthorSummary, 0, 64)
for rows.Next() {
var (
id int64
name string
books int
addedAt time.Time
)
if err := rows.Scan(&id, &name, &books, &addedAt); err != nil {
return nil, 0, fmt.Errorf("abs_media_store: scan author: %w", err)
}
out = append(out, abs.AuthorSummary{ID: fmt.Sprintf("%d", id), Name: name, NumBooks: books})
}
return out, total, rows.Err()
}
// listLibraryAuthorsLive is the pre-materialized-view live aggregation, kept as
// a fallback for ListLibraryAuthors when the MV has no rows for the library.
func (s *ABSMediaStore) listLibraryAuthorsLive(ctx context.Context, libraryID int64, limit, offset int, sortBy string, sortDesc bool, access catalog.AccessFilter) ([]abs.AuthorSummary, int, error) {
conditions := []string{`mi.type = 'audiobook'`}
argIdx := 2
args := []any{}
argIdx := 1
if libraryID != 0 {
conditions = append(conditions, fmt.Sprintf(`EXISTS (
SELECT 1 FROM media_item_libraries mil
@@ -600,22 +814,56 @@ func (s *ABSMediaStore) ListLibraryAuthors(ctx context.Context, libraryID int64,
argIdx++
}
appendAudiobookAccessConditions("mi", access, &conditions, &args, &argIdx)
sql := `
where := strings.Join(conditions, " AND ")
var total int
countSQL := `SELECT COUNT(*) FROM (
SELECT p.id
FROM media_items mi
JOIN item_people ip ON ip.content_id = mi.content_id AND ip.kind = 7
JOIN people p ON p.id = ip.person_id
WHERE ` + where + `
GROUP BY p.id, p.name
) t`
if err := s.Pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil {
return nil, 0, fmt.Errorf("abs_media_store: count authors (live): %w", err)
}
if total == 0 {
return []abs.AuthorSummary{}, 0, nil
}
dir := "ASC"
if sortDesc {
dir = "DESC"
}
var orderBy string
switch sortBy {
case "addedAt":
orderBy = "p.created_at " + dir + ", p.id"
case "numBooks":
orderBy = "num_books " + dir + ", LOWER(p.name)"
default: // name
orderBy = "LOWER(p.name) " + dir
}
dataSQL := `
SELECT p.id, p.name, COUNT(DISTINCT mi.content_id) AS num_books
FROM media_items mi
JOIN item_people ip ON ip.content_id = mi.content_id AND ip.kind = 7
JOIN people p ON p.id = ip.person_id
WHERE ` + strings.Join(conditions, " AND ") + `
WHERE ` + where + `
GROUP BY p.id, p.name
ORDER BY LOWER(p.name)
LIMIT $1
`
rows, err := s.Pool.Query(ctx, sql, args...)
ORDER BY ` + orderBy
dataArgs := append([]any(nil), args...)
if limit > 0 {
dataSQL += fmt.Sprintf(` LIMIT $%d OFFSET $%d`, argIdx, argIdx+1)
dataArgs = append(dataArgs, limit, offset)
}
rows, err := s.Pool.Query(ctx, dataSQL, dataArgs...)
if err != nil {
return nil, fmt.Errorf("abs_media_store: list authors: %w", err)
return nil, 0, fmt.Errorf("abs_media_store: list authors (live): %w", err)
}
defer rows.Close()
out := make([]abs.AuthorSummary, 0, limit)
out := make([]abs.AuthorSummary, 0, 64)
for rows.Next() {
var (
id int64
@@ -623,27 +871,27 @@ func (s *ABSMediaStore) ListLibraryAuthors(ctx context.Context, libraryID int64,
books int
)
if err := rows.Scan(&id, &name, &books); err != nil {
return nil, fmt.Errorf("abs_media_store: scan author: %w", err)
return nil, 0, fmt.Errorf("abs_media_store: scan author (live): %w", err)
}
out = append(out, abs.AuthorSummary{ID: fmt.Sprintf("%d", id), Name: name, NumBooks: books})
}
return out, rows.Err()
return out, total, rows.Err()
}
// ListLibrarySeries returns distinct series from audiobook_series for the
// audiobook library, with per-series book count and up to 4 book preview
// rows (content_id + title + updated_at) used by the ABS mobile client
// to render the LazySeriesCard cover stack.
func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64, limit int, access catalog.AccessFilter) ([]abs.SeriesSummary, error) {
func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64, limit, offset int, access catalog.AccessFilter) ([]abs.SeriesSummary, int, error) {
if s.Pool == nil {
return nil, nil
return nil, 0, nil
}
if limit <= 0 {
limit = 100
if offset < 0 {
offset = 0
}
args := []any{limit}
conditions := []string{`mi.type = 'audiobook'`}
argIdx := 2
args := []any{}
argIdx := 1
if libraryID != 0 {
conditions = append(conditions, fmt.Sprintf(`EXISTS (
SELECT 1 FROM media_item_libraries mil
@@ -653,12 +901,30 @@ func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64,
argIdx++
}
appendAudiobookAccessConditions("mi", access, &conditions, &args, &argIdx)
where := strings.Join(conditions, " AND ")
// Count distinct multi-book series (HAVING COUNT > 1) so pagination totals
// match what the data query returns.
var total int
countSQL := `SELECT COUNT(*) FROM (
SELECT s.series_name
FROM audiobook_series s JOIN media_items mi ON mi.content_id = s.content_id
WHERE ` + where + `
GROUP BY s.series_name HAVING COUNT(*) > 1
) t`
if err := s.Pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil {
return nil, 0, fmt.Errorf("abs_media_store: count series: %w", err)
}
if total == 0 {
return []abs.SeriesSummary{}, 0, nil
}
// Two-stage: window-rank books inside each series (lowest series_index
// first), then aggregate the top 4 ids/titles/updated_at into parallel
// arrays. `book_ids[]` is text because content_id is text in this
// schema; parallel `titles[]` and `updated_ats[]` keep iteration
// straightforward in Go without composite type plumbing.
sql := `
dataSQL := `
WITH ranked AS (
SELECT
s.series_name,
@@ -673,7 +939,7 @@ func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64,
FROM audiobook_series s
JOIN media_items mi
ON mi.content_id = s.content_id
WHERE ` + strings.Join(conditions, " AND ") + `
WHERE ` + where + `
)
SELECT
series_name,
@@ -684,15 +950,18 @@ func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64,
FROM ranked
GROUP BY series_name
HAVING MAX(series_count) > 1
ORDER BY LOWER(series_name)
LIMIT $1
`
rows, err := s.Pool.Query(ctx, sql, args...)
ORDER BY LOWER(series_name)`
dataArgs := append([]any(nil), args...)
if limit > 0 {
dataSQL += fmt.Sprintf(` LIMIT $%d OFFSET $%d`, argIdx, argIdx+1)
dataArgs = append(dataArgs, limit, offset)
}
rows, err := s.Pool.Query(ctx, dataSQL, dataArgs...)
if err != nil {
return nil, fmt.Errorf("abs_media_store: list series: %w", err)
return nil, 0, fmt.Errorf("abs_media_store: list series: %w", err)
}
defer rows.Close()
out := make([]abs.SeriesSummary, 0, limit)
out := make([]abs.SeriesSummary, 0, 64)
for rows.Next() {
var (
name string
@@ -702,7 +971,7 @@ func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64,
updatedAts []time.Time
)
if err := rows.Scan(&name, &books, &ids, &titles, &updatedAts); err != nil {
return nil, fmt.Errorf("abs_media_store: scan series: %w", err)
return nil, 0, fmt.Errorf("abs_media_store: scan series: %w", err)
}
previews := make([]abs.SeriesBookPreview, 0, len(ids))
for i := range ids {
@@ -719,7 +988,7 @@ func (s *ABSMediaStore) ListLibrarySeries(ctx context.Context, libraryID int64,
// series row yet, so the slug is stable for a given name.
out = append(out, abs.SeriesSummary{ID: name, Name: name, NumBooks: books, Books: previews})
}
return out, rows.Err()
return out, total, rows.Err()
}
// GetAuthorByID looks up the author by people.id and returns the row
internal/audiobooks/service.go
+31
View File
@@ -3,6 +3,8 @@ package audiobooks
import (
"context"
"fmt"
"log/slog"
"time"
"github.com/Silo-Server/silo-server/internal/audiobooks/abs"
"github.com/Silo-Server/silo-server/internal/audiobooks/abssocket"
@@ -145,11 +147,20 @@ func (s *Service) BuildABSHandler(deps ABSHandlerDeps) *abs.Handler {
socketServer = abssocket.New(secretFn, tokenValidator, nil, nil)
}
// GET /me only has the token's userID; source a resolver from the concrete
// SiloCredValidator (which holds the pgx pool) so /me can show the real
// display username instead of the numeric id.
var usernameResolver func(ctx context.Context, userID, profileID string) string
if scv, ok := deps.Auth.(*SiloCredValidator); ok {
usernameResolver = scv.ResolveUsername
}
h := abs.New(abs.Dependencies{
MediaStore: mediaStore,
TokenStore: tokenStore,
CredValidator: deps.Auth,
AccessResolver: deps.AccessResolver,
UsernameResolver: usernameResolver,
Config: configProvider,
Publisher: nil, // EventPublisher: no-op stub; Socket.io handles realtime
Recommender: buildABSRecommender(deps),
@@ -170,6 +181,26 @@ func (s *Service) BuildABSHandler(deps ABSHandlerDeps) *abs.Handler {
return deps.Detail.PresignImageURL(ctx, path, "poster", variant)
},
})
// Keep the audiobook author materialized view fresh in the background so the
// /authors endpoint stays a fast indexed read as scans add authors. The
// migration populates it initially; this refreshes it on a cadence.
if deps.Pool != nil && mediaStore != nil {
go func() {
ctx := context.Background()
time.Sleep(30 * time.Second) // let startup settle before the first refresh
if err := mediaStore.RefreshAuthorCounts(ctx); err != nil {
slog.Warn("abs: initial author-count refresh failed", "err", err)
}
ticker := time.NewTicker(15 * time.Minute)
defer ticker.Stop()
for range ticker.C {
if err := mediaStore.RefreshAuthorCounts(ctx); err != nil {
slog.Warn("abs: author-count refresh failed", "err", err)
}
}
}()
}
s.ABSHandler = h
return h
}
internal/ebooks/enrichment.go
+101 -7
View File
@@ -11,6 +11,7 @@ import (
"log/slog"
"os"
"reflect"
"regexp"
"strconv"
"strings"
"sync"
@@ -330,7 +331,11 @@ func (e *Enricher) enrichWithProviders(ctx context.Context, item enrichmentItemR
return fmt.Errorf("%w: no metadata providers configured for folder %d", errEnrichmentSkipped, item.FolderID)
}
accumulator, accumulatedIDs, providerErrs := collectEbookMetadata(ctx, item, providers)
var owner providerIDOwnerLookup
if e.providerIDs != nil {
owner = e.providerIDs
}
accumulator, accumulatedIDs, providerErrs := collectEbookMetadata(ctx, item, providers, owner)
if !accumulator.HasMetadata && accumulator.PosterPath == "" && accumulator.Overview == "" {
if err := ctx.Err(); err != nil {
@@ -367,11 +372,21 @@ func (e *Enricher) enrichWithProviders(ctx context.Context, item enrichmentItemR
return nil
}
// providerIDOwnerLookup reports the content item (if any) that already owns a
// given set of durable provider IDs. *catalog.ProviderIDRepository satisfies it.
type providerIDOwnerLookup interface {
FindContentIDByProviderIDs(ctx context.Context, providerIDs map[string]string, itemType, excludeContentID string) (string, error)
}
// collectEbookMetadata queries every provider in the chain and accumulates
// IDs and metadata. Individual provider failures are collected (not fatal) so
// the caller can distinguish "providers answered, no match" from "providers
// were unreachable".
func collectEbookMetadata(ctx context.Context, item enrichmentItemRow, providers []metadata.Provider) (*metadata.MetadataResult, map[string]string, []error) {
// were unreachable". When owner is non-nil, a search-result provider ID already
// claimed by a different content item is skipped: distinct books that resolve to
// the same provider work (e.g. two series volumes searched as the bare series
// name) must not steal each other's identity, which would mis-tag the loser and
// violate the (provider, provider_id, item_type) uniqueness constraint on persist.
func collectEbookMetadata(ctx context.Context, item enrichmentItemRow, providers []metadata.Provider, owner providerIDOwnerLookup) (*metadata.MetadataResult, map[string]string, []error) {
searchQuery, accumulatedIDs := buildEbookSearchQuery(item)
var providerErrs []error
@@ -394,11 +409,32 @@ func collectEbookMetadata(ctx context.Context, item enrichmentItemRow, providers
continue
}
for k, v := range results[0].ProviderIDs {
if v != "" {
if _, exists := accumulatedIDs[k]; !exists {
accumulatedIDs[k] = v
if v == "" {
continue
}
if _, exists := accumulatedIDs[k]; exists {
continue
}
if owner != nil {
ownerID, ownErr := owner.FindContentIDByProviderIDs(ctx, map[string]string{k: v}, ebookContentType(), item.ContentID)
if ownErr != nil {
// Don't claim an ID we couldn't verify is free, and surface
// the error so the item retries rather than terminally
// stamping as "no match".
providerErrs = append(providerErrs, fmt.Errorf("%s ownership check %s=%s: %w", p.Slug(), k, v, ownErr))
continue
}
if ownerID != "" {
slog.Info("ebook enrichment: provider id already owned by another item; skipping match",
"provider", k,
"provider_id", v,
"content_id", item.ContentID,
"owned_by", ownerID,
)
continue
}
}
accumulatedIDs[k] = v
}
slog.Debug("ebook enrichment: search result",
"provider", p.Slug(),
@@ -770,13 +806,71 @@ func filterEbookPeople(people []models.ItemPerson) []models.ItemPerson {
return authors
}
// cleanEbookSearchTitle normalizes a stored title for provider search. Scanner
// titles are often filesystem-derived: underscores stand in for colons or
// spaces ("Exit Strategy_ The Murderbot" / "LTB_067_Micky_Maus"), and
// path-fallback titles keep a trailing " - <Author>" segment. Both wreck a
// title search, so collapse underscores to spaces and drop a trailing author
// suffix (the author is searched as its own field).
// ebookTrailingGroupRE matches a single trailing (...) or [...] group.
var ebookTrailingGroupRE = regexp.MustCompile(`\s*[\(\[]([^\)\]]*)[\)\]]\s*$`)
// ebookSeriesNoiseRE flags a parenthetical as series/edition noise rather than
// part of the real title: a book/volume/part marker, a "#N", or a bare year.
var ebookSeriesNoiseRE = regexp.MustCompile(`(?i)\b(book|bk|vol|volume|series|part|saga|edition|novella?)\b|#\s*\d|^\s*\d{1,4}\s*$|\b(19|20)\d{2}\b`)
// ebookYearOnlyRE matches a parenthetical that is nothing but a year. Years are
// already carried by SearchQuery.Year, so they are dropped from the text rather
// than folded back in.
var ebookYearOnlyRE = regexp.MustCompile(`^\s*(19|20)\d{2}\s*$`)
func cleanEbookSearchTitle(title, author string) string {
title = strings.ReplaceAll(title, "_", " ")
if a := strings.TrimSpace(author); a != "" {
// Strip the author only when it is a true trailing suffix (optionally
// followed by a series/volume parenthetical). Anchoring to the end avoids
// truncating valid title text when " - <author>" appears mid-title.
authorSuffixRE := regexp.MustCompile(`(?i)\s-\s*` + regexp.QuoteMeta(a) + `(?:\s*[\(\[][^)\]]*[\)\]])*\s*$`)
title = authorSuffixRE.ReplaceAllString(title, "")
}
// Normalize trailing series/edition parentheticals. A bare year ("(2019)")
// is dropped because SearchQuery.Year already carries it. A series/volume
// marker ("(The Raven Brothers Book 4)", "[#3]") is UNWRAPPED — its words
// are kept, only the brackets removed — because the volume number is the
// per-volume disambiguator: dropping it makes every entry in a series search
// as the bare series name and collapse onto a single provider work. Other
// parentheticals ("(Illustrated)") are meaningful title text and survive.
for {
m := ebookTrailingGroupRE.FindStringSubmatch(title)
if m == nil {
break
}
inner := strings.TrimSpace(m[1])
base := strings.TrimSpace(title[:len(title)-len(m[0])])
if base == "" {
break // never reduce the title to nothing
}
if ebookYearOnlyRE.MatchString(inner) {
title = base
continue // peel stacked groups (e.g. a year behind a series marker)
}
if ebookSeriesNoiseRE.MatchString(inner) {
title = base + " " + inner
break
}
break // meaningful parenthetical — leave intact
}
return strings.Join(strings.Fields(title), " ")
}
func buildEbookSearchQuery(item enrichmentItemRow) (metadata.SearchQuery, map[string]string) {
accumulatedIDs := filterEbookProviderIDs(item.ProviderIDs)
if accumulatedIDs == nil {
accumulatedIDs = map[string]string{}
}
return metadata.SearchQuery{
Title: item.Title,
Title: cleanEbookSearchTitle(item.Title, item.Author),
Author: item.Author,
Year: item.Year,
ContentType: ebookContentType(),
ProviderIDs: accumulatedIDs,
internal/ebooks/enrichment_test.go
+88 -1
View File
@@ -267,7 +267,7 @@ func TestCollectEbookMetadataAccumulatesProviderErrors(t *testing.T) {
},
}
accumulator, ids, errs := collectEbookMetadata(context.Background(), enrichmentItemRow{ContentID: "c1", Title: "t"}, providers)
accumulator, ids, errs := collectEbookMetadata(context.Background(), enrichmentItemRow{ContentID: "c1", Title: "t"}, providers, nil)
if len(errs) != 2 || !errors.Is(errs[0], searchErr) || !errors.Is(errs[1], getErr) {
t.Fatalf("provider errors = %v, want both broken-provider errors", errs)
@@ -280,6 +280,63 @@ func TestCollectEbookMetadataAccumulatesProviderErrors(t *testing.T) {
}
}
type fakeProviderIDOwner struct {
ownerByID map[string]string // provider_id -> owning content id
err error
}
func (f *fakeProviderIDOwner) FindContentIDByProviderIDs(_ context.Context, ids map[string]string, _ string, exclude string) (string, error) {
if f.err != nil {
return "", f.err
}
for _, v := range ids {
if owner, ok := f.ownerByID[v]; ok && owner != exclude {
return owner, nil
}
}
return "", nil
}
func TestCollectEbookMetadataSkipsProviderIDOwnedByAnotherItem(t *testing.T) {
providers := []metadata.Provider{
&fakeEbookMetadataProvider{
slug: "bookinfo",
results: []metadata.SearchResult{{ProviderIDs: map[string]string{"bookinfo": "40817436"}}},
result: &metadata.MetadataResult{HasMetadata: true, Overview: "book one"},
},
}
owner := &fakeProviderIDOwner{ownerByID: map[string]string{"40817436": "other-book"}}
_, ids, errs := collectEbookMetadata(context.Background(), enrichmentItemRow{ContentID: "c2", Title: "t"}, providers, owner)
if len(errs) != 0 {
t.Fatalf("unexpected provider errors: %v", errs)
}
if _, ok := ids["bookinfo"]; ok {
t.Fatalf("provider id owned by another item was claimed: %v", ids)
}
}
func TestCollectEbookMetadataSurfacesOwnershipCheckError(t *testing.T) {
checkErr := errors.New("db down")
providers := []metadata.Provider{
&fakeEbookMetadataProvider{
slug: "bookinfo",
results: []metadata.SearchResult{{ProviderIDs: map[string]string{"bookinfo": "40817436"}}},
},
}
owner := &fakeProviderIDOwner{err: checkErr}
_, ids, errs := collectEbookMetadata(context.Background(), enrichmentItemRow{ContentID: "c2", Title: "t"}, providers, owner)
if len(errs) != 1 || !errors.Is(errs[0], checkErr) {
t.Fatalf("provider errors = %v, want the ownership-check error", errs)
}
if _, ok := ids["bookinfo"]; ok {
t.Fatalf("provider id claimed despite failed ownership check: %v", ids)
}
}
func TestRunBatchDoesNotRecordFailuresForSkippedItems(t *testing.T) {
items := []enrichmentItemRow{{ContentID: "skipped-1"}}
@@ -571,3 +628,33 @@ func (f *fakeEbookImageCacher) CacheImage(_ context.Context, req metadata.CacheI
Ext: ".webp",
}, nil
}
func TestCleanEbookSearchTitle(t *testing.T) {
cases := []struct {
title, author, want string
}{
{"Exit Strategy_ The Murderbot Di - Martha Wells", "Martha Wells", "Exit Strategy The Murderbot Di"},
{"LTB.067_-_Micky_Maus_Superstar", "", "LTB.067 - Micky Maus Superstar"},
{"Club Dark Lace_ Complete Dark Lace", "", "Club Dark Lace Complete Dark Lace"},
{"All of Us - A. F. Carter", "a. f. carter", "All of Us"},
// A " - <token>" that is not the trailing author must be preserved.
{"Alice - Bob and Carol", "Bob", "Alice - Bob and Carol"},
{"Plain Title", "Some Author", "Plain Title"},
{" spaced out ", "", "spaced out"},
// Series/volume markers are kept (unwrapped) so distinct volumes search
// distinctly instead of collapsing onto one provider work.
{"Just One Night (The Raven Brothers Book 4)", "", "Just One Night The Raven Brothers Book 4"},
{"Mistborn (The Mistborn Saga #1)", "", "Mistborn The Mistborn Saga #1"},
{"The Wheel of Time (Book 1)", "", "The Wheel of Time Book 1"},
{"The Wheel of Time (Book 2)", "", "The Wheel of Time Book 2"},
{"White Out [Badlands Thriller]", "", "White Out [Badlands Thriller]"},
{"Salem's Lot (2019)", "", "Salem's Lot"},
{"The Hobbit (Illustrated)", "", "The Hobbit (Illustrated)"},
{"Exit Strategy_ Murderbot Di - Martha Wells (Book 4)", "Martha Wells", "Exit Strategy Murderbot Di"},
}
for _, tc := range cases {
if got := cleanEbookSearchTitle(tc.title, tc.author); got != tc.want {
t.Errorf("cleanEbookSearchTitle(%q,%q)=%q want %q", tc.title, tc.author, got, tc.want)
}
}
}
internal/libraryingest/executor.go
+9
View File
@@ -441,6 +441,15 @@ func scopeMatchPaths(folder *models.MediaFolder, mode scopeMode, scopePath strin
if folder == nil {
return nil
}
// Audiobook/podcast/ebook/manga libraries use scanner-driven grouping where
// the scanner assigns content_ids by folder root. Running the concurrent
// match drainer while the scan writes files causes per-file content_ids to
// be created for unlinked files. Skip concurrent matching; the post-scan
// drain handles these libraries correctly.
switch strings.ToLower(strings.TrimSpace(folder.Type)) {
case "audiobook", "audiobooks", "podcast", "podcasts", "ebook", "ebooks", "manga", "comics":
return nil
}
switch mode {
case scopeModeLibrary:
return cleanRoots(folder.Paths)
internal/metadata/plugin_provider.go
+9 -1
View File
@@ -180,8 +180,16 @@ func (p *PluginProvider) Search(ctx context.Context, query SearchQuery) ([]Searc
return nil, fmt.Errorf("encode provider ids for plugin search: %w", err)
}
// The plugin search contract carries a single free-text Query. When the
// caller supplies an author hint (ebooks), fold it in so title-only matches
// that need disambiguation — or messy filename titles — can still resolve.
queryText := strings.TrimSpace(query.Title)
if author := strings.TrimSpace(query.Author); author != "" {
queryText = strings.TrimSpace(queryText + " " + author)
}
response, err := client.Search(ctx, &pluginv1.SearchMetadataRequest{
Query: query.Title,
Query: queryText,
ItemType: query.ContentType,
Year: int32(query.Year),
ProviderIds: providerIDs,
internal/metadata/types.go
+1
View File
@@ -100,6 +100,7 @@ type MatchHints struct {
// SearchQuery is passed to SearchProvider.Search().
type SearchQuery struct {
Title string
Author string // optional creator hint (e.g. ebook author) folded into the search query
Year int
ContentType string // media_items.type value
ProviderIDs map[string]string // Accumulated IDs from prior providers
internal/scanner/audiobook_scan.go
+6
View File
@@ -153,6 +153,12 @@ func (s *Scanner) audiobookFolderShouldSkip(ctx context.Context, folder *models.
if contentID == "" {
return "", false, nil
}
// All files must share the same content_id; fragmented folders need reconcile.
for _, mf := range existing[1:] {
if mf.ContentID != contentID {
return "", false, nil
}
}
items, err := s.itemRepo.GetByIDs(ctx, []string{contentID})
if err != nil {
return "", false, fmt.Errorf("get item for skip check: %w", err)
internal/scanner/ebook.go
+166 -1
View File
@@ -3,6 +3,7 @@ package scanner
import (
"archive/zip"
"bytes"
"encoding/binary"
"encoding/hex"
"encoding/xml"
"fmt"
@@ -85,7 +86,9 @@ func parseEbookFile(path string) (book parsedEbook, err error) {
book, err = parseEbookCBZ(path)
case ".pdf":
book, err = parseEbookPDF(path)
case ".mobi", ".azw", ".azw3", ".cbr":
case ".mobi", ".azw", ".azw3":
book, err = parseEbookMOBI(path)
case ".cbr":
book = parsedEbook{Format: strings.TrimPrefix(format, ".")}
default:
err = fmt.Errorf("unsupported ebook format: %s", filepath.Ext(path))
@@ -442,6 +445,168 @@ func parseEbookCBZ(path string) (parsedEbook, error) {
return book, nil
}
// maxMOBIHeaderScanSize bounds how much of a MOBI/AZW file we read. All
// metadata (PalmDOC header, MOBI header, EXTH records, full title) lives in
// record 0 at the file start, so a fixed window covers it without streaming the
// whole book.
const maxMOBIHeaderScanSize = 256 * 1024
// MOBI/AZW/AZW3 share the Palm Database (PDB) container: a PDB header, a record
// offset list, then record 0 holding the PalmDOC header (16 bytes), the MOBI
// header, and the optional EXTH metadata block. parseEbookMOBI extracts the
// title, authors, and ISBN that Calibre and most tools write into EXTH, so these
// formats no longer fall back to the filename with no author or identifier.
func parseEbookMOBI(path string) (parsedEbook, error) {
book := parsedEbook{Format: strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")}
file, err := os.Open(path)
if err != nil {
return book, err
}
defer file.Close()
header := make([]byte, maxMOBIHeaderScanSize)
n, err := io.ReadFull(file, header)
if err != nil && err != io.ErrUnexpectedEOF && err != io.EOF {
return book, err
}
header = header[:n]
// PDB record count (uint16 BE @76) and first record-info entry (@78) give
// the offset of record 0, which holds the headers.
if len(header) < 78+8 {
return book, nil
}
if binary.BigEndian.Uint16(header[76:78]) < 1 {
return book, nil
}
rec0Off := int(binary.BigEndian.Uint32(header[78:82]))
if rec0Off <= 0 || rec0Off >= len(header) {
return book, nil
}
// PDB database name (bytes 0..31, NUL-padded) is the last-resort title.
pdbName := decodeMOBIString(trimTrailingNUL(header[0:32]), 65001)
rec0 := header[rec0Off:]
if len(rec0) < 16+8 || string(rec0[16:20]) != "MOBI" {
if pdbName != "" {
book.Title = pdbName
}
return book, nil
}
mobi := rec0[16:] // skip the 16-byte PalmDOC header
mobiHeaderLen := int(binary.BigEndian.Uint32(mobi[4:8]))
var encoding uint32
if len(mobi) >= 16 {
encoding = binary.BigEndian.Uint32(mobi[12:16]) // 65001=UTF-8, else CP1252
}
// Full title: offset (relative to rec0 start) and length at MOBI+0x44/0x48.
if len(mobi) >= 0x4C {
nameOff := int(binary.BigEndian.Uint32(mobi[0x44:0x48]))
nameLen := int(binary.BigEndian.Uint32(mobi[0x48:0x4C]))
if nameOff > 0 && nameLen > 0 && nameOff+nameLen <= len(rec0) {
book.Title = decodeMOBIString(rec0[nameOff:nameOff+nameLen], encoding)
}
}
// The EXTH metadata block, when present, immediately follows the MOBI header.
// Detect it by its "EXTH" magic rather than the header flag, whose offset
// varies across MOBI versions.
if mobiHeaderLen > 0 && mobiHeaderLen+4 <= len(mobi) &&
string(mobi[mobiHeaderLen:mobiHeaderLen+4]) == "EXTH" {
parseMOBIEXTH(mobi[mobiHeaderLen:], encoding, &book)
}
if book.Title == "" {
book.Title = pdbName
}
return book, nil
}
// parseMOBIEXTH walks the EXTH record list, pulling the metadata fields the
// catalog/enricher uses. EXTH record types: 100 author, 101 publisher,
// 103 description, 104 ISBN, 503 updated title, 524 language.
func parseMOBIEXTH(data []byte, encoding uint32, book *parsedEbook) {
if len(data) < 12 || string(data[0:4]) != "EXTH" {
return
}
// data carries the rest of record 0, not just the EXTH block. Bound parsing
// to the declared EXTH length so a bad record count can't walk full-text
// bytes and assign junk metadata.
exthLen := int(binary.BigEndian.Uint32(data[4:8]))
if exthLen < 12 || exthLen > len(data) {
return
}
data = data[:exthLen]
count := int(binary.BigEndian.Uint32(data[8:12]))
pos := 12
var isbn string
for i := 0; i < count; i++ {
if pos+8 > len(data) {
break
}
recType := binary.BigEndian.Uint32(data[pos : pos+4])
recLen := int(binary.BigEndian.Uint32(data[pos+4 : pos+8]))
if recLen < 8 || pos+recLen > len(data) {
break
}
payload := data[pos+8 : pos+recLen]
pos += recLen
switch recType {
case 100: // author
book.Authors = append(book.Authors, splitEbookAuthors(decodeMOBIString(payload, encoding))...)
case 101: // publisher
if book.Publisher == "" {
book.Publisher = decodeMOBIString(payload, encoding)
}
case 103: // description
if book.Description == "" {
book.Description = cleanEbookDescription(decodeMOBIString(payload, encoding))
}
case 104: // ISBN
if isbn == "" {
isbn = decodeMOBIString(payload, encoding)
}
case 503: // updated title (overrides the MOBI full-name title)
if t := decodeMOBIString(payload, encoding); t != "" {
book.Title = t
}
case 524: // language
if book.Language == "" {
book.Language = decodeMOBIString(payload, encoding)
}
}
}
if isbn != "" {
if normalized := normalizeEbookISBN(isbn); normalized != "" {
book.ISBN = normalized
}
}
}
// decodeMOBIString decodes EXTH/header bytes using the MOBI text-encoding code
// (65001 = UTF-8, anything else defaults to CP1252, the MOBI default).
func decodeMOBIString(data []byte, encoding uint32) string {
if len(data) == 0 {
return ""
}
if encoding != 65001 {
if decoded, err := charmap.Windows1252.NewDecoder().Bytes(data); err == nil {
return strings.TrimSpace(string(decoded))
}
}
return strings.TrimSpace(string(data))
}
func trimTrailingNUL(b []byte) []byte {
if i := bytes.IndexByte(b, 0); i >= 0 {
return b[:i]
}
return b
}
// naturalPathLess orders archive entry names case-insensitively with digit
// runs compared numerically, so unpadded page numbers ("2.jpg" before
// "10.jpg") and chapter directories ("ch2/" before "ch10/") sort in reading
internal/scanner/ebook_scan.go
+89
View File
@@ -453,6 +453,21 @@ func (s *Scanner) reconcileEbookFile(ctx context.Context, folder *models.MediaFo
if parsed.Title == "" {
parsed.Title = ebookTitleFromPath(filePath)
}
if len(parsed.Authors) == 0 {
if author := ebookAuthorFromPath(filePath); author != "" {
parsed.Authors = []string{author}
// A path-derived title carries the same " - Author" suffix; drop it
// so the title, group key, and enrichment query stay clean. Compare
// normalized so case/spacing variants (e.g. "a. f. carter") match
// the recovered author and don't leave a duplicated suffix.
if idx := strings.LastIndex(parsed.Title, " - "); idx >= 0 {
suffixAuthor := strings.TrimSpace(parsed.Title[idx+len(" - "):])
if normalizeEbookIdentityPart(suffixAuthor) == normalizeEbookIdentityPart(author) {
parsed.Title = strings.TrimSpace(parsed.Title[:idx])
}
}
}
}
groupKey := ebookContentGroupKey(&parsed, filePath)
unlock := groupLocks.lock(groupKey)
@@ -885,6 +900,80 @@ func ebookTitleFromPath(filePath string) string {
return strings.TrimSuffix(base, filepath.Ext(base))
}
// ebookAuthorFromPath recovers an author for libraries that shelve books as
// ".../<Author>/<Title>/<Title> - <Author>.ext" but embed no author in the file
// (common for PDF/MOBI/AZW3). It returns a value only when two independent path
// signals agree: the grandparent directory name and the filename's trailing
// " - X" segment. Authorless layouts — magazines, language courses, flat dumps —
// satisfy neither or only one signal, so they never get a junk author that would
// poison the enrichment search.
func ebookAuthorFromPath(filePath string) string {
base := ebookTitleFromPath(filePath)
idx := strings.LastIndex(base, " - ")
if idx < 0 {
return ""
}
fromName := strings.TrimSpace(base[idx+len(" - "):])
if fromName == "" {
return ""
}
grandparent := strings.TrimSpace(filepath.Base(filepath.Dir(filepath.Dir(filePath))))
switch grandparent {
case "", ".", string(filepath.Separator):
return ""
}
if normalizeEbookIdentityPart(fromName) != normalizeEbookIdentityPart(grandparent) {
return ""
}
// Position alone cannot tell author from title: some libraries shelve
// books as "<Title>/<Author>/<Author> - <Title>" (inverted) which also
// satisfies the grandparent==suffix check and would assign the title as
// the author. Require the candidate to look like a person name; series and
// title folders ("De legenden van de Alfen") fail this and are rejected.
if !looksLikePersonName(grandparent) {
return ""
}
// Return the directory form, which carries canonical casing ("A. F. Carter"
// rather than a lowercased filename suffix).
return grandparent
}
// looksLikePersonName reports whether value is shaped like an author name. A
// "Last, First" comma form is accepted outright; otherwise every token must be
// capitalized or a known name particle (van, de, von, ...). Title/series
// strings contain lowercase content words and so are rejected.
func looksLikePersonName(value string) bool {
value = strings.TrimSpace(value)
if value == "" {
return false
}
if strings.ContainsAny(value, "0123456789") {
return false
}
if strings.Contains(value, ",") {
return true
}
particles := map[string]struct{}{
"van": {}, "von": {}, "de": {}, "der": {}, "den": {}, "het": {}, "di": {},
"da": {}, "del": {}, "della": {}, "la": {}, "le": {}, "el": {}, "du": {},
"dos": {}, "das": {}, "bin": {}, "al": {}, "ter": {}, "te": {}, "ten": {},
"op": {}, "'t": {},
}
hasUpper := false
for _, token := range strings.Fields(value) {
r := []rune(token)[0]
if unicode.IsUpper(r) {
hasUpper = true
continue
}
if _, ok := particles[strings.ToLower(token)]; ok {
continue
}
return false
}
return hasUpper
}
func normalizeEbookIdentityPart(value string) string {
value = strings.ToLower(strings.TrimSpace(value))
var b strings.Builder
internal/scanner/ebook_test.go
+131
View File
@@ -3,6 +3,7 @@ package scanner
import (
"archive/zip"
"context"
"encoding/binary"
"errors"
"os"
"path/filepath"
@@ -1894,3 +1895,133 @@ func TestParseEbookFB2RejectsOversizedFile(t *testing.T) {
t.Fatalf("parseEbookFile error = %v, want size-cap rejection", err)
}
}
// buildTestMOBI assembles a minimal but valid MOBI/AZW container: a PDB header,
// a single record-info entry pointing at record 0, and record 0 holding a
// PalmDOC header, a MOBI header, an EXTH block, and the full-title string.
func buildTestMOBI(t *testing.T) string {
t.Helper()
exthRec := func(typ uint32, val string) []byte {
b := make([]byte, 8+len(val))
binary.BigEndian.PutUint32(b[0:4], typ)
binary.BigEndian.PutUint32(b[4:8], uint32(8+len(val)))
copy(b[8:], val)
return b
}
var recs []byte
recs = append(recs, exthRec(100, "A H Lee")...) // author
recs = append(recs, exthRec(503, "The Sea: A Novel")...) // updated title
recs = append(recs, exthRec(104, "9780306406157")...) // ISBN
exth := make([]byte, 12)
copy(exth[0:4], "EXTH")
binary.BigEndian.PutUint32(exth[4:8], uint32(12+len(recs)))
binary.BigEndian.PutUint32(exth[8:12], 3)
exth = append(exth, recs...)
const mobiHeaderLen = 232
mobi := make([]byte, mobiHeaderLen)
copy(mobi[0:4], "MOBI")
binary.BigEndian.PutUint32(mobi[4:8], mobiHeaderLen)
binary.BigEndian.PutUint32(mobi[12:16], 65001) // text encoding: UTF-8
fullName := []byte("The Sea")
nameOff := 16 + mobiHeaderLen + len(exth) // relative to record 0 start
binary.BigEndian.PutUint32(mobi[0x44:0x48], uint32(nameOff))
binary.BigEndian.PutUint32(mobi[0x48:0x4C], uint32(len(fullName)))
var rec0 []byte
rec0 = append(rec0, make([]byte, 16)...) // PalmDOC header (zeroed)
rec0 = append(rec0, mobi...)
rec0 = append(rec0, exth...)
rec0 = append(rec0, fullName...)
const rec0Off = 86 // 78-byte PDB header + one 8-byte record-info entry
pdb := make([]byte, rec0Off)
copy(pdb[0:32], "The Sea") // PDB database name (last-resort title)
copy(pdb[60:64], "BOOK")
copy(pdb[64:68], "MOBI")
binary.BigEndian.PutUint16(pdb[76:78], 1) // record count
binary.BigEndian.PutUint32(pdb[78:82], rec0Off) // record 0 offset
path := filepath.Join(t.TempDir(), "book.mobi")
if err := os.WriteFile(path, append(pdb, rec0...), 0o644); err != nil {
t.Fatalf("write test mobi: %v", err)
}
return path
}
func TestParseEbookMOBIEXTH(t *testing.T) {
got, err := parseEbookFile(buildTestMOBI(t))
if err != nil {
t.Fatalf("parseEbookFile error = %v", err)
}
if got.Format != "mobi" {
t.Fatalf("Format = %q, want mobi", got.Format)
}
if got.Title != "The Sea: A Novel" {
t.Fatalf("Title = %q, want EXTH updated title", got.Title)
}
if len(got.Authors) != 1 || got.Authors[0] != "A H Lee" {
t.Fatalf("Authors = %v, want [A H Lee]", got.Authors)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want 9780306406157", got.ISBN)
}
}
func TestEbookAuthorFromPath(t *testing.T) {
cases := []struct {
name string
path string
want string
}{
{
name: "corroborated grandparent and filename suffix",
path: "/books/Books_English/Lisa Jewell/The House We Grew Up In (135563)/The House We Grew Up In - Lisa Jewell.azw3",
want: "Lisa Jewell",
},
{
name: "case and spacing differences still match",
path: "/books/Books_English/A. F. Carter/All of Us (57890)/All of Us - a. f. carter.pdf",
want: "A. F. Carter",
},
{
name: "no dash in filename",
path: "/books/Books_German/Schweizer Familie 11.04.2019.pdf",
want: "",
},
{
name: "suffix does not match grandparent dir",
path: "/books/Books_English/Stephen King/Salem's Lot (8507)/Salem's Lot - Some Other Name.pdf",
want: "",
},
{
name: "empty suffix",
path: "/books/X/Author/Title/Title - .pdf",
want: "",
},
{
name: "inverted series folder rejected (grandparent is title)",
path: "/books/Books_Dutch/De legenden van de Alfen/Heinz, Markus (2118)/Heinz, Markus - De legenden van de Alfen.epub",
want: "",
},
{
name: "comma surname form accepted",
path: "/books/Books_Dutch/Mersbergen, Jan van/De laatste ontsnapping (8702)/De laatste ontsnapping - Mersbergen, Jan van.epub",
want: "Mersbergen, Jan van",
},
{
name: "particle in name accepted",
path: "/books/Books_English/Dean R. Koontz/Midnight (7408)/Midnight - Dean R. Koontz.epub",
want: "Dean R. Koontz",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := ebookAuthorFromPath(tc.path); got != tc.want {
t.Fatalf("ebookAuthorFromPath(%q) = %q, want %q", tc.path, got, tc.want)
}
})
}
}
internal/scanner/scanner.go
+2 -1
View File
@@ -1480,7 +1480,8 @@ func (s *Scanner) syncFolderScopedAudioLibraryState(ctx context.Context, folderI
if _, err := s.fileRepo.Pool().Exec(ctx, `
INSERT INTO media_item_roots (media_folder_id, canonical_root_path, content_id)
SELECT DISTINCT mf.media_folder_id, mf.canonical_root_path, mf.content_id
SELECT DISTINCT ON (mf.media_folder_id, mf.canonical_root_path)
mf.media_folder_id, mf.canonical_root_path, mf.content_id
FROM media_files mf
JOIN media_items mi ON mi.content_id = mf.content_id
WHERE mf.media_folder_id = $1
migrations/sql/20260702060558_media_items_content_type_covering_index.sql
+15
View File
@@ -0,0 +1,15 @@
-- +goose NO TRANSACTION
-- +goose Up
-- Covering index for the ABS audiobook count/list path. The unfiltered
-- /api/libraries/{id}/items COUNT(*) seeks content_ids from the media_folder
-- index, then probed media_items by PK just to verify type='audiobook' — one
-- heap fetch per row (~255K probes, ~1M buffer hits, ~500ms). Adding type to
-- the content_id index lets that probe run index-only, cutting the count to
-- ~100ms. Also speeds the data query's per-row type check.
-- CONCURRENTLY avoids locking media_items writes during the build; it cannot
-- run inside a transaction, hence the NO TRANSACTION annotation above.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_media_items_content_type
ON public.media_items USING btree (content_id, type);
-- +goose Down
DROP INDEX CONCURRENTLY IF EXISTS public.idx_media_items_content_type;
migrations/sql/20260702060600_abs_audiobook_author_counts_mv.sql
+39
View File
@@ -0,0 +1,39 @@
-- +goose Up
-- +goose StatementBegin
-- Precomputed audiobook author list for the ABS-compat /libraries/{id}/authors
-- endpoint. The live query GROUP BYs all authors of a 100K+ library on every
-- page (~800ms) plus a COUNT(DISTINCT) (~500ms); paging the full author list
-- for an iOS sync then blows past the background-task window. This MV turns the
-- list into an indexed paginated read. added_at = people.created_at (= real ABS
-- author.createdAt) so clients can sort newest-first and run incremental syncs.
-- Refreshed periodically by the audiobooks service (REFRESH ... CONCURRENTLY).
CREATE MATERIALIZED VIEW IF NOT EXISTS abs_audiobook_author_counts AS
SELECT
p.id AS person_id,
p.name AS name,
mil.media_folder_id AS library_id,
COUNT(DISTINCT mi.content_id) AS num_books,
p.created_at AS added_at
FROM media_items mi
JOIN media_item_libraries mil ON mil.content_id = mi.content_id
JOIN item_people ip ON ip.content_id = mi.content_id AND ip.kind = 7
JOIN people p ON p.id = ip.person_id
WHERE mi.type = 'audiobook'
GROUP BY p.id, p.name, mil.media_folder_id, p.created_at
WITH DATA;
-- Unique index is required for REFRESH MATERIALIZED VIEW CONCURRENTLY.
CREATE UNIQUE INDEX IF NOT EXISTS idx_abs_author_counts_pk
ON abs_audiobook_author_counts (library_id, person_id);
-- Name sort (default ABS author ordering).
CREATE INDEX IF NOT EXISTS idx_abs_author_counts_name
ON abs_audiobook_author_counts (library_id, LOWER(name));
-- addedAt-desc sort for client incremental syncs.
CREATE INDEX IF NOT EXISTS idx_abs_author_counts_added
ON abs_audiobook_author_counts (library_id, added_at DESC, person_id);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP MATERIALIZED VIEW IF EXISTS abs_audiobook_author_counts;
-- +goose StatementEnd