7 Commits

Include renames

Author	SHA1	Message	Date
3 people	383973ec22	feat(metadata): improve match accuracy and localized titles (#461) ... * feat(metadata): improve match accuracy and localized titles * fix(metadata): address matching review findings * test(catalog): align empty alias snapshot scope --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>	2026-07-24 12:02:52 -04:00
3 people	e140bd9424	feat(metadata,scanner): trailers and extras for movies and series (#322) ... * feat(metadata,scanner): trailers and extras for movies and series Remote provider videos (TMDB trailers/teasers/featurettes/...) are fetched through the unified match/refresh pipeline into the new item_videos table, filtered per-library via media_folders.trailer_kinds, merged across providers with site/provider dedup, and lockable via FieldVideos. The movie scanner stops discarding supplemental directories (Trailers/, Featurettes/, Behind The Scenes/, ...) and classifies them — plus Jellyfin-style filename suffixes (-trailer, -behindthescenes, ...) and series-root supplemental dirs — into the new media_extras entity backed by ordinary media_files rows (extra_id ownership, content_id/episode_id NULL so existing version/matching queries stay structurally blind to extras). Series Extras/SxxExx season-0 mapping is unchanged. Extras are playable watch targets via a GetWatchDetail fallback tier (episodes precedent), with contentid.ForLocal minting stable ids. API: ItemDetail gains additive videos/extras arrays (single + batch parity); library settings expose trailer_kinds. jellycompat now populates RemoteTrailers, LocalTrailerCount/SpecialFeatureCount, and serves real /LocalTrailers + /SpecialFeatures items playable through PlaybackInfo. Requires silo-plugin-sdk v0.9.0 (VideoRecord) before go.mod can bump; builds locally via go.work against the SDK feat/metadata-videos branch. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): trailers and extras sections, library trailer-kinds setting TrailersSection (YouTube thumbnails + youtube-nocookie modal) and ExtrasSection (plays extras through the standard watch controller) on movie and series detail pages; admin library form gains a trailer-kinds allow-list synced with the server default (all provider kinds), now also honored on library create. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): scan extra_id in scanMediaFiles; review cleanups scanMediaFiles (the plural row scanner behind GetByContentID/GetByFolder/ GetByExtraID and 20+ other queries) was missing the scan destination for the new extra_id column, which would have failed every media-file read at runtime with a column/destination count mismatch. Also: extend the batch equivalence test to seed item_videos/media_extras so the new videos/extras prefetch wiring is actually proven; drop the one-off pgxRows interface for the repo-wide pgx.Rows convention; reuse formatClock instead of a third duration formatter in ExtrasSection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump silo-plugin-sdk to v0.9.0 for VideoRecord Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(matching): exclude extras files from match queues and bulk content linking Dev verification caught extras media_files rows (content_id NULL by design) being swept into the movie/series match queues and the root-claim bulk relink: a '-featurette' suffix extra was matched onto its parent as a version, and a Trailers/ file minted a spurious local skeleton item that shadowed the extra's watch target. Add 'extra_id IS NULL' to the queue eligibility conditions, root/group claim relinks, observed-root content assignment, and the admin unmatched-files listing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): authorize local extras files through their parent item Dev verification: playback/start (and the shared MediaFileAuthorizer used by markers/subtitles/ebook reader) resolved file ownership only via episode_id/content_id, so extras files (extra_id only) 404ed. Add an ExtraLookup tier that resolves media_extras and gates on the parent item's access, mirroring the episode->series pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): resolve local extras through GetItemDetail for compat playback jellycompat PlaybackInfo (and any per-item consumer resolving arbitrary content ids) goes through GetItemDetail, which lacked the extras tier that GetWatchDetail has — so Jellyfin clients got zero MediaSources for extras. Add buildExtraItemDetail (minimal detail + ordinary playback surface, parent-gated access) as the fourth resolution tier, and map the extra type to Jellyfin's Video kind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): allow youtube-nocookie embeds in CSP; trailer modal a11y The frontend CSP's frame-src blocked the trailer modal's youtube-nocookie.com iframe (found on dev verification). Also add the missing sr-only DialogDescription and drop the redundant allowFullScreen attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address PR review findings for trailers/extras - Extras watch/item detail no longer stamp SeriesID/SeriesTitle for movie-owned extras (players key episodic post-roll flows off series_id); series-owned extras keep them (Codex). - processExtraFiles resolves the parent and upserts media_extras before the unchanged fast-path, and the fast-path now also compares mtime, so rematched parents / reclassified kinds / same-size replacements converge (Codex + CodeRabbit). - media_files upsert clears content/episode linkage atomically when extra_id is set (ownership mutual exclusion in one statement); the now-redundant MarkFileAsExtra helper is removed (CodeRabbit). - ScanFile's extras branch runs syncPresentLibraryState + reconcileLibraryMemberships so converting a primary file to an extra cleans stale library membership immediately (CodeRabbit). - media_extras migration adds the media_files FK as NOT VALID + VALIDATE to avoid a full-scan exclusive lock on large tables (CodeRabbit). - trailer_kinds input is trimmed/lowercased/deduped and unknown values are dropped instead of silently widening the allow-list to 'other' (CodeRabbit). - Extras authorization branches match the episode branch's posture: unconfigured lookup is a config error, nil extra is a 404 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 18:52:44 -04:00
3 people	97ac2b4eed	feat(audiobooks): audiobookshelf support — ABS conformance, perf, ebooks (#289) ... * fix(ebooks): fold author hint into metadata search query The ebook enricher loaded each item's author but buildEbookSearchQuery dropped it, and metadata.SearchQuery had no field to carry it — so the plugin only ever received the title. Title-only searches collide or miss, leaving items without metadata or a cover. Add SearchQuery.Author and fold it into the plugin search query text (the SearchMetadataRequest contract carries a single free-text Query, so no proto change is needed). Gated to callers that set Author (ebooks); movie/TV search is unchanged. Verified live against OpenLibrary/GoogleBooks: improves disambiguation on clean titles. Note: messy filename-derived titles (series prefixes, trailing "(… Book N)") still need title normalization, and a large tail of niche/self-published ebooks is simply absent from the free sources — neither is addressed here. AI-use disclosure: authored with Claude Code. (cherry picked from commit ba1265909c4fb87e1a8eab64b0b0c183aa95acc1) * feat(scanner): extract MOBI/AZW/AZW3 metadata from EXTH headers These formats previously had no parser — parseEbookFile returned only the format string, so title fell back to the filename with no author and no ISBN, leaving ~21k books unmatchable by the metadata enricher. Parse the Palm Database container (PDB header → record 0 → PalmDOC + MOBI header → EXTH block) and extract title, authors, ISBN, publisher, and language. EXTH is located by its magic rather than the header flag, and field offsets (encoding @12, full-name @0x44/0x48) were verified against real .mobi/.azw3 files. Verified live against real library files: azw3 → title "The Sea", author "A H Lee" mobi → title "Brotherband 3: The Hunters", author "John Flanagan", ISBN 9781742750637 AI-use disclosure: authored with Claude Code. (cherry picked from commit 7af194b711de97bc79855f08a9a4f9732c49db74) * fix(ebooks): recover author from path and clean provider search title - ebookAuthorFromPath: recover an author for ".../<Author>/<Title>/<Title> - <Author>.ext" layouts when the file embeds none, gated on two agreeing path signals (grandparent dir == filename suffix) so magazines/courses never get a junk author; strip the suffix from a path-derived title. - cleanEbookSearchTitle: normalize filesystem-mangled titles before search (underscore->space, drop trailing " - <author>") to lift hit rate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 36a16cb58c3e5276aa4c0bdf8577008070f6abea) * fix(scanner): gate path-author on person-name shape ebookAuthorFromPath's grandparent==suffix corroboration also matched inverted layouts ("<Title>/<Author>/<Author> - <Title>"), assigning the title as the author. Require the candidate directory to look like a person name (comma form, or all-capitalized tokens plus name particles) so series and title folders ("De legenden van de Alfen") are rejected, and return the canonical directory form for proper casing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit ff720bd268a23bff0e94c70f15cb7ecfb8efcb1f) * fix(ebooks): strip series/book-number parentheticals from search title cleanEbookSearchTitle now peels trailing "(... Book N)", "[#3]", "(2019)" groups that don't belong in a provider title query, while leaving meaningful parentheticals ("(Illustrated)") intact. Enrichment-side only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 2273636c04fd9ef483003a9558972a2104fdd3a6) * fix(ebooks): keep volume number in search title and dedup provider IDs Two distinct ebooks (e.g. series volumes named only by series + book number) were collapsing onto a single provider work, then fighting over the same media_item_provider_ids row: - cleanEbookSearchTitle stripped trailing "(... Book N)" / "[#3]" groups entirely, so every volume of a series searched as the bare series name and matched the same provider work. The plugin search contract carries only a single free-text Query, so the volume number is now UNWRAPPED into the query (brackets dropped, words kept) instead of discarded, giving distinct volumes distinct searches. Bare-year groups are still dropped (SearchQuery.Year carries them); meaningful parentheticals ("(Illustrated)") still survive. - collectEbookMetadata now consults FindContentIDByProviderIDs before accumulating a search-result provider ID. An ID already owned by a different content item is skipped, so the loser is not mis-tagged with the winner's metadata and ReplaceByContentID no longer violates the (provider, provider_id, item_type) unique constraint. The previous behavior logged duplicate-key errors every sweep and re-enriched the failing item forever (CPU/RAM churn). A failed ownership check is surfaced as a provider error so the item retries rather than terminally stamping as "no match". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 942fdef6cb0e167b2d9223e9a968b3010b7b3ec8) * fix(ebooks): address CodeRabbit review on PR #185 - cleanEbookSearchTitle: anchor author-suffix strip to a trailing match (optionally followed by a series/volume parenthetical) so a mid-title " - <token>" no longer truncates valid title text - ebook scan: strip the recovered author suffix using normalized comparison so case/spacing variants (e.g. "a. f. carter") don't leave a duplicate - parseMOBIEXTH: bound parsing to the declared EXTH length so a corrupt record count can't read full-text bytes as junk metadata - add regression test for a non-trailing " - <token>" in the title Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 0f45af8143e04dfd5b4a5ee3e949dcd943eedbd1) * fix(audiobooks): pass author in search query and retry on provider errors Set SearchQuery.Author so the host adapter folds author into the plugin free-text query (parity with ebooks). Track provider errors during enrichment; when nothing matched and a provider errored, return an error without stamping last_refreshed so the sweep retries instead of terminally burning the item on a transient failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit f45dd2104c5d6415324e80f91f6684bc39858459) * fix(scanner): consolidate fragmented multi-file audiobook content_ids on rescan audiobookFolderShouldSkip used ListByObservedRootPath which returns all files for a root path regardless of content_id. When a multi-file audiobook had files fragmented across multiple content_ids (e.g. from concurrent refreshes), the file count matched disk so the skip check returned true and the reconcile never ran to merge them. Now verifies all DB files share the same content_id before skipping; any fragmentation forces a full reconcile which consolidates to one content_id via FindContentIDByRootPath → upsertAudiobookMediaFiles. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit e91c33e88a7d09e802e6afd8af246c6c954d0498) * fix(ingest): skip concurrent match drainer for audiobook/podcast/ebook/manga libraries The concurrent scoped match drainer ran during scan for all library types. For audiobook libraries, the scanner assigns content_ids by folder root (one item per multi-file folder). Running the drainer concurrently caused it to process files with content_id=NULL (cleared by complete refresh) as individual items, creating one media_item per file instead of one per folder. This manifested as 41-file audiobooks fragmenting into dozens of orphaned single-file content_ids on every refresh. These library types use scanner-driven grouping; the post-scan drain step handles them correctly. Returning nil matchScopes skips the concurrent drainer entirely for these types. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit 93ae9d22ce315874fa22a958b88ca1766075695f) * fix(abs): match real audiobookshelf auth + session-sync contract Align the ABS-compat auth flow with real audiobookshelf (v2.26+) so third-party clients (yaabsa, Plappa, native iOS) authenticate and sync playback correctly: - login/refresh: always emit user.accessToken; x-return-tokens gates only the refresh token (body vs HttpOnly refresh_token cookie) - /auth/refresh returns the full login envelope (was a thin token map) - /me returns the full user object (toOldJSONForBrowser), shared with login/authorize via a single absUserObject() builder - /logout returns 200 {redirect_url:null} and clears the cookie (was 204) - add POST /session/{sid}/sync (real ABS heartbeat path); it was PATCH-only, so the official client's sync POST 404'd and playback progress never synced Verified against advplyr/audiobookshelf server/{Auth.js,models/User.js, controllers,routers}. Unit tests updated/added; full abs suite green. Not yet live-verified. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 336e932471d4be021d82299106a120611783836a) * fix(abs): conform browse/list items to real audiobookshelf minified shape Strict ABS clients (yaabsa, Plappa) crash or drop items when the browse list shape only approximates real audiobookshelf. Match the serializers: - add media.id + media.libraryItemId (= ContentID) to LibraryItemMedia; yaabsa BookMedia.id is required non-null and was missing → the whole item failed to parse ("Null is not a subtype of String") - rebuild the minified list shape to LibraryItem.toOldJSONMinified + Book.toOldJSONMinified + oldMetadataToJSONMinified key-for-key (ino, path, isFile, numFiles/size, media.{id,tags,numTracks,numAudioFiles, numChapters,size,ebookFormat}, flat author/series metadata) - force media.numTracks/numAudioFiles >= 1 in the browse projection so Plappa doesn't drop items reporting 0 audio files - default /items list to minified (real ABS list is always minified); minified=0 opts into the full shape Verified against advplyr/audiobookshelf models/{Book,LibraryItem}.js. Adds minified_test.go key-set conformance guards; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 6c9387a8c4b60be3dbe541049ed8c06344b10717) * fix(abs): conform /items/{id} detail to real audiobookshelf expanded shape Match real audiobookshelf LibraryItem.toOldJSONExpanded + Book.toOldJSONExpanded + oldMetadataToJSONExpanded so strict clients decode the item-detail page with the same model they use elsewhere: - add expanded outer keys to LibraryItem (oldLibraryItemId, lastScan, scanVersion, libraryFiles, size) and populate libraryFiles + summed size from the item's media files in the detail builder - add media.size (Book.toOldJSONExpanded) - make the typed Metadata the full expanded superset: subtitle, titleIgnorePrefix, authorName, authorNameLF, narratorName, seriesName, descriptionPlain, publishedDate, asin, language, abridged; drop the omitempty that previously dropped description/publishedYear/isbn/ publisher when empty (a missing key crashes strict clients) Verified against advplyr/audiobookshelf models/Book.js + LibraryItem.js. Adds items_detail_test.go expanded key-set guard; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 8bd485f0291e9db9f32e13a68609a68ee8a945ec) * fix(abs): conform authors/series endpoints to real audiobookshelf shapes Match the real audiobookshelf serializers so strict clients decode the authors/series browse + detail responses: - GET /libraries/{id}/authors now branches like LibraryController.getAuthors: bare { authors: [...] } when not paginated, paged { results, total, ... } only when limit+page are present (was always paged → clients keying on `authors` got keyNotFound) - author objects carry the full Author.toOldJSON key set (id, asin, name, description, imagePath, libraryId, addedAt, updatedAt, numBooks); silo has no analog for asin/description/imagePath/timestamps so they are null/0 - series objects carry the full Series.toOldJSON key set (adds nameIgnorePrefix, description, libraryId, addedAt, updatedAt) - series/author books are now FULL minified library items (real ABS shape) instead of thin {id,media:{metadata:{title}}} stubs that crash Plappa; author items moved to the real-ABS `libraryItems` key Verified against advplyr/audiobookshelf controllers/LibraryController.js and models/{Author,Series}.js. Tests updated + envelope-branch guard added; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 8a22eb0900ed881d500ded315a508e1a07da14f3) * fix(abs): add libraryId to collection/playlist objects (real ABS shape) Real audiobookshelf Collection.toOldJSON and Playlist.toOldJSON both carry a libraryId; silo's emitters omitted it, so a strict client modeling the object with a required libraryId crashed. silo collections/playlists are cross-library user-personal, so emit the virtual audiobook library id. The books[]/items[] entries already carry the full LibraryItem shape and inherit the browse-conformance fixes (media.id etc.). Envelopes were already correct (paged for library-scoped, {collections}/{playlists} for global). Verified against advplyr/audiobookshelf models/{Collection,Playlist}.js. Envelope key-set tests updated; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit f7d2ff0565f05c3a6ef7f36f2b1f252bd373fa7a) * fix(abs): conform library object + /libraries/{id} to real audiobookshelf The library object was only {id,name,mediaType}; real audiobookshelf Library.toOldJSON has 12 keys, so a strict client decoding the library model crashed on the missing ones. Also GET /libraries/{id} always wrapped the object in { library: ... }, but real ABS returns it directly unless ?include=filterdata is requested. - audiobookLibraryMap now emits the full Library.toOldJSON shape (folders[] as LibraryFolder.toOldJSON, displayOrder, icon, provider, settings, lastScan, lastScanVersion, createdAt, lastUpdate). This also enriches the libraries[] on the login envelope, which shares the builder. - handleLibraryDetail returns the library object DIRECTLY without include, and wraps in { filterdata, issues, numUserPlaylists, customMetadataProviders, library } (adds the missing customMetadataProviders) with include=filterdata. GET /libraries already returned { libraries: [...] } (correct). Verified against advplyr/audiobookshelf models/Library.js + controllers/LibraryController.js. Adds libraries_shape_test.go; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit d05a2f2af1caf21a4ad04577a4787ba02cff091c) * fix(abs): conform personalized recent-series shelf to real ABS series shape The /libraries/{id}/personalized "Recent Series" shelf emitted thin {id,name,numBooks,libraryId,books:[]} entities with an always-empty cover stack. Emit the full real-ABS series object (seriesObjectABS, adds nameIgnorePrefix/description/addedAt/updatedAt) with minified book items (seriesBookMinified) — the same shape as /libraries/{id}/series so the shelf card decodes identically and shows real covers. Book shelves already used full minified items; the shelves array is a bare array (matches real ABS getUserPersonalizedShelves). abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 7c586f8c923cbc481f6f94e304af48dee379a999) * fix(abs): conform listening-sessions to real audiobookshelf PlaybackSession shape silo's /me/listening-sessions returned a thin 5-field session object (id, libraryItemId, userId, timeListening, currentTime) wrapped in the generic pagedEnvelope shape ({results,sortBy,filterBy,minified}). Real audiobookshelf clients (Flutter/Swift strict decoders) expect the MeController.getListeningSessions envelope ({total,numPages,page,itemsPerPage,sessions}) and each session to carry the full PlaybackSession.toJSON() key set, so the missing keys (notably mediaType, mediaMetadata, displayTitle, displayAuthor, coverPath, duration, chapters, deviceInfo, playMethod, mediaPlayer, serverVersion, date, dayOfWeek, startTime, startedAt, updatedAt, libraryId, bookId, episodeId) crashed with keyNotFound errors. Both handleListeningSessions and handleListeningSessionDetail now build the response via a shared sessionToABS() that reuses buildSiloPlayMediaMetadata (already used by /play) to hydrate mediaMetadata/displayTitle/displayAuthor from MediaStore, batching lookups via GetAudiobooksByIDs for the list endpoint. Lookups are best-effort: a missing/inaccessible item falls back to a stub MediaItem so every key is still emitted, never a crash. Verified against advplyr/audiobookshelf server/controllers/MeController.js (getListeningSessions) and server/objects/PlaybackSession.js (toJSON()) on GitHub master. Known placeholders (real ABS fields we can't populate without extra cost): chapters (empty array — would require a per-session media-files fetch), duration (0 — total book duration isn't tracked on the session row), startTime (0 — not persisted separately from currentTime), deviceInfo (static "unknown" device, matching the /play endpoint's existing placeholder — no device info is persisted per session). AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 9471497c99b96c8d3defc6c8112c913f7c55924b) * feat(abs): add offline session sync endpoints (/session/local, /session/local-all) The official ABS mobile app records playback while offline and POSTs those PlaybackSession objects back on reconnect via SessionController.syncLocal and syncLocalSessions. silo was missing both endpoints, so offline listening progress was silently lost. Add them to the bearerAuth-protected session group (both /abs/api and /api prefixes) alongside /session/{sid}/sync and /close. POST /session/local decodes one PlaybackSession and updates the caller's resume position via ProgressStore.UpdateProgressPosition (the same call handleSessionSync uses), emitting user_item_progress_updated. POST /session/local-all decodes {sessions:[...]} and loops each robustly — a malformed or unknown item marks that one result failed without sinking the batch — returning {results:[...]}. No new store persistence or migration; podcast/episode sessions are accepted as no-ops. Verified against advplyr/audiobookshelf server/controllers/SessionController.js and server/managers/PlaybackSessionManager.js. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 008a4df948d855a4bfe62b24f89bfc484f088033) * fix(abs): conform library search + items-in-progress to real audiobookshelf Real ABS's libraryItemsBookFilters.search() (delegated from LibraryController.search) returns { book, narrators, tags, genres, series, authors } with no "podcast" key for a book library, and each book entry is only { libraryItem } — no matchKey/matchText, which our handler was inventing. Search now matches those keys, drops the fabricated matchKey/matchText fields, and best-effort populates authors/series buckets via client-side substring filtering over the existing aggregate listers (narrators/tags/genres stay empty-but-present since silo has no backing aggregation query for them yet). MeController.getAllLibraryItemsInProgress wraps items as { ...libraryItem.toOldJSONMinified(), progressLastUpdate }; our handler was emitting a hand-rolled subset of fields plus a nested userMediaProgress object that doesn't exist in the real response. items-in-progress now reuses the existing Minify() projection and merges a flat progressLastUpdate (ms) field to match. Verified against advplyr/audiobookshelf controllers/{Library,Me}Controller.js and server/utils/queries/{libraryItemsBookFilters,authorFilters}.js. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 998ff55f3cf27d504f0e5aec8c7c29fa57f10247) * fix(abs): /ping returns success:true and /status carries authMethods The ABS apps validate a server address by reading response.success from GET /ping; silo returned {pong:true,...} with no `success`, so the app reported "unable to reach" even though the server responded 200. Also /status was missing authMethods/authFormData, which the app reads to render the login form. - /ping now includes {"success": true} (pong/server/version kept as extras) - /status now returns {app,serverVersion,isInit,language,authMethods, authFormData} matching real audiobookshelf Server.js Verified against advplyr/audiobookshelf server/Server.js. Adds ping_status_test.go; abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit df732d09355303608bc4d5fc2138555e37497a02) * fix(abs): mount login + auth/refresh under /api prefix Clients that post to /api/login (and /api/auth/refresh) got a 404 because login/refresh were only mounted at root and /abs/api — while the rest of the authenticated ABS surface (/api/me, /api/authorize, /api/libraries, covers) is served under both /api and /abs/api. The 404 surfaced in the client as a generic "unknown error occurred" on sign-in. Mount /login and /auth/refresh under all three prefixes ("", /api, /abs/api), matching the authenticated groups. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 18071e180b02131cda354303eadd1ff3a0708065) * fix(abs): accept form-encoded login bodies (not just JSON) Real audiobookshelf (express body-parser + passport local) accepts both application/json and application/x-www-form-urlencoded credential bodies. Silo only json-decoded the body, so a form-encoded client got 400 "invalid request body" — surfaced in the app as a generic "unknown error" on sign-in (confirmed live: JSON creds -> 200, identical form-encoded creds -> 400). Buffer the body once, try JSON, then fall back to url.ParseQuery for the form-encoded case. Adds login_body_test.go (form + JSON both reach the validator). abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 408dc33debb7a7ee363778094511ccfdd1ee70d2) * fix(abs): emit full real-ABS serverSettings (OpenID/auth fields) silo's login/authorize serverSettings omitted the auth + OpenID fields that real audiobookshelf ServerSettings.toJSONForBrowser includes (authLoginCustomMessage, authOpenID*, rateLimitLogin*, backupPath, allowedOrigins). OIDC-aware strict clients (Prologue, iOS/Swift) decode serverSettings into a model that requires those keys, so their absence throws keyNotFound and the ENTIRE login response fails to decode — the client stays on the login screen with a generic "unknown error" even though the server returned 200. Simpler clients that don't model OpenID were unaffected. Emit real ABS's OIDC-disabled defaults; authActiveAuthMethods still advertises only "local" so no client initiates the OpenID flow. Diagnosed from a packet capture (Prologue posts /login? with X-Return-Tokens and gets a 200 it can't decode) + real ABS ServerSettings.js. Verified against advplyr/audiobookshelf. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 283826c952824e97233e46e057f37385ddc3054b) * fix(abs): GET /me returns the real display username, not the userID /me built its user object from the token claims and passed the numeric userID as the username, so clients saw "98" instead of "puksthepirate". Login gets the display name from the credential validator, but /me only has the token, so it needs a lookup. Add an optional UsernameResolver to the abs Dependencies; wire it from the concrete SiloCredValidator (which holds the pgx pool) via a new ResolveUsername method that mirrors Validate's display-name logic — the profile name when a profile is set and named, else the account username. handleMe uses it and falls back to the userID when unresolved. abs package compiles + tests pass; the audiobooks package (service.go, cred_validator.go) could not be linked locally (pre-existing bimg/libvips pkg-config gap) and is validated at the Docker build. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 39ff3e3350aad087757f7fb0e94d5a7f10c08ae5) * fix(abs): always emit AudioTrack keys + correct media.duration Two item-detail issues that made Prologue report "Unable to load book contents" (can't press Start Listening): - AudioTrack used omitempty on chapters/metaTags/format/bitRate/codec/ metadata/etc, so empty values dropped those keys. Real ABS AudioFile/ AudioTrack always emit them; strict clients (Prologue, yaabsa) decode tracks into a required-field model and throw keyNotFound on the missing keys, failing the whole track decode. Removed omitempty and emit chapters/metaTags as [] / {} (non-nil) in both track builders. - media.duration used the item's Runtime, which is often stale/mis-scanned (e.g. 222s for a 3.7h book) and desyncs the player scrubber. Now sum the track durations (real ABS: sum of audio file durations), falling back to Runtime only when there are no tracks. Verified against advplyr/audiobookshelf models/Book.js (AudioFile/AudioTrack) via a live packet capture of Prologue's item-detail decode failure. abs suite green. AI-use: implemented with Claude Code (Opus 4.8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 8210ed2fc63e782f158b4168ef693e671ae19638) * perf(abs): push down library browse filters + author counts MV The ABS audiobook library-serving path was slow on large libraries (~255k items): /libraries/{id}/items?filter=authors.{id} loaded and hydrated the whole library into Go before filtering (~4.8s each), and /libraries/{id}/authors ran a full GroupAggregate + COUNT(DISTINCT) per page (~53s full sync) — slow enough to trip ABS client sync timeouts (e.g. Prologue). - Push author/series/narrator/no-series filters into indexed SQL EXISTS predicates in ListAudiobooks; paginate + COUNT in SQL. Semantically equivalent to the prior Go-side filter (kind=7 author, kind=8 narrator, exact-case match, no-series sentinel). - Add covering index media_items(content_id, type) so the count/list type check runs index-only (CONCURRENTLY, NO TRANSACTION — no write-lock on the live table). - Serve /authors from a materialized view (abs_audiobook_author_counts) refreshed every 15min, with a live-query fallback when the view is empty/unrefreshed so the endpoint never blanks on a fresh deploy. Conformance preserved: keeps authorObjectABS/seriesObjectABS shapes and the limit&&page envelope decision; adds a regression test for the bare {authors:[...]} envelope on limit-only requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 0d55754051dd3ad016b3cec6a0921307071d3219) * perf(abs): index-back audiobook search via trigram GIN SearchAudiobooks matched the raw media_items.title with ILIKE '%q%' OR'd with an author/narrator EXISTS. The un-indexed raw-title column plus the OR forced a full seq scan of the ~255k-item library on every search (~560ms on library 18). Reshape into a UNION of two index-driven arms that reuse the search infrastructure the rest of the catalog already relies on: the title arm matches media_items.title_normalized (idx_media_items_title_normalized_trgm) via the shared normalize_search_text(), the people arm matches people.name (idx_people_name_trgm). GROUP BY content_id keeps the best rank when an item matches both; a normalize_search_text($2) <> '' guard stops a punctuation-only query from degenerating into ILIKE '%%'. No new index or migration — the trigram indexes already existed and were simply unused. ~560ms -> ~35ms, both indexes engaged, no seq scan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 98bbd1712719ecd03e4db87f840774cec788f177) * perf(abs): index-ordered item paging + cached library count The unfiltered /libraries/{id}/items path that ABS clients page through to sync a library recomputed COUNT(*) over the whole library on every page (~150ms each) and ordered by LOWER(sort_title), LOWER(title) — an expression matching no index, forcing a full in-memory sort of all ~255k rows per page (~324ms shallow, ~543ms deep). A full sync is thousands of pages, so both costs dominated indexing time. - Order by lower(coalesce(nullif(btrim(sort_title),''), title)), content_id so the page is served by an ordered index scan on the existing idx_media_items_sort_key (~324ms -> ~1ms). content_id (PK) is a stable tiebreaker, making sequential pagination deterministic — the prior ordering could skip/repeat rows when sort keys collided. - Memoize the per-page COUNT in a 60s TTL cache keyed on the fully rendered count SQL + bound args, so it covers every input the WHERE depends on (library, pushed-down filter, all access predicates) and can't drift as access logic evolves. Expired entries swept on write. No new index or migration — reuses idx_media_items_sort_key. total may lag up to 60s during an active scan; clients re-sync. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 32e26c2f99a1ffc071f600071c2ea7ddcd3397b4) * fix(abs): address PR review — access-aware authors, offline progress create, cookie refresh, body limits - media_store: ListLibraryAuthors bypassed per-item access when reading the author materialized view (keyed by library only), leaking authors of books hidden by a content-rating cap or excluded media types. Take the access-aware live path whenever the filter carries an item-level predicate. - session_local: offline sync used UPDATE-only UpdateProgressPosition, so a book listened to entirely offline (no progress row yet) had its position silently dropped while still reporting progressSynced. Create the row via UpsertProgress when none exists; keep the monotonic update path for existing rows. - login: handleRefresh never read the refresh_token cookie, so cookie-flow ABS clients got 400 refreshToken required once the access token expired. Read the cookie as a third source after header and body. - session_local: cap /session/local and /session/local-all request bodies at 1 MiB via io.LimitReader, matching the rest of the package. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-05 00:16:34 -04:00
3 people	14ffc91dfb	[codex] Expand provider image cache queue (#176) ... * feat(metadata): expand provider image cache queue * fix(metadata): harden provider image cache queue Addresses bug-review feedback from Codex/CodeRabbit on the metadata image cache pipeline. All findings validated against the code before fixing; false positives (rows/connection deadlock, PhotoSourcePath merge coupling) were confirmed non-issues and left unchanged. - Honor metadata.cache_images for the background processor. The cache_metadata_images task was registered whenever S3 was configured, so merely enabling object storage downloaded the entire provider-artwork catalog even with caching disabled. Add ImageCacheProcessor.SetEnabled, gate RunOnce/RunUntilIdle on it, and wire it (with hot reload) from cfg.Metadata.CacheImages in main.go. - Guard terminal job updates with lease ownership. EnqueueBatch can repurpose a running row with a new source; MarkSucceeded/MarkFailed keyed on id alone let a stale worker finalize the replacement job and drop the new artwork. Thread locked_by through and add status='running' AND locked_by=$n guards. - Avoid uploading stale jobs onto the live artwork key. Verify the target still references the job's source (CurrentTargetSourcePath) before CacheImage, so a job whose source an admin/refresh already replaced cannot overwrite the deterministic storage object. - COALESCE nullable external IDs in EnqueueExistingProviderArtwork. A NULL tmdb_id/tvdb_id/imdb_id on any candidate failed the scan and aborted the whole cache run; matches the existing item_repo pattern. - Stop re-downloading the catalog every 30 days. Discovery now skips targets whose *_path is already a cached relative path, making the cached row the durable dedup marker instead of the prunable job row. - Decouple catalog sweeps from queue draining. RunOnce no longer runs discovery per batch; RunUntilIdle sweeps only when the queue drains and throttles full sweeps to every 15m, so idle installs stop full-scanning every entity table each minute. - Requeue claimed-but-unstarted jobs on cancellation. Acquire the semaphore before spawning workers and RequeueClaimed any jobs not yet started, instead of leaving them locked until the 15m lease expires. - Skip the backoff sleep after the final upload attempt in putObjectWithRetry (saves ~1.5s on permanent failures). - Add the s3/file/local/upload/generated exclusion to the seasons and episodes backfill in migration 20260617184537 for consistency with the later migration (the bad backfill was inert downstream, but the asymmetry is removed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-18 10:07:58 -04:00
4 people	c4cbcddeae	feat(manga): manga library type — series grouping, reading loop, AniList/MangaDex metadata + status badge (#138) ... * docs: design spec for manga library type (host sub-project) Forks the ebooks library type into a 'manga' type: series detected from the folder tree as a first-class type='manga' item, .cbz/.cbr chapters stay readable ebook items linked via a new manga_chapters table, browse shows series cards, enrichment targets the series item at content level 'manga'. Hands off to a follow-on plugin spec for the manga metadata source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for manga library type (host) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): manga filename index/volume parser * feat(scanner): manga series-name-from-folder detection Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(plan): align manga DB/scanner tasks to scanner pure-planner pattern (no test-DB) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(scanner): manga parser corpus regression Add TestParseMangaIndexCorpus — 36 real-world scanlation filenames covering bare chapter, decimal chapter, v/vol-prefix volume, and c/ch-prefix chapter patterns; asserts <5% miss rate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(db): manga_chapters link table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): manga_chapters repository + pure chapter-write mapping Adds mangaChapterWrite (pure, unit-tested), upsertMangaChapter, and listMangaChapters following the ebook/audiobook thin-SQL pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): recognize manga library type Add isMangaLibraryType helper (unexported, matching the style of isEbookLibraryType / isAudiobookLibraryType) with a corresponding TestIsMangaLibraryType unit test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(api): manga library content level Map library type "manga" to content level ["manga"] in metadataContentLevelsForLibraryType so that seedDefaultChain seeds a manga-level metadata provider chain when a manga library is created. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): route manga libraries to a manga scan path Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): group manga chapters under a manga series item Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): give manga series item a library membership so it browses Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): browse manga libraries as series Accept "manga" as a valid media_scope so a manga library browses only its type='manga' series items; the per-chapter type='ebook' items are naturally excluded because MediaScopeItemTypes("manga") expands to {"manga"}. Add the manga default library sections (scoped to media_scope='manga') so the library feed shows series cards. Refresh the two media_scope validation error messages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): manga series detail lists chapters For a type='manga' item, attach its chapters to the detail response via a new MangaDetailExtension. fetchMangaChapters joins manga_chapters to media_items on the chapter content ID, scopes to the series, and orders by chapter_index (NULLS LAST) then sort_title — matching the scanner's chapter ordering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga detail types + library browse scoping Add MangaChapter/MangaDetailExtension TS types mirroring the host catalog structs, wire manga? onto ItemDetail, and admit "manga" as a QueryDefinition.media_scope. Scope manga libraries to media_scope=manga in browse (host expands it to type=manga series items) while reusing the ebook sort universe via getLibrarySortRelevanceScope. Add isMangaLibraryType. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga series detail with volume-grouped chapter list Add MangaContent detail view: a DetailHero series header plus a chapter list grouped by volume. groupMangaChapters (pure, unit-tested) buckets chapters by their volume token, orders chapters within a group by chapter_index (nulls last) and orders groups by their minimum index; loose (volume-less) chapters collapse into a trailing "Chapters" group. Each chapter links to the existing ebook reader by content_id alone (file_id is optional — the reader resolves the file server-side), reusing buildMediaPlayHref. Admit "manga" into ItemDetail.type and wire the detail switch. Continue-reading is deferred (needs per-chapter progress fan-out / a last-read timestamp not in the current payload). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): handle manga in playable-type + collection filter-scope unions Adding "manga" to the shared ItemDetail["type"] and QueryDefinition["media_scope"] unions leaked into consumers with narrower local types, breaking the production tsc build. Fixes: - mediaNavigation: admit "manga" into PlayableMediaType. Manga series are not directly playable (you open the detail page and read a chapter, itself an ebook item), so buildMediaPlayHref falls through to the item href for them, like series/season. - FilterRuleEditor: add "manga" to FilterRuleMediaScope and relabel "watched" -> "Read" for manga as well as ebook (manga is read). - CollectionGuidedRulesEditor: add "manga" to GuidedFormState.mediaScope, a "Manga" media-type option, ebook-like "Read Status" labels, and map manga -> ebook sort-relevance scope (manga has no dedicated sort scope). - CatalogFilterBar (cascading leak surfaced after the above): add a "Manga" scope option and map manga -> ebook sort-relevance scope in both scope handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): offer manga as a library type in the create dialog Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): strip scene-release junk from manga series names Add cleanMangaSeriesName which repeatedly strips trailing parenthetical groups (year, year-range, Digital, release-group tags) then trims any dangling dash, so folder names like "404 Demons (Digital) (Oak)" resolve to "404 Demons". Wire it into mangaSeriesFromPath so both the series title and the mangaSeriesGroupKey identity key use the cleaned value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): flat volume/chapter manga list; nest only multi-chapter volumes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): parse manga index after stripping series-name prefix Numbers inside a series title (e.g. "404 Demons", "365 Days to the Wedding") were wrongly grabbed as the chapter number because parseMangaIndex matched the first bare number in the full filename. mangaIndexForFile now strips the series-name prefix before delegating to parseMangaIndex, so only the number that follows the title is used. reconcileMangaFile in manga_scan.go is updated to call mangaIndexForFile instead of parseMangaIndex directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): stop missing-file reconcile from deleting manga series items Manga series items are file-less virtual parents; the shared ReconcileFolderMembership swept them every scan because they have no media_file. Exclude type='manga' from file-presence membership reconciliation, and add a manga-scan step that deletes only series with zero remaining chapters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ebooks): exclude manga chapters from individual ebook enrichment Manga chapters are type='ebook' parts of a series; the ebook enrichment sweep was searching each one against book sources (Gutenberg/Anna's/etc.) and failing in a pointless storm. Exclude items with a manga_chapters link; series-level enrichment is handled separately. * docs: design spec for manga metadata plugin + series enrichment (sub-project 2) New silo-plugin-manga-metadata (AniList, high-confidence matching) + a host MangaEnricher for type='manga' series; default-enabled metadata source for manga libraries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for manga metadata plugin + series enrichment Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db): manga_enrichment_state table Mirrors ebook_enrichment_state: dedicated failure counter for the manga enrichment sweep so it does not contend with media_items.refresh_failures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(manga): series enricher (claims type='manga', resolves manga chain) * feat(manga): sync_manga_metadata task + enricher wiring * feat(catalog): expose manga chapter/volume counts in browse Add manga_chapter_count and manga_volume_count to browse cards so the frontend can render a Vols N / Ch N chip on manga series. The counts come from two index-backed correlated subqueries over manga_chapters in the browse SELECT (mangaCountColumns), scanned positionally before added_at and nilled out for non-manga rows. Threaded through models.MediaItem and exposed on the itemListResponse JSON card. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sections): scope manga home recent sections to type=manga series A manga library mixes type='manga' series with type='ebook' chapters, so the auto-generated home 'Recently Added/Released in <Library>' rows surfaced the junk chapter filenames. Add GeneratedHomeLibraryRecentConfigScoped which emits the modern QueryDefinition shape (library_ids + media_scope) so a manga library's generated home rows filter to type='manga' only. Other library types are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(catalog): exclude manga chapters from browse/section/search surfaces Manga CHAPTER items (type='ebook' rows linked into a type='manga' series via manga_chapters) were leaking into catalog browse, section resolution, and search as standalone items showing junk filenames. They are internal sub-units of the series and only the series should appear. There is no single shared item-listing chokepoint: browse, the query/preview executor, and search each build their own WHERE. Add a shared, index-backed anti-join predicate (manga_chapters.chapter_content_id is the PK) via mangaChapterExclusionWhere and wire it into all three builders. By-id fetch paths that legitimately resolve chapters (ebook reader, continue-reading, series detail chapter list) use separate queries and are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): use #NN as the manga volume for Vol.YYYY #NN releases mangaVolYearIssue early-return was returning the year token (e.g. "Vol.2003") as the volume label, which the frontend couldn't prettify to "Volume N". Now returns "v<issue>" (e.g. "v04") so the existing frontend regex ^v?(\d+)$ renders it as "Volume 4" correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): manga count chip on posters Add an optional manga_chapter_count / manga_volume_count to the browse item type and render a top-right "Vols N" / "Ch N" chip on ItemCard, strictly gated on type==='manga'. The label prefers "Vols" when the volume count dominates, "Ch" otherwise; the chip is hidden when the chapter count is missing or non-positive. No other card type renders it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): manga reader back returns to series (no loop) The ebook reader's back action defaulted to the chapter's own item detail (/item/<chapter>), whose back returned to the reader — an infinite loop for manga chapters. The reader now honors an explicit backTo search param when present, navigating there instead. Absent for normal ebooks, so their back behavior is unchanged. Only manga chapter rows pass backTo, keeping the fix manga-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga chapter row actions (read/mark-read/download) Each manga chapter/volume row now offers Read (the existing reader link, now carrying a backTo to the series), Mark-read (the shared watched-state mutation per chapter content_id), and Download (lazily fetches the chapter's file versions on demand and opens the shared DownloadVersionPicker, gated on user.download_allowed). The volume-unit / loose-chapter / section structure from buildMangaList is unchanged. Scoped to MangaContent only; EbookContent is untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): validate reader backTo param is a safe in-app relative path Prevents open-redirect / javascript:-URI XSS from a crafted ?backTo= URL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): include per-chapter read state in manga detail Manga chapters are ebook items, so a chapter is "read" when the viewer's ebook_reader_progress row crosses the finished threshold. fetchMangaChapters now LEFT JOINs that table scoped to the AccessFilter's user_id/profile_id and exposes a per-chapter Read bool on MangaChapter, threaded through buildMangaExtension. The detail payload previously carried no read state, so the row toggle always started unread. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga rows reflect read state on load MangaChapter now carries an optional read flag from the detail payload, and MangaRow seeds its mark-read toggle from chapter.read instead of always starting unread. The optimistic toggle + shared watched mutation are unchanged; only the initial value is seeded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sections): exclude manga chapters from recently-added/released/random + other library-listing sections Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sections): manga recently-added/released cards show the latest volume's cover * fix(manga): keep enrichment honest about no-match vs enriched, batch 50->200 - sweep stats now separate enriched / no_match / failed: a stamped no-match was counted (and logged) as an enrichment, which masked a collapse of the real match rate during the backfill - batch size 50 -> 200 (SILO_MANGA_ENRICH_BATCH overrides): with the plugin serving GetMetadata from its search cache an item costs one rate-limited AniList request, so a sweep still fits the 5-minute task interval Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): size enrich batch to the 5-minute interval at AniList's real budget 140 items x ~2.1s/request fits the interval; an overlong sweep makes the task manager drop the next trigger and the effective rate falls below the AniList budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): manga count chip data missing from library browse manga_chapter_count/manga_volume_count were only added to BrowseRepository, but /library/{id}?tab=library flows through previewQuerySource -> QueryExecutor.PreviewPage, which selects qualifiedListItemColumns and scans with scanItems - so manga cards never carried the counts and the Vols/Ch poster chip stayed hidden. Append mangaCountColumns to the preview-page SELECT and scan them via a new scanItemsWithMangaCounts (nil for non-manga rows, mirroring scanBrowseItems). Extract listItemScanDests so the three scan variants share one destination list instead of duplicating the 48-column scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): manga chip reads 'X Volumes · X Chapters', menu verbs say Read - chip: show distinct-volume and loose-chapter counts side by side instead of the single 'Vols N'/'Ch N' heuristic; mangaCountColumns now counts DISTINCT volume tokens (rows sharing a volume are one volume) and only un-volumed rows as chapters - watched-state labels: type='manga' fell through to the video default, so the card dot menu and detail page said 'Mark Watched' - manga now uses the ebook reading verbs (Mark Read / Mark Unread, 'Marked as read' toast) - format MangaContent.test.tsx (pre-existing prettier miss) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): backdrop enrichment - banner hero art + backdrop-only backfill - cache remote backdrops like posters (cacheRemoteImages generalizes the poster-only path; failures keep the provider URL, which still renders) - claim arm for enriched items missing a backdrop: fetched by stored provider ID (search skipped - no rate spend, no re-match risk) and only the backdrop is written; stamping after the attempt keeps banner-less series from being re-claimed every sweep - backfill = one-time SQL clearing last_refreshed for poster-set/ backdrop-empty manga Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): reading-loop UX - continue CTA, next chapter, series-aware cards, file details Fixes the four high-priority findings from the manga UX review plus a file-inspector request: - H1: series hero gets a Continue / Start Reading / Read Again CTA targeting the first unread chapter (firstUnreadChapter over the ordered list), plus an overflow menu (View Details, admin Refresh Metadata) - H2: the reader resolves its owning manga series (chapter detail now carries series_id/series_title) and offers next-chapter navigation: a header next button and an end-of-book floating CTA at >=99.5% progress; back defaults to the series even without a backTo param - H3: chapter rows show a persistent read check + muted title, and the mark-read mutation carries series_id so the series detail cache invalidates (read states no longer revert on revisit) - H4: continue-reading cards for manga chapters present the series: sections payload resolves chapter->series linkage, the card heading/image link to the series, and meta lines launch the reader - View Details: manga series menus (card dot menu + detail overflow) open a file inspector showing folder paths and per-chapter file names/sizes via GET /catalog/items/{id}/manga-files; paths are stripped for viewers without file-path visibility (item-versions policy) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): UX mediums - richer detail page, smarter list, manga sort scope Second batch from the manga UX review (M1-M7): - M1: multi-chapter volume sections are collapsible (fully read sections start collapsed) with sticky headers, and long series get a 'Jump to <next unread>' anchor above the list - M2: the series hero shows the author line (HeroCrewLine learns Author credits with person links; DetailHero now renders crewLine and genre chips independently) and Volumes/Chapters badges - M3: browse-card count chip abbreviates to '12 Vol - 3 Ch' so it fits narrow cards without occluding covers - M4: manga gets its own sort scope: Duration/Bitrate (meaningless for file-less series rows) disappear, reading labels (Date Read / Reads) apply, Author stays - M5: global search labels manga results 'Manga' instead of the raw type - M6: chapters carry the viewer's reading fraction; part-read rows show an inline progress bar + percent - M7: chapter rows show the extracted cover thumbnail (presigned poster_url on the chapters payload) instead of a generic icon Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): UX lows - volume token dedupe, comic reader chrome, empty-state hint - buildMangaList buckets volumes by canonical numeric token so mixed release naming (v01 + 1) yields one Volume 1 instead of duplicates - cbz/cbr readers start with the side panel closed and hide prose-only chrome (reading ruler, TTS, typography/font controls, hyphenation, writing mode) while keeping comic-relevant settings (theme, brightness, margin, right-to-left, spread, flow) - manga empty state mentions chapters appear after the library scan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): publication status badge via new SDK status field - vendor the unpublished plugin SDK (adds MetadataItem.status) under internal/compat/ with a relative go.mod replace, following the zishang520-webtransport-go convention; swap to the published module before the upstream PR - map plugin status into MetadataResult.ShowStatus, persist it during manga enrichment, and show it as the hero status badge (show_status was already on the detail payload and MetadataBadges) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): generalize backdrop pass to secondary fields (backdrop + status) The backdrop-only claim arm becomes a secondary-fields pass: enriched items missing a backdrop and/or publication status are claimed, fetched by stored provider ID, and only the missing secondary fields are written. Lets the new status field backfill across the already-enriched library instead of applying only to future enrichments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): merge ShowStatus through MergeMetadata/MergeGlobalMetadata The new MetadataResult.ShowStatus never reached the accumulated result the manga enricher persists from - the field-by-field merges didn't know it, so the status backfill pass obtained nothing. Regression-tested on both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): keep scanner identity IDs out of the metadata flow filterMangaProviderIDs passed the scanner's manga_series identity row through, so the search-skip-when-already-matched guard saw provider IDs on every item and never searched: unmatched items went straight to a by-ID fetch with no usable ID and were stamped as terminal no-match without a single provider request (and the MangaDex fallback was never consulted). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: gitignore docker-compose.override.yml (local deployment override) The override unpublishes the bundled redis/postgres host ports (ports: !override []). It is a per-deployment, local-only file: ignoring it keeps a rebase from main and git clean -fd from disturbing it, and keeps it out of any PR. Its accidental absence once exposed Redis to the internet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): code-review fixes — no-match guard, sort comparator, volume-count consistency - enrichWithProviders: set accumulator.HasMetadata after a provider result merges (MergeMetadata doesn't propagate it). Without this, a confident match carrying only genres/authors/status/year but no cover and no overview failed the no-match check and was discarded + terminally stamped. - byChapterIndex: both un-indexed chapters yield POSITIVE_INFINITY, so the subtraction was Infinity-Infinity=NaN (Array.sort treats NaN as 0, leaving order undefined). Compare explicitly for a stable order. - MangaContent volume/chapter badges: derive counts from the rendered buildMangaList entries (which canonicalize v01 ≡ 1) instead of raw distinct volume tokens, so the badge can no longer say '2 Volumes' over one row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(manga): clarify the enrichment claim's secondary arm is admin-reset-only The secondary arm (poster present, backdrop/status missing) requires last_refreshed IS NULL, so it is only reachable when an operator resets last_refreshed to backfill a newly-added field — not an automatic periodic re-check (which would re-fetch banner-less series every sweep). Documents the intent so it does not read as dead code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): collapse continue-reading chapters per series; batch provider-id lookup - Continue Reading now collapses multiple in-progress chapters of the same manga into one card (most recently read kept), mirroring the episode→series collapse. The reading section resolves chapter→series linkage into itemMeta (applyMangaChapterSeriesMeta) and runs the shared collapseContinueWatchingSeriesCandidates, which the reading path previously skipped. - claimBatch resolves provider IDs for the whole batch in one query via the new ProviderIDRepository.GetByContentIDs (content_id = ANY), replacing the per-item GetByContentID N+1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): manga publication-status chip on browse cards + more legible chips - Color-coded publication status pill (Ongoing/Completed/Hiatus/Cancelled/ Upcoming) in the manga card's top-left corner, mirroring the vol/chapter count chip top-right. Strictly manga-gated; show_status was already on the browse payload. - New .glass-chip (78% surface vs glass-subtle's 40%) for the manga count + status pills so the labels stay legible over busy cover art. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(manga): depend on published silo-plugin-sdk v0.7.0 Replace the vendored internal/compat/silo-plugin-sdk copy with a normal dependency on the published SDK module at v0.7.0, which adds MetadataItem.status (publication/airing status) consumed by the manga status badge at internal/metadata/plugin_provider.go. - go.mod: pin v0.7.0, drop the local-path replace directive - remove the vendored internal/compat/silo-plugin-sdk tree - Dockerfile: drop the vendored-SDK COPY - strip the manga design docs/plans from docs/superpowers (internal) Requires Silo-Server/silo-plugin-sdk#4 merged and tagged v0.7.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(manga): exclude chapters from the matcher's unmatched-item lister Manga chapters are type='ebook' items that stay status='pending' by design - provider metadata lives on the type='manga' series item. The scan-final RetryUnmatchedItemsByFolderAndPathPrefix listed all of them and ran a rate-limited ebook-plugin search per chapter: 31,564 chapters x ~1s = 8h46m appended to a 2-minute manga library scan (observed live), every one a guaranteed no-match. Earlier runs never survived to completion, so the library's last_scanned_at stayed NULL forever. Add the same manga_chapters NOT EXISTS guard the ebook enricher's claim query already uses. Verified live: the same library now scans in 27s with retried_items=0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): never probe-repair ebook/comic files (ebook+manga detail-page killer) NeedsCriticalProbeRepair was always true for BaseType 'ebook' files (epub, pdf, cbz, cbr — incl. manga chapters): buildEbookMediaFile leaves ProbeUpdatedAt nil and they have no audio/video, so probeEnsurer.Ensure spawned ffprobe per file on every detail/watch load and never converged (ffprobe errors on zip/rar, result never persisted). Short-circuit probe-repair for ebook base type — they're read directly and never use the transcode/playback probe pipeline. SHARED fix: benefits both the ebooks and manga library types. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf+fix(ebooks): parallelize detail extension + preserve finished read-state - buildEbookExtension ran its 3 related-content queries (series, also-by-author, similar) sequentially; run them concurrently like buildAudiobookExtension so ebook detail latency is the slowest query, not their sum. - PGEbookReaderProgressStore.Upsert did an unconditional SET progress=EXCLUDED; a routine autosave (e.g. reopening a finished book) could drop it below the 0.9 finished threshold and silently un-mark it read (and clear the manga chapter checkmark, which rides on the same row). Guard: once finished, progress only moves on an explicit unread (row delete); below threshold it tracks freely. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(manga): batch chapter presign, index volume counts, quiet scan log - fetchMangaChapters presigned each chapter poster individually; a long-running series has hundreds of chapters. Batch them in one PresignImageURLs call, and add the missing rows.Err() check (was silently returning partial lists). - The browse manga count chip's count(DISTINCT volume) subquery wasn't covered by manga_chapters_series (series_content_id, chapter_index); add idx_manga_chapters_series_volume (series_content_id, volume) so both count subqueries are index-only. - Downgrade the per-chapter "manga scan: indexed" log from Info to Debug (one line per .cbz; the 500-file progress log already covers operator visibility). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(manga): address PR #138 code-review findings Folds PR #142 into the manga branch (already done via fast-forward) and remediates the issues surfaced in the #138 code review. Correctness: - Preserve the scanner's manga_series identity anchor through enrichment. ReplaceByContentID's DELETE was unconditional, so the first successful enrichment wiped the manga_series provider-id row the scanner relies on for idempotency, causing duplicate series + metadata loss on the next scan. excludedProviderIDs now also means "not deleted", and the DELETE preserves those rows. (internal/catalog/provider_id_repo.go) - Fall back to the series cover when the latest chapter has no poster. Poster columns default to '' (not NULL), so the manga series-card poster override blanked cards via a plain COALESCE; wrap operands in NULLIF. (internal/sections/fetcher.go) - Keep backTo a real query param on reader links when libraryId is absent. It was string-concatenated with '&', producing a malformed URL on deep-links; route it through the query helper instead. (web/src/lib/mediaNavigation.ts, EbookReader.tsx, MangaContent.tsx) Quality: - Hide manga chapters from favorites/watchlist browse, matching the exclusion enforced on every other listing surface. (internal/catalog/favorites_browse.go) - Centralize the manga chapter exclusion predicate into a single exported catalog.MangaChapterExclusionWhere, removing four duplicated copies. (catalog, sections, ebooks) - Skip the two manga count subqueries on browse scopes that cannot contain manga (non-manga type filters), substituting NULL placeholders. (internal/catalog/browse.go) - Normalize provider publication status (AniList/MangaDex/SDK variants) into a stable label set so show_status carries one manga value-domain. (internal/manga/enrichment.go) Adds unit tests for the poster NULLIF contract, browse gating, and status normalization. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: regenerate go.sum after rebase onto main Drops stale silo-plugin-sdk v0.6.0 and other leftover hashes from the intermediate rebased states; go.mod is now on the published v0.7.0 tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): adapt manga scan to ebookFileShouldSkip 3-value signature main changed ebookFileShouldSkip to also return the existing content ID; the manga scan path only needs the unchanged flag, so discard the new return. Resolves a silent semantic conflict from the rebase onto main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Silo Server Developer <warmasterx555@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 20:13:10 -04:00
4 people	eb6024573e	feat(audiobooks): make audiobook libraries first-class catalog items (#73) ... * docs(audiobooks): design spec for plugin absorption Plan to absorb silo-plugin-audiobooks into silo-server as a first-party feature. Audiobooks land in silo's existing SPA; ABS clients connect directly. Hard constraints: reuse existing tables (media_items, media_files, user_watch_progress, user_playback_sessions, people, item_people, library_collections); only two new tables (abs_sessions, podcast_feeds) and at most one column add (media_libraries.kind); silo's main :8080 listener handles ABS Socket.io natively. Out of scope: audiobook requests flow, smart collections, share links, external recommender, custom metadata providers, separate audiobook SPA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 1 (discovery + schema) First of six sub-plans for the absorption. Six tasks: a discovery audit that resolves the spec's Risk questions, four idempotent SQL migrations (abs_sessions, podcast_feeds, media_libraries.kind, audiobooks.enabled feature flag), and an empty-but-compiling internal/audiobooks package scaffolded into cmd/silo. Lands as a strict no-op for users (feature flag defaults to false). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): discovery findings for absorption sub-plan 1 Locks schema/code decisions for migrations 139-142 and downstream sub-plans. Resolves open Risk questions from the absorption design spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 139 add abs_sessions table Parallel of jellycompat_sessions for Audiobookshelf-compatible clients. Lets ABS mobile/desktop apps maintain a device-bound session that silo's audiobooks/abs handlers will validate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): match codebase conventions in migration 139 Lowercases type keywords in the abs_sessions CREATE TABLE body to match neighboring migrations, fixes the client_version column alignment, and replaces the misleading "parallel to jellycompat_sessions" header comment with a more accurate description of the table's role. Cosmetic only — the running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 140 add podcast_feeds table Side table on media_items for RSS-subscribed podcasts. Holds feed URL, ETag/Last-Modified for conditional fetches, last-refresh timestamp, and the per-feed refresh interval consumed by the upcoming podcastfeed.Refresher scheduled task. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): uppercase PRIMARY KEY in migration 140 Aligns with the codebase convention (type keywords lowercase, constraint keywords uppercase) established in migration 139's post-style-fix form. Cosmetic only — running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(audiobooks): migration 141 no-op for media_folders.type Sub-plan 1 originally reserved migration 141 to add a 'kind' column to media_libraries discriminating audiobook/podcast libraries. Discovery audit (sub-plan 1 Task 1) found that the actual table is media_folders and it already has a type text NOT NULL column with no CHECK constraint or enum, so 'audiobooks' and 'podcasts' can be added as future values without DDL. Landing this migration as a documented no-op preserves the version numbering audit trail and pins the decision in git history. The matching down migration is also a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 142 add audiobooks.enabled flag Server-settings row that gates the absorbed audiobooks feature. Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent sub-plans branch on this flag and operators flip it to 'true' at cutover. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scaffold internal/audiobooks package Empty-but-compiling Service that reads the audiobooks.enabled feature flag from server_settings. Wired into cmd/silo so the package is referenced from the binary; no routes mounted, no scheduled tasks registered, no DB writes. Subsequent sub-plans hang scanner branches, ABS handlers, Socket.io, podcast refresher, and SPA pages off this Service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): cosmetic cleanups in scaffolded package Two pre-emptive cleanups flagged by code review before sub-plan 2 copies the patterns: 1. Sort the internal/audiobooks import after internal/adminjob in cmd/silo/main.go (alphabetical). 2. Drop the redundant "audiobooks: " prefix from the Enabled() error wrap; matches how every other top-level service package (watchstate, scanqueue, metadata, etc.) formats errors. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 2 (scanner) Second of six sub-plans. 10 tasks: PersonKind constants for Author and Narrator, audio-extension recognizer, library-type helpers, a walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter extraction via ffprobe, single-file and multi-file audiobook parsers, scanner write path producing media_items.type='audiobook', and a filesystem podcast parser (RSS deferred to sub-plan 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add Author and Narrator PersonKind constants Discovery audit confirmed item_people.kind is unconstrained smallint with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook people-links written by the upcoming scanner branches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add audio-extension recognizer for scanner Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by upcoming audiobook and podcast scanner branches to filter directory walks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library-type recognizers for scanner dispatch isAudiobookLibraryType and isPodcastLibraryType match singular and plural forms case-insensitively, mirroring isMovieLibraryType. Used by upcoming scanner walk branches (Task 4) that filter audio files into audiobook and podcast libraries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(scanner): replace movieLibrary bool with typed walkMode Lets walkLogicalTree dispatch on multiple library shapes (video, movie, audiobook, podcast) without proliferating boolean flags. Behavior for existing video and movie libraries is unchanged; audiobook and podcast modes will be consumed by the upcoming audiobook.go and podcast.go parsers in later tasks of this sub-plan. walkModeFor() derives the mode from a media_folders.type string; unknown types default to walkModeVideo to preserve prior behavior for any caller still passing a raw type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose ffprobe format tags on ProbeData The audiobook scanner needs format-level tags (title, artist, album, date) for media_items metadata; ffprobe already parses them in ffprobeFormat.Tags but ProbeData previously discarded them. Add FormatTags map[string]string to ProbeData, populate it in convertProbeData via a new normalizeFormatTags helper that lowercases keys and trims values. Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and format tags, and a test that verifies ProbeFile() returns both correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): parser for single-file audiobook folders parseAudiobookFolder reads tags + chapters via the existing ProbeFile (now that Task 5 exposes FormatTags on ProbeData) and produces a parsedAudiobook struct. Title falls back from "title" tag to "album"; author from "artist" -> "album_artist" -> "composer"; series from "album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools). Year parsed from "date" or "year" tags, tolerating ISO dates and parenthesized forms. Single-file case only; multi-file folders (one audio file per chapter) return a placeholder error and arrive in Task 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): multi-file audiobook folder support Folders containing N audio files (one per chapter/part) get one parsedAudiobookFile per file; each file's chapter list is synthesized as a single chapter with title = filename stem. Title/author/series/ year come from the first file's tags. Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in favor of the existing firstNonEmpty already in probe.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scanner write path produces audiobook media_items ScanAudiobookFolder walks an audiobooks-typed media folder and treats each immediate subdirectory as one audiobook. For each parsed audiobook it upserts: - one media_items row with type='audiobook' - one media_files row per audio file (with chapters JSONB) - author/narrator links in item_people (kind=7, kind=8) Adds itemRepo and personRepo to the Scanner struct, wired from fileRepo.Pool() in NewScanner — no constructor signature change needed. ScanFolder dispatches to this path when folder.Type='audiobooks', bypassing the per-file movie/TV pipeline because audiobooks are folder-scoped entities. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): filesystem podcast scanner ScanPodcastFolder walks a podcasts-typed media folder, treating each subdirectory as a podcast show and each audio file inside as an episode. Writes media_items.type='podcast' + episodes rows + media_files rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5; this task covers filesystem-only ingestion. ScanFolder dispatches to this path when folder.Type='podcasts'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 5 (podcasts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose audiobooks/podcasts library types in admin UI Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown in the admin libraries page so operators can flag a folder as an audiobook or podcast library. Extends contentLevelsForType() so the admin UI's downstream filtering treats those types correctly (audiobook -> ['audiobook'], podcasts -> ['podcast', 'podcast_episode']). Backend scanner branches for these types were already wired in sub-plan 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge origin/main adds 139_media_requests at the same number our local audiobook branch had used for abs_sessions. Renumber ours to 147 to free up 139 for the upstream migration. The schema_versions row is updated in lockstep on the running database so the migrator sees the abs_sessions migration as already applied at its new version. Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook feature flag, abs playback sessions, podcast episode guid, audiobook series, audiobook title cleanup) stay where they are — they don't collide with anything on origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge origin/main added 140_user_permissions at the same version this branch had used for podcast_feeds. Renumber ours to 157 (next free above the collections-unify migration at 156) so 140 is free for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees podcast_feeds as already applied at its new version. Same pattern as d59c1cb (renumber 139_abs_sessions to 147 for the prior main merge). Pending migrations after this rename: 132 (downloaded subtitles admin index, main), 140 (user_permissions, main), and 156 (unify_user_collections, this branch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 141_media_folders_kind_noop to 159 for origin/main merge Same shape as eb8f67d (the 140→157 renumber from the previous main merge). origin/main added 141_episode_title_sort_index at the same version this branch had used for media_folders_kind_noop. Renumber ours to 159 (next free above the audiobook_series truncate at 158) so 141 is open for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees media_folders_kind_noop as already applied at its new version. Pending migrations on silo-prod after this rename: 141 (episode_title_sort_index, main) and any other newer ones from main that the branch hasn't picked up yet. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 142_audiobooks_feature_flag to 160 for origin/main merge Companion to 3c6f062's 141 renumber — origin/main also added 142_episode_catalog_entries (alongside 141_episode_title_sort_index) at a version this branch had used for the audiobooks feature flag. Renumber ours to 160 so 142 is open for the upstream migration; schema_versions on silo-prod is updated in lockstep so the migrator sees audiobooks_feature_flag as already applied at its new version. This was the only remaining collision (verified by checking for duplicate version prefixes across migrations/). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audiobooks): address foundation review comments * fix(audiobooks): tighten scanner identity handling * fix(audiobooks): propagate scanner cancellation * chore(audiobooks): adopt goose migration layout * docs(audiobooks): implementation plan sub-plan 3 (API + frontend MVP) Third of six sub-plans. 9 tasks: three REST endpoints (list/detail/ progress), TanStack Query hooks + types, three React pages (Library/Detail/Player), and navigation integration. Scoped to MVP — author/series indices, smart collections, share links, and other nice-to-haves from the spec are deferred. Streaming reuses silo's existing /api/v1/stream/{session_id}; no new transcode code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): list endpoint at GET /api/v1/audiobooks Paginated list of media_items with type='audiobook' scoped to the caller's accessible libraries via the existing access filter. Mirrors silo's existing list-style handlers for movies and series. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): detail endpoint at GET /api/v1/audiobooks/{id} Returns the media_items row, its media_files (with chapters JSONB), author/narrator extracted from item_people (kinds 7/8), and the caller's per-profile listening progress from user_watch_progress. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): progress endpoint at POST /api/v1/audiobooks/{id}/progress UPSERTs user_watch_progress for the caller's (user_id, profile_id, content_id). Body carries position_seconds; clients are expected to post every 5-10s during playback plus on pause/seek (matching silo's existing video progress cadence). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): frontend types and TanStack Query hooks TypeScript types match the JSON shapes from the new /api/v1/audiobooks endpoints (list, detail, progress). Three hooks: useAudiobookLibrary (list), useAudiobook (detail), and useReportAudiobookProgress (mutation that invalidates the detail query on success so progress updates reflect immediately). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library grid page at /audiobooks Renders a paginated grid of audiobook cards using the useAudiobookLibrary hook. Each card links to /audiobooks/book/{id}. Cards show poster, title, and year; falls back to a "No cover" placeholder when the audiobook has no poster_url. Empty state hints to operators that they need to set a library's type to 'audiobooks'. Routes themselves are wired in Task 8 (navigation integration). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): detail page with chapter list Renders cover, title, author, narrator, year, and overview alongside a chapter list. Clicking a chapter opens an inline sticky AudiobookPlayer at that chapter's start. A "Resume" button restarts playback at the saved progress position if present. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): HTML5 audio player with chapter navigation Single-file audiobook playback for MVP. Multi-file queuing arrives in a follow-up. Streams via the existing /api/v1/direct-download GET endpoint. Position is reported to /api/v1/audiobooks/{id}/progress every 10s while playing plus on pause/seek/end. Skip-30s, playback rate select, chapter list panel. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): wire navigation and routes Adds an Audiobooks entry to the sidebar and registers the two new routes (/audiobooks for the library grid, /audiobooks/book/:id for detail). The player renders inline inside the detail page; no dedicated player route is required for MVP. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audiobooks): address native API review comments * feat(audiobooks): add ABS compatibility and polish * fix(audiobooks): stabilize ABS playback progress reporting * fix(audiobooks): clean up ABS branch review fixes * chore(audiobooks): adopt goose layout for ABS migrations * fix(audiobooks): align player seek bar props * feat(audiobooks): make libraries first-class catalog items * feat(admin): add server restart endpoint * fix(audiobooks): address review comment findings --------- Co-authored-by: RXWatcher <14085001+RXWatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-06-07 15:57:05 -04:00
Silo Server Migration	c085b12fd1	Initial Silo migration	2026-05-22 23:26:56 -04:00