codex/bound-transcode-segments
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
881c96864b |
feat(playback): finalize platform-neutral protocol v3 (#567)
* docs(playback): add v3 neutral-contract finalization plan Supersedes the wire-contract sections of the 2026-07-12 v3 plan: server-owned attempt keys, delivery-keyed negotiation without Media3 engine names, tiered capability evidence, neutral device/output context, track/quality replan operations, audio-only planning, and coordinated no-back-compat rollout across server, Android, Apple, and web. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(playback): make v3 attempt keys server-owned and replace engines with deliveries Contract core of the platform-neutral v3 finalization (plan sections 3.1 and 3.2), breaking on purpose — v3 is dark and all clients move together: - Every PlanV3 now carries plan_attempt_key, an opaque server-computed token clients store and echo in attempted_plan_keys; ReplanRequestV3 gains bounded local_mutations that the replan handler folds into the failed plan's key. Clients never hash anything. - KotlinName() is deleted from DeliveryV3, StreamProtocolV3 and SubtitleModeV3; the attempt-key canonical string now uses lowercase wire tokens, and PlanRecipeVersionV3 bumps to v3.3 so no key or plan ID computed under the old canonicalization can collide. - EngineV3 leaves the wire: ClientPlaybackContextV3.Engines (media3_*) becomes Deliveries keyed original_http|progressive|hls, with EngineCapabilityV3 renamed DeliveryCapabilityV3. PlanV3.Engine is removed; the planner, subtitle policy and quirk registry re-key on delivery class, and the media3_only feature token is deleted. - Validated-claim strings drop the prefix: media3_h264_decode -> h264_decode, media3_audio_decode -> audio_decode. - Golden fixtures in testdata/protocol_v3 are regenerated by Go and are now the cross-repo source of truth. Part of the playback protocol v3 neutral-contract train (steps 2-3 of docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(playback): add v3 evidence tiers and neutral device/output context Implement plan sections 3.3 and 3.4 of the v3 neutral-contract pass: - ClientCodecCapabilitiesV3 gains required video_evidence and audio_evidence closed enums (exact | platform_attested | declared). Planner strictness follows the tier: exact keeps the strict decode-entry validation, platform_attested validates codec/resolution/bit-depth/ frame-rate but skips profile/level matching, declared grants copy routes from the flat codec lists. Only exact audio evidence earns passthrough claims. The detailed_decode_capabilities feature token is deleted (subsumed by video_evidence=exact), and evidence-blocked direct routes carry the new evidence_insufficient_for_direct reason/warning. - DeviceContextV3 is now platform/os_version/manufacturer/model plus a bounded platform_details map (<=16 entries, <=128 chars); the Android Build dump fields are gone. Fire TV quirks keep matching on manufacturer/model (brand fallback removed with the field). - output_route_generation (int64, dual-location) becomes an optional opaque output_context_id string on the output context; the dual-location consistency validation is deleted. Attempt keys, plan invalidation, route events, and the planstore column follow (new Goose migration). - Feature advertisement collapses to the top-level client_features list only; ClientPlaybackContextV3.Features is deleted and ReplanRequestV3 gains an optional client_features refresh. - PlanRecipeVersionV3 bumped v3.3 -> v3.4; fixtures re-keyed. Part of the playback protocol v3 neutral-contract finalization plan (docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(playback): add v3 intent replans, quality menu, and audio-only routes Protocol v3 could only replan after a failure, so changing the audio track or the quality still required the legacy audio PATCH and the client-recipe transcode start — the two endpoints v3 is meant to replace. Clients also had to own a resolution ladder to render a quality menu, and a source with no video track was terminaled by the video/HDR gates, keeping audiobooks on the legacy path. Add track_change and quality_change replan operations. They carry no failure classification and route through the existing replan transaction, so they inherit its idempotency, capacity reservation, and staged-successor commit for free. Because nothing failed, the previous route stays eligible: neither the attempted-key history nor the failed-plan exclusion applies to them. Publish the server ladder on the plan as available_qualities so the quality menu is server-owned; the rungs come from the same resolutionLabelV3 and ladderBitrateKbpsV3 helpers the planner itself uses, not a parallel table. Plan audio-only sources through their own reduced route family: original_http when the client decodes the codec, otherwise a progressive AAC conversion. The plan advertises audio/mp4 for that remux and the transport now serves the same value, because a declared-tier client probes the advertised MIME with isTypeSupported before attaching a source buffer, and "video/mp4" on a stream with no video track is exactly the mismatch that makes the probe lie. Name the protocol's string vocabulary (dynamic ranges, transformations, executors, validated claims, terminal reasons) as constants while touching these lines, so the wire values have one definition. Part of #135 * docs(playback): publish the v3 protocol contract and fix subtitle ordinals Protocol v3 exists only as Go code today, so the Android and Apple ports have no authority to implement against other than reading this repository. Publish the contract as a normative document, machine-checkable schemas, and generated golden fixtures, and fix the one place where the server's own wire output disagreed with the ordinal space it publishes. - docs/architecture/playback-protocol-v3.md is self-contained enough for a third-party client: endpoints and status codes, evidence tiers and their bound-matching rules, delivery classes, the timeline model, replan semantics, registries, track identity, plan identity, quality, and transformations. - docs/design/schemas/playback-v3/ carries JSON Schemas for the five wire shapes plus valid and invalid fixtures, following the client-diagnostics layout. internal/playback/contract validates every fixture against its schema, so a schema that drifts from the Go types fails the Go suite. - cmd/playbackfixtures generates internal/playback/testdata/protocol_v3 from the production planner. `make playback-fixtures` writes them and `make verify-playback-fixtures` (wired into CI) fails when they are stale. These files are what the client ports consume, so drift would otherwise surface as a playback bug on three platforms at once. The subtitle fix: combined ordinals are one dense space over externals, then embedded tracks, then downloaded ones, but the legacy URL builder skipped burn-in-only tracks while assigning indices, so every track after a DVD/DVB track was numbered one too low and resolved to its neighbour. Ordinal assignment now lives in playback.BuildSubtitleInventoryV3 and both the plan inventory and the legacy `subtitle_urls` shape project from it; the legacy shape still filters burn-in-only entries but keeps each track's real index. Part of #135 * feat(web): migrate the players to the neutral playback v3 contract The web player was the last client still speaking the legacy start protocol: it picked its own file version from a codec probe, posted an ffmpeg recipe to start a transcode, PATCHed an endpoint to change audio tracks, and derived its own quality ladder. None of that survives a server-owned plan, and none of it produced telemetry the apps could be compared against. Video player: starts with a v3 request that advertises `declared` evidence from `isTypeSupported` probes and the three delivery classes, then consumes the returned plan for its URL, timeline, tracks and warnings. Quality and track changes become replans (`quality_change`, `track_change`), the quality menu renders `available_qualities` instead of computing rungs, and playback failures emit `route-events` so web failures land in the same diagnostics as Android and Apple. The duration comes from `source.duration_seconds` rather than the playback engine, and the "how was this delivered" overlay reads the plan's delivery and server transformations instead of comparing codec strings. Audiobook player: starts against the audio-only planner path with a single `original` rung, and takes its seek anchor from `timeline.player_start_seconds` so the progressive-remux route (which anchors the stream and restarts the player clock at zero) does not seek twice. Server side, `disable_progress_persistence` left the wire, so the rule it encoded is now derived. Resume state is keyed on the item, but every part of a multipart presentation shares that key while carrying its own file-local clock — persisting part 4's position would store "12 minutes in" as the book's resume point. `PresentationPartTotal > 1` expresses that directly and generalizes to multipart movies and split episodes, and a client can no longer forget to ask or lie about it. `useTranscodeQuality` and the legacy response types are deleted, and `WEBTEST_KNOWN_FAILURES` loses the audiobook entry along with its fix. Part of #135 * feat(playback)!: make v3 the only playback protocol Protocol v3 shipped behind a flag, alongside the legacy start path it was designed to replace. Running both meant every planner change had to be made twice, in two shapes that disagree about who decides the route: the legacy body carried a decision the client had already made, while v3 asks the server to make it. This deletes the legacy half. Removed: - `handleStartPlaybackLegacy` and its request/response bodies. The `POST /playback/start` route stays, but the protocol-version dispatch envelope is now a strict v3 decode — a body that does not declare `protocol_version: 3` gets `426 client_upgrade_required` so an outdated app can render a clear "update required" state instead of misreading a plan. Deliberately not a `400`: the request may be well-formed for the protocol it was written against. - `POST /playback/transcode/start`, superseded by the `quality_change` replan operation, and `PATCH /playback/{session_id}/audio`, superseded by `track_change`. Both mutated a session without re-planning. - The shadow planner and both rollout settings rows. With v3 the only protocol, `playback.protocol_v3_enabled` would mean "no playback at all"; `playback.protocol_v3_shadow_enabled` gated a comparison against a path that no longer exists. `409 protocol_disabled` on route-events goes with them, and capability `enabled` is now constant `true` (the field stays — clients feature-detect against it). - Version-selection helpers in `internal/playback/resolver.go` that only legacy start reached. `Resolve`/`ClientCapabilities`/`PlayDecision` stay: downloads consumes them. `internal/jellycompat` has its own resolution surface and is untouched. Behaviour the legacy handlers owned and v3 now owns explicitly: series version and audio-track preferences are persisted on start and on a `track_change` replan (not on failure recovery, whose forced route is not a user choice); an omitted `start_position` resolves to the profile's saved resume point; and an omitted audio track resolves through the series preference, the profile audio language, then the library override. Both are settled before planning, because the plan's timeline is cut at the start position. Spec §2.2 documents this as "omission is a request, not a default". The encode-target clamp that lived in the deleted transcode handler is already enforced in the planner, twice — `availableQualitiesV3` omits rungs at or above the source height, and the encode path clamps `targetHeight` to it. Unchanged: progress, stop, HLS manifest and segment delivery, the realtime control socket, stream tokens and restart reconstruction, watch together, downloads, jellycompat. Every removal is recorded in the pre-lock removals table in docs/architecture/v1-scope.md. Part of #135 * fix(scanner): stop recording embedded cover art as a video track ffprobe reports embedded cover art as a video stream carrying disposition.attached_pic. convertProbeData appended every "video" stream to VideoTracks without consulting isMainVideoStream, the predicate that already existed for duration decisions, so the picture was persisted as a playable track. That misreports the file twice: - An audio file with a cover picks up a video track, so it no longer satisfies MediaFile.IsAudioOnly and the v3 planner routes an audiobook through the video path instead of planAudioOnlyV3. - When the picture is ordered ahead of the real stream, the flat codec_video/resolution/hdr columns describe the poster: a 954x720 h264 episode was stored as mjpeg 480x480. Filter attached_pic streams out of the track loop. The guard is the disposition flag, not the codec name, so a genuine MJPEG video is still probed as video — the library has one. Already-probed rows self-heal on the next playback: NeedsCriticalProbeRepair already reprobes tracks missing color_range, which covers 21 of the 23 affected rows, and applyProbeData overwrites VideoTracks wholesale. The remaining two need a rescan; nothing persisted records attached_pic, and keying repair off still-image codec names would reprobe the genuine MJPEG file on every playback forever. Part of the playback v3 neutral-contract work: it is what lets Android drop AUDIOBOOK_COVER_ART_CODECS, which fabricated decode support the client cannot honestly claim under video_evidence: "exact". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(playback): publish subtitle URLs even when playback starts with subtitles off The v3 plan's subtitle inventory is the authoritative track list a client builds its subtitle menu from, but the handler only rewrote it with session-scoped URLs when a track was actually selected. A start or replan that resolved to `subtitle.mode: "off"` therefore returned the planner's URL-less inventory, so a client whose picker reads the inventory had a menu it could not fetch anything from. The Cast path hits this every time: it starts with subtitles off and needs the receiver's text tracks up front. attachSubtitleArtifactV3 now scopes and publishes the inventory unconditionally and gates only the artifact stamping on the selection. Spec §8 records that the `url` on a sidecar entry does not depend on the current selection. Part of the v3 neutral-contract finalization. * chore(playback): reconcile neutral v3 with main * fix(playback): preserve subtitle intent across replans * fix(playback): retain subtitle inventory on adapted routes * fix(playback): software-decode High10 AVC for QSV * fix(playback): scale High10 frames before QSV upload * fix(playback): preserve empty subtitle inventories * fix(playback): freeze terminal attempt contract * chore(playback): name fixture contract tokens * fix(playback): close v3 conformance review gaps * chore(playback): name conformance category * fix(playback): complete v3 conformance contract * fix(playback): keep schema fixtures generated * fix(playback): emit schema-valid conformance arrays * fix(playback): omit empty replan failures * fix(web): omit empty replan failures * fix(playback): close neutral v3 contract gaps * fix(playback): harden v3 replan, transcode, and quality-ladder edge cases Review remediation for the neutral v3 cutover, server side: - A failed replan no longer overwrites the durable StartResponse with a terminal or advances the replan request ID; an idempotent start replay of a still-healthy session returns the original plan. - SoftwareVideoDecode is now derived inside the transcode layer from source facts (codec/profile/bit depth) carried on TranscodeOpts, so jellycompat, downloads, recipe-card reconstruction, and transcode nodes get the High10 software-decode fix, not just the v3 handler. video_to_h264 recipe version bumps to 2 so mixed-version node pools that would silently drop the flag fail validation instead. - Local transport startup shares the 30s ManifestStartupTimeout; a timeout with the process still running stays retryable and is no longer persisted as a durable terminal against the attempt. - Sparse replan bodies (failure_recovery et al) no longer reset a user-selected quality preference to auto; the empty-value guard now covers every operation. - availableQualitiesV3 publishes no fixed rungs when the source height is unknown, keeping the no-upscaling ladder contract. - The proxy remux path serves audio-only fMP4 as audio/mp4 via a new additive AudioOnly token claim, matching the integrated path. - Plain text subtitle sidecars accept any requested extension again (served as VTT), restoring the permissive v1 behavior; ASS and bitmap handling is unchanged. - The 4K-disallowed terminal message discloses when a lower-resolution alternate exists but was pinned away by quality "original". Part of #135. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): keep playback alive through failed replans and honest audio claims Review remediation for the neutral v3 cutover, web player: - A failed or refused replan no longer unmounts the player: the fatal error screen is reserved for loads with no adopted plan, and replan failures surface through the existing non-fatal replanError path. - changeQuality rolls its optimistic preference back when the replan is refused or errors, so a failed switch is not silently applied by the next unrelated replan and the menu shows the real active rung. - The capability probe now tests mp3/vorbis codecs and mp3/flac/ogg containers (MediaSource with a canPlayType fallback), restoring direct play for mp3 audiobooks instead of per-part AAC re-encodes. - Reanchor seeks issued while a replan is in flight coalesce and run when it settles instead of being silently dropped with the scrubber pinned to a phantom position. - Subtitle refresh/translation replans use the resume anchor while the media element has no metadata, so a subtitle_ready broadcast during startup no longer restarts a resumed stream at 0:00. - An exhausted failure-recovery chain sets a visible error instead of returning silently. Part of #135. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): accept video-only and VP9 probe metadata Treat audio and video probe completeness independently so legitimate video-only assets converge without repeated ffprobe repair. Allow unknown codec profile/level metadata to fall through to server adaptation while preserving exact direct-decode constraints. Fixes #574 * fix(playback): address protocol v3 review findings * fix(playback): harden lease and probe repair decisions * fix(playback): close remaining v3 review gaps * fix(playback): recover failed transcode starts * fix(playback): address remaining review-bot findings on v3 replan and audio planning Server: - The deferred replan lease release is bounded by a 3s timeout so a saturated pool or DB outage cannot wedge a handler goroutine that holds the per-session store lock on an uncancellable context. - planAudioOnlyV3 honors the request bandwidth cap: an over-cap source skips the original_http direct route and converts to AAC with the same bandwidth_cap_applied warning and decision reason the video ladder uses. Unknown source bitrate never triggers the cap. - A copy-audio progressive plan rejected only by a per-delivery audio_decode_codecs subset retries as an AAC conversion instead of returning adaptation_unavailable, and the AAC recipe respects the delivery's max_channels. Web: - failure_recovery replans issued while another replan is in flight queue (superseding a pending seek reanchor) instead of being silently dropped with the fatal overlay already suppressed. - A terminal response to a fresh non-preserving start clears the previous plan and stops its session, so episode navigation cannot keep rendering the prior item under the new title. - A refused recovery replan for a transport-dead plan surfaces the error and re-arms the plan failure key, so transient recovery failures no longer strand an endless spinner; the audiobook player gets the same guard reset. - A track-less subtitle_translation_completed hands off to the refreshed persisted track once the inventory settles, clearing the live overlay, instead of pinning the synthetic live track forever. Part of #135. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): reuse HLS transport for sidecar replans * fix(playback): stabilize copy HLS remount timeline * fix(playback): address v3 review findings * fix(playback): satisfy player contract types --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
baa33768d6 |
fix(scanner): stop reporting an unusable ffprobe as an empty folder (#468)
* fix(scanner): stop reporting an unusable ffprobe as an empty folder parseAudiobookFolder and parsePodcastShow signalled "this folder holds no audio files" by wrapping os.ErrNotExist, and their reconcile callers skipped on that. exec also wraps fs.ErrNotExist when the configured ffprobe binary cannot be run, so a wrong playback.ffmpeg_path made every candidate folder look empty: the scan logged processed=N failed=0, indexed nothing, and gave the operator no clue why the library stayed empty. Introduce an errFolderHasNoMedia sentinel that deliberately does not wrap os.ErrNotExist, and skip on that instead. A folder that disappears between the scan walk and the parse still maps to the sentinel, so a mid-scan rename or delete stays a quiet skip rather than a scan failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scanner): bound the all-failed scan summary instead of joining every failure The sentinel change in this PR makes a previously-unreachable path reachable. Before it, a misconfigured ffprobe made every candidate folder look empty, so the scan skipped everything and `failed` stayed 0 — the `failedCount == processedCount` branch never fired. Now that an unusable ffprobe propagates as a real failure, that branch is the expected outcome of a first scan with a bad `playback.ffmpeg_path`, and it joins one wrapped error per failed folder. On the 240k-folder library the scan code is written for, `errors.Join` over that slice produces a ~64 MB error string (measured) that is written verbatim into `scan_runs.error_message` and republished over the Redis events channel and the admin SSE stream. The `failures` slice itself also grew unbounded for the whole scan even when the all-failed guard could not fire (any rescan with `skipped > 0`), holding hundreds of megabytes across a multi-hour scan before discarding it. Add a `scanFailures` collector that retains the first 20 failures and counts the rest, joining them with a trailing "and N more failures (elided)". The retained sample still names the cause, which is the entire purpose of the summary. The same 64 MB case now produces 5.4 KB. The ebook and manga scans have the identical shape and the same exposure via their own probe failures, so all four call sites share the collector rather than fixing the two audio paths alone. `failMu` now guards only `cancelErr` and is renamed `cancelMu` to match. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
8fc054c15d |
fix(scanner): never purge files under unreachable library roots (#372)
* fix(scanner): never purge files under unreachable library roots An unreachable root is not a removed root. When one root of a multi-root library dies (unmounted share, dead drive) while another root still has files, the whole-library empty-root guard does not fire — the surviving root produced files — so the scan marks everything under the dead root missing_since (desired: hides it from browse/playback) and then, with the default scanner.empty_trash_after_scan=true + 24h file_removal_grace, the next scan after the grace hard-deletes every row under the dead root. A week-long drive outage silently destroys the root's entire catalog state: probe data, intro/credits markers, file hashes. Worse, membership reconciliation immediately purges media_items whose only files lived on the dead root, cascading user collections (library_collection_items has ON DELETE CASCADE) and deleting cached artwork. This change makes "temporarily offline" survivable: - Probe each configured root at scan start (os.Stat + IsDir + ReadDir, factored into the new internal/rootcheck package and shared with the admin mount-check endpoint). Unreachable roots are skipped by the walk but their scopes still reconcile, so files are still marked missing. - The trash sweep (DeleteMissingByFolder) now excludes rows whose path sits under an unreachable root, using the same exact-path + escaped prefix-LIKE matching as ListIDsOutsideRoots (a sibling root that merely shares a string prefix is never protected). With all roots reachable the emitted SQL is unchanged. - Membership removal still happens — browse/home hide items via media_item_libraries, so removal is what keeps a dead-root-only title out of the catalog — but the orphan media_items purge exempts items whose files sit under an unreachable root. Their metadata, artwork, and collection links survive; when the root returns, the upsert clears missing_since and syncPresentLibraryState re-inserts the membership, restoring the item with zero re-probing or re-matching. - The folder surfaces scan_warning_code='dead_root' with a message naming the unreachable roots; a fully healthy scan or a successful mount check clears it, mirroring empty_root. The admin UI shows a badge and banner. - Deliberate deletion is untouched: removing a path from the library config still purges via ListIDsOutsideRoots, files under reachable roots keep the exact 24h-grace purge, the empty-root guard and the autoscan dead-mount guard are unchanged. The audiobook/podcast/ebook reconcile paths share the same folder-wide sweep and orphan purge, so they get the same guard. Covered by tests: an end-to-end two-root scan (root dies -> rows survive a zero-grace sweep and warning is set; root returns -> rows resurrect with their original ids and the warning clears; deleting a file under a reachable root still purges), repo-level sweep-protection and sibling-prefix tests, orphan-purge exemption, and rootcheck unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): probe uncompacted roots and take dead-root path on full outage Review follow-ups: (1) probe every configured path instead of the compacted traversal roots, so a nested child mount that dies under a reachable parent is still protected from the sweep; (2) when every configured root is unreachable, bypass the empty-root confirm flow (without consuming the one-time cleanup allowance), mark files missing, and raise dead_root instead of empty_root; (3) dead_root warning banner no longer shows empty-root confirm-deletion guidance as its fallback hint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(scanner): simplify dead-root protection plumbing - extract pathscope.CoverageClauses as the single builder for the exact-path + escaped prefix-LIKE root predicate; scanner's rootCoverageClauses delegates to it and catalog's excludeOrphansUnderProtectedPrefixes reuses it instead of hand-rolling the same clause loop - extract Scanner.sweepMissingAndReconcile to replace the identical trash-sweep + membership-reconcile + S3-image-cleanup block that was triplicated across the audiobook, ebook, and podcast scans (callers keep their flavor-specific log lines so messages stay constant) - add unreachableConfiguredRoots helper for the repeated probeUnreachableRoots(ctx, folder.ID, cleanScanRoots(folder.Paths)) expression in scanPaths and ScanFile - drop the unread Path field from rootcheck.Result - move the dead/empty-root warning text constants in AdminLibraries.tsx out of the middle of the import block Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): close dead-root protection gaps found in review Remediates the confirmed findings from the deep review of this PR: - Scoped audiobook scans (autoscan file events, subtree scans) ran the folder-wide sweep while probing only the scoped clone's Paths, so a healthy-subtree event could hard-delete a dead sibling root's rows. sweepMissingAndReconcile now reloads the folder's configured roots from the DB and probes them uncompacted, which also protects nested child mounts in the audiobook/ebook/podcast reconcilers. - A lost mount that leaves an empty, stat-able mountpoint probed as reachable and kept the historical purge timeline. A reachable root that is a literally empty directory while cataloged rows remain under it is now treated as suspect: rows are only marked missing, the sweep and orphan purge exempt it, dead_root is raised, and the mount-check endpoint reports it (additive suspect_empty field) instead of clearing the warning. Arming the one-time empty-cleanup allowance completes the deletion, including in the mixed case where other roots are healthy. Roots that still have directory entries keep the historical grace-then-purge path. - Confirmed empty cleanup (allow_empty_cleanup_once) no longer force-deletes rows under probe-dead roots: an outage is not a confirmation, so a dead sibling root's catalog survives a confirmed cleanout of a reachable empty root. - Root probes are now bounded (rootcheck.ProbeWithTimeout, 5s): a hung network mount degrades into the protected unreachable path with a probe_timeout error code instead of stalling every scan of the folder indefinitely. - Documented the cross-library limitation of the orphan-purge exemption next to the query it applies to. All behavior is pinned by new DB-backed tests (suspect-empty protection + confirmed completion, confirmed-cleanup dead-root survival, scoped/nested-root sweep protection, suspect-root query, probe timeout). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): address dead-root review findings --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ba2bac7f55 |
fix(scanner): reconcile deleted files promptly and honor file_removal_grace (#312)
* fix(scanner): reconcile deleted files promptly and honor file_removal_grace Upgraded/replaced media files left dead playable versions until the next full library scan: autoscan file-scope changes for deleted paths failed resolution (ClassifyPath stats the path) and were silently dropped, and scanner.file_removal_grace was dead config — empty_trash_after_scan purged missing-marked rows folder-wide immediately. - Add scantrigger.Resolver.ResolveVanishedPath: maps a vanished video file to a subtree scan of its parent dir, a vanished dir to itself, and an entry directly under a library root to a guarded library scan. Rejects still-existing paths and requires the matched library root to exist on disk so an unmounted share never triggers reconciliation. - Autoscan falls back to it for file-scope and legacy parent-dir changes that fail with a RequestError, closing the dead-version window from ~24h (daily full scan) to roughly one poll interval. - DeleteMissingByFolder now takes the removal grace and only deletes rows missing longer than it (default 24h, restart-required setting); missing rows are already hidden from every client surface, so the grace only preserves per-file state (probe/match/markers) in case the file returns. Remove the uncalled DeleteMissing sweeper. Part of #311 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): only treat ENOENT as a vanished path in ResolveVanishedPath Permission or other stat failures must not reconcile still-existing files as missing; reject them with a RequestError instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scanner): address PR 312 review nitpicks - Extract shared matchEnabledFolder/normalizeTrigger helpers so ResolveMissingSubtree and ResolveVanishedPath cannot drift. - Warn when a negative scanner.file_removal_grace is clamped to 0. - Add disabled-library test for ResolveVanishedPath. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
97ac2b4eed |
feat(audiobooks): audiobookshelf support — ABS conformance, perf, ebooks (#289)
* fix(ebooks): fold author hint into metadata search query
The ebook enricher loaded each item's author but buildEbookSearchQuery
dropped it, and metadata.SearchQuery had no field to carry it — so the
plugin only ever received the title. Title-only searches collide or miss,
leaving items without metadata or a cover.
Add SearchQuery.Author and fold it into the plugin search query text
(the SearchMetadataRequest contract carries a single free-text Query, so
no proto change is needed). Gated to callers that set Author (ebooks);
movie/TV search is unchanged.
Verified live against OpenLibrary/GoogleBooks: improves disambiguation on
clean titles. Note: messy filename-derived titles (series prefixes,
trailing "(… Book N)") still need title normalization, and a large tail
of niche/self-published ebooks is simply absent from the free sources —
neither is addressed here.
AI-use disclosure: authored with Claude Code.
(cherry picked from commit ba1265909c4fb87e1a8eab64b0b0c183aa95acc1)
* feat(scanner): extract MOBI/AZW/AZW3 metadata from EXTH headers
These formats previously had no parser — parseEbookFile returned only the
format string, so title fell back to the filename with no author and no
ISBN, leaving ~21k books unmatchable by the metadata enricher.
Parse the Palm Database container (PDB header → record 0 → PalmDOC +
MOBI header → EXTH block) and extract title, authors, ISBN, publisher,
and language. EXTH is located by its magic rather than the header flag,
and field offsets (encoding @12, full-name @0x44/0x48) were verified
against real .mobi/.azw3 files.
Verified live against real library files:
azw3 → title "The Sea", author "A H Lee"
mobi → title "Brotherband 3: The Hunters", author "John Flanagan",
ISBN 9781742750637
AI-use disclosure: authored with Claude Code.
(cherry picked from commit 7af194b711de97bc79855f08a9a4f9732c49db74)
* fix(ebooks): recover author from path and clean provider search title
- ebookAuthorFromPath: recover an author for ".../<Author>/<Title>/<Title> -
<Author>.ext" layouts when the file embeds none, gated on two agreeing
path signals (grandparent dir == filename suffix) so magazines/courses
never get a junk author; strip the suffix from a path-derived title.
- cleanEbookSearchTitle: normalize filesystem-mangled titles before search
(underscore->space, drop trailing " - <author>") to lift hit rate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 36a16cb58c3e5276aa4c0bdf8577008070f6abea)
* fix(scanner): gate path-author on person-name shape
ebookAuthorFromPath's grandparent==suffix corroboration also matched
inverted layouts ("<Title>/<Author>/<Author> - <Title>"), assigning the
title as the author. Require the candidate directory to look like a person
name (comma form, or all-capitalized tokens plus name particles) so series
and title folders ("De legenden van de Alfen") are rejected, and return the
canonical directory form for proper casing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit ff720bd268a23bff0e94c70f15cb7ecfb8efcb1f)
* fix(ebooks): strip series/book-number parentheticals from search title
cleanEbookSearchTitle now peels trailing "(... Book N)", "[#3]", "(2019)"
groups that don't belong in a provider title query, while leaving
meaningful parentheticals ("(Illustrated)") intact. Enrichment-side only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2273636c04fd9ef483003a9558972a2104fdd3a6)
* fix(ebooks): keep volume number in search title and dedup provider IDs
Two distinct ebooks (e.g. series volumes named only by series + book
number) were collapsing onto a single provider work, then fighting over
the same media_item_provider_ids row:
- cleanEbookSearchTitle stripped trailing "(... Book N)" / "[#3]" groups
entirely, so every volume of a series searched as the bare series name
and matched the same provider work. The plugin search contract carries
only a single free-text Query, so the volume number is now UNWRAPPED
into the query (brackets dropped, words kept) instead of discarded,
giving distinct volumes distinct searches. Bare-year groups are still
dropped (SearchQuery.Year carries them); meaningful parentheticals
("(Illustrated)") still survive.
- collectEbookMetadata now consults FindContentIDByProviderIDs before
accumulating a search-result provider ID. An ID already owned by a
different content item is skipped, so the loser is not mis-tagged with
the winner's metadata and ReplaceByContentID no longer violates the
(provider, provider_id, item_type) unique constraint. The previous
behavior logged duplicate-key errors every sweep and re-enriched the
failing item forever (CPU/RAM churn). A failed ownership check is
surfaced as a provider error so the item retries rather than terminally
stamping as "no match".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 942fdef6cb0e167b2d9223e9a968b3010b7b3ec8)
* fix(ebooks): address CodeRabbit review on PR #185
- cleanEbookSearchTitle: anchor author-suffix strip to a trailing match
(optionally followed by a series/volume parenthetical) so a mid-title
" - <token>" no longer truncates valid title text
- ebook scan: strip the recovered author suffix using normalized comparison
so case/spacing variants (e.g. "a. f. carter") don't leave a duplicate
- parseMOBIEXTH: bound parsing to the declared EXTH length so a corrupt
record count can't read full-text bytes as junk metadata
- add regression test for a non-trailing " - <token>" in the title
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 0f45af8143e04dfd5b4a5ee3e949dcd943eedbd1)
* fix(audiobooks): pass author in search query and retry on provider errors
Set SearchQuery.Author so the host adapter folds author into the
plugin free-text query (parity with ebooks). Track provider errors
during enrichment; when nothing matched and a provider errored, return
an error without stamping last_refreshed so the sweep retries instead
of terminally burning the item on a transient failure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit f45dd2104c5d6415324e80f91f6684bc39858459)
* fix(scanner): consolidate fragmented multi-file audiobook content_ids on rescan
audiobookFolderShouldSkip used ListByObservedRootPath which returns all
files for a root path regardless of content_id. When a multi-file audiobook
had files fragmented across multiple content_ids (e.g. from concurrent
refreshes), the file count matched disk so the skip check returned true
and the reconcile never ran to merge them.
Now verifies all DB files share the same content_id before skipping; any
fragmentation forces a full reconcile which consolidates to one content_id
via FindContentIDByRootPath → upsertAudiobookMediaFiles.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit e91c33e88a7d09e802e6afd8af246c6c954d0498)
* fix(ingest): skip concurrent match drainer for audiobook/podcast/ebook/manga libraries
The concurrent scoped match drainer ran during scan for all library types.
For audiobook libraries, the scanner assigns content_ids by folder root
(one item per multi-file folder). Running the drainer concurrently caused
it to process files with content_id=NULL (cleared by complete refresh)
as individual items, creating one media_item per file instead of one per
folder. This manifested as 41-file audiobooks fragmenting into dozens of
orphaned single-file content_ids on every refresh.
These library types use scanner-driven grouping; the post-scan drain step
handles them correctly. Returning nil matchScopes skips the concurrent
drainer entirely for these types.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit 93ae9d22ce315874fa22a958b88ca1766075695f)
* fix(abs): match real audiobookshelf auth + session-sync contract
Align the ABS-compat auth flow with real audiobookshelf (v2.26+) so
third-party clients (yaabsa, Plappa, native iOS) authenticate and sync
playback correctly:
- login/refresh: always emit user.accessToken; x-return-tokens gates
only the refresh token (body vs HttpOnly refresh_token cookie)
- /auth/refresh returns the full login envelope (was a thin token map)
- /me returns the full user object (toOldJSONForBrowser), shared with
login/authorize via a single absUserObject() builder
- /logout returns 200 {redirect_url:null} and clears the cookie (was 204)
- add POST /session/{sid}/sync (real ABS heartbeat path); it was
PATCH-only, so the official client's sync POST 404'd and playback
progress never synced
Verified against advplyr/audiobookshelf server/{Auth.js,models/User.js,
controllers,routers}. Unit tests updated/added; full abs suite green.
Not yet live-verified.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 336e932471d4be021d82299106a120611783836a)
* fix(abs): conform browse/list items to real audiobookshelf minified shape
Strict ABS clients (yaabsa, Plappa) crash or drop items when the browse
list shape only approximates real audiobookshelf. Match the serializers:
- add media.id + media.libraryItemId (= ContentID) to LibraryItemMedia;
yaabsa BookMedia.id is required non-null and was missing → the whole
item failed to parse ("Null is not a subtype of String")
- rebuild the minified list shape to LibraryItem.toOldJSONMinified +
Book.toOldJSONMinified + oldMetadataToJSONMinified key-for-key (ino,
path, isFile, numFiles/size, media.{id,tags,numTracks,numAudioFiles,
numChapters,size,ebookFormat}, flat author/series metadata)
- force media.numTracks/numAudioFiles >= 1 in the browse projection so
Plappa doesn't drop items reporting 0 audio files
- default /items list to minified (real ABS list is always minified);
minified=0 opts into the full shape
Verified against advplyr/audiobookshelf models/{Book,LibraryItem}.js.
Adds minified_test.go key-set conformance guards; abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6c9387a8c4b60be3dbe541049ed8c06344b10717)
* fix(abs): conform /items/{id} detail to real audiobookshelf expanded shape
Match real audiobookshelf LibraryItem.toOldJSONExpanded +
Book.toOldJSONExpanded + oldMetadataToJSONExpanded so strict clients
decode the item-detail page with the same model they use elsewhere:
- add expanded outer keys to LibraryItem (oldLibraryItemId, lastScan,
scanVersion, libraryFiles, size) and populate libraryFiles + summed
size from the item's media files in the detail builder
- add media.size (Book.toOldJSONExpanded)
- make the typed Metadata the full expanded superset: subtitle,
titleIgnorePrefix, authorName, authorNameLF, narratorName, seriesName,
descriptionPlain, publishedDate, asin, language, abridged; drop the
omitempty that previously dropped description/publishedYear/isbn/
publisher when empty (a missing key crashes strict clients)
Verified against advplyr/audiobookshelf models/Book.js + LibraryItem.js.
Adds items_detail_test.go expanded key-set guard; abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8bd485f0291e9db9f32e13a68609a68ee8a945ec)
* fix(abs): conform authors/series endpoints to real audiobookshelf shapes
Match the real audiobookshelf serializers so strict clients decode the
authors/series browse + detail responses:
- GET /libraries/{id}/authors now branches like LibraryController.getAuthors:
bare { authors: [...] } when not paginated, paged { results, total, ... }
only when limit+page are present (was always paged → clients keying on
`authors` got keyNotFound)
- author objects carry the full Author.toOldJSON key set (id, asin, name,
description, imagePath, libraryId, addedAt, updatedAt, numBooks); silo has
no analog for asin/description/imagePath/timestamps so they are null/0
- series objects carry the full Series.toOldJSON key set (adds
nameIgnorePrefix, description, libraryId, addedAt, updatedAt)
- series/author books are now FULL minified library items (real ABS shape)
instead of thin {id,media:{metadata:{title}}} stubs that crash Plappa;
author items moved to the real-ABS `libraryItems` key
Verified against advplyr/audiobookshelf controllers/LibraryController.js and
models/{Author,Series}.js. Tests updated + envelope-branch guard added; abs
suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8a22eb0900ed881d500ded315a508e1a07da14f3)
* fix(abs): add libraryId to collection/playlist objects (real ABS shape)
Real audiobookshelf Collection.toOldJSON and Playlist.toOldJSON both carry
a libraryId; silo's emitters omitted it, so a strict client modeling the
object with a required libraryId crashed. silo collections/playlists are
cross-library user-personal, so emit the virtual audiobook library id.
The books[]/items[] entries already carry the full LibraryItem shape and
inherit the browse-conformance fixes (media.id etc.). Envelopes were
already correct (paged for library-scoped, {collections}/{playlists} for
global).
Verified against advplyr/audiobookshelf models/{Collection,Playlist}.js.
Envelope key-set tests updated; abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit f7d2ff0565f05c3a6ef7f36f2b1f252bd373fa7a)
* fix(abs): conform library object + /libraries/{id} to real audiobookshelf
The library object was only {id,name,mediaType}; real audiobookshelf
Library.toOldJSON has 12 keys, so a strict client decoding the library
model crashed on the missing ones. Also GET /libraries/{id} always wrapped
the object in { library: ... }, but real ABS returns it directly unless
?include=filterdata is requested.
- audiobookLibraryMap now emits the full Library.toOldJSON shape (folders[]
as LibraryFolder.toOldJSON, displayOrder, icon, provider, settings,
lastScan, lastScanVersion, createdAt, lastUpdate). This also enriches the
libraries[] on the login envelope, which shares the builder.
- handleLibraryDetail returns the library object DIRECTLY without include,
and wraps in { filterdata, issues, numUserPlaylists,
customMetadataProviders, library } (adds the missing
customMetadataProviders) with include=filterdata.
GET /libraries already returned { libraries: [...] } (correct). Verified
against advplyr/audiobookshelf models/Library.js +
controllers/LibraryController.js. Adds libraries_shape_test.go; abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit d05a2f2af1caf21a4ad04577a4787ba02cff091c)
* fix(abs): conform personalized recent-series shelf to real ABS series shape
The /libraries/{id}/personalized "Recent Series" shelf emitted thin
{id,name,numBooks,libraryId,books:[]} entities with an always-empty cover
stack. Emit the full real-ABS series object (seriesObjectABS, adds
nameIgnorePrefix/description/addedAt/updatedAt) with minified book items
(seriesBookMinified) — the same shape as /libraries/{id}/series so the
shelf card decodes identically and shows real covers.
Book shelves already used full minified items; the shelves array is a bare
array (matches real ABS getUserPersonalizedShelves). abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7c586f8c923cbc481f6f94e304af48dee379a999)
* fix(abs): conform listening-sessions to real audiobookshelf PlaybackSession shape
silo's /me/listening-sessions returned a thin 5-field session object
(id, libraryItemId, userId, timeListening, currentTime) wrapped in the
generic pagedEnvelope shape ({results,sortBy,filterBy,minified}). Real
audiobookshelf clients (Flutter/Swift strict decoders) expect the
MeController.getListeningSessions envelope
({total,numPages,page,itemsPerPage,sessions}) and each session to carry
the full PlaybackSession.toJSON() key set, so the missing keys (notably
mediaType, mediaMetadata, displayTitle, displayAuthor, coverPath,
duration, chapters, deviceInfo, playMethod, mediaPlayer, serverVersion,
date, dayOfWeek, startTime, startedAt, updatedAt, libraryId, bookId,
episodeId) crashed with keyNotFound errors.
Both handleListeningSessions and handleListeningSessionDetail now build
the response via a shared sessionToABS() that reuses
buildSiloPlayMediaMetadata (already used by /play) to hydrate
mediaMetadata/displayTitle/displayAuthor from MediaStore, batching
lookups via GetAudiobooksByIDs for the list endpoint. Lookups are
best-effort: a missing/inaccessible item falls back to a stub
MediaItem so every key is still emitted, never a crash.
Verified against advplyr/audiobookshelf server/controllers/MeController.js
(getListeningSessions) and server/objects/PlaybackSession.js (toJSON())
on GitHub master.
Known placeholders (real ABS fields we can't populate without extra
cost): chapters (empty array — would require a per-session media-files
fetch), duration (0 — total book duration isn't tracked on the session
row), startTime (0 — not persisted separately from currentTime),
deviceInfo (static "unknown" device, matching the /play endpoint's
existing placeholder — no device info is persisted per session).
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9471497c99b96c8d3defc6c8112c913f7c55924b)
* feat(abs): add offline session sync endpoints (/session/local, /session/local-all)
The official ABS mobile app records playback while offline and POSTs those
PlaybackSession objects back on reconnect via SessionController.syncLocal and
syncLocalSessions. silo was missing both endpoints, so offline listening
progress was silently lost. Add them to the bearerAuth-protected session group
(both /abs/api and /api prefixes) alongside /session/{sid}/sync and /close.
POST /session/local decodes one PlaybackSession and updates the caller's resume
position via ProgressStore.UpdateProgressPosition (the same call handleSessionSync
uses), emitting user_item_progress_updated. POST /session/local-all decodes
{sessions:[...]} and loops each robustly — a malformed or unknown item marks that
one result failed without sinking the batch — returning {results:[...]}. No new
store persistence or migration; podcast/episode sessions are accepted as no-ops.
Verified against advplyr/audiobookshelf server/controllers/SessionController.js
and server/managers/PlaybackSessionManager.js.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 008a4df948d855a4bfe62b24f89bfc484f088033)
* fix(abs): conform library search + items-in-progress to real audiobookshelf
Real ABS's libraryItemsBookFilters.search() (delegated from
LibraryController.search) returns { book, narrators, tags, genres,
series, authors } with no "podcast" key for a book library, and each
book entry is only { libraryItem } — no matchKey/matchText, which our
handler was inventing. Search now matches those keys, drops the
fabricated matchKey/matchText fields, and best-effort populates
authors/series buckets via client-side substring filtering over the
existing aggregate listers (narrators/tags/genres stay empty-but-present
since silo has no backing aggregation query for them yet).
MeController.getAllLibraryItemsInProgress wraps items as
{ ...libraryItem.toOldJSONMinified(), progressLastUpdate }; our handler
was emitting a hand-rolled subset of fields plus a nested
userMediaProgress object that doesn't exist in the real response.
items-in-progress now reuses the existing Minify() projection and merges
a flat progressLastUpdate (ms) field to match.
Verified against advplyr/audiobookshelf controllers/{Library,Me}Controller.js
and server/utils/queries/{libraryItemsBookFilters,authorFilters}.js.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 998ff55f3cf27d504f0e5aec8c7c29fa57f10247)
* fix(abs): /ping returns success:true and /status carries authMethods
The ABS apps validate a server address by reading response.success from
GET /ping; silo returned {pong:true,...} with no `success`, so the app
reported "unable to reach" even though the server responded 200. Also
/status was missing authMethods/authFormData, which the app reads to render
the login form.
- /ping now includes {"success": true} (pong/server/version kept as extras)
- /status now returns {app,serverVersion,isInit,language,authMethods,
authFormData} matching real audiobookshelf Server.js
Verified against advplyr/audiobookshelf server/Server.js. Adds
ping_status_test.go; abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit df732d09355303608bc4d5fc2138555e37497a02)
* fix(abs): mount login + auth/refresh under /api prefix
Clients that post to /api/login (and /api/auth/refresh) got a 404 because
login/refresh were only mounted at root and /abs/api — while the rest of the
authenticated ABS surface (/api/me, /api/authorize, /api/libraries, covers)
is served under both /api and /abs/api. The 404 surfaced in the client as a
generic "unknown error occurred" on sign-in.
Mount /login and /auth/refresh under all three prefixes ("", /api, /abs/api),
matching the authenticated groups.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 18071e180b02131cda354303eadd1ff3a0708065)
* fix(abs): accept form-encoded login bodies (not just JSON)
Real audiobookshelf (express body-parser + passport local) accepts both
application/json and application/x-www-form-urlencoded credential bodies.
Silo only json-decoded the body, so a form-encoded client got 400 "invalid
request body" — surfaced in the app as a generic "unknown error" on sign-in
(confirmed live: JSON creds -> 200, identical form-encoded creds -> 400).
Buffer the body once, try JSON, then fall back to url.ParseQuery for the
form-encoded case.
Adds login_body_test.go (form + JSON both reach the validator). abs suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 408dc33debb7a7ee363778094511ccfdd1ee70d2)
* fix(abs): emit full real-ABS serverSettings (OpenID/auth fields)
silo's login/authorize serverSettings omitted the auth + OpenID fields that
real audiobookshelf ServerSettings.toJSONForBrowser includes
(authLoginCustomMessage, authOpenID*, rateLimitLogin*, backupPath,
allowedOrigins). OIDC-aware strict clients (Prologue, iOS/Swift) decode
serverSettings into a model that requires those keys, so their absence throws
keyNotFound and the ENTIRE login response fails to decode — the client stays
on the login screen with a generic "unknown error" even though the server
returned 200. Simpler clients that don't model OpenID were unaffected.
Emit real ABS's OIDC-disabled defaults; authActiveAuthMethods still advertises
only "local" so no client initiates the OpenID flow.
Diagnosed from a packet capture (Prologue posts /login? with X-Return-Tokens
and gets a 200 it can't decode) + real ABS ServerSettings.js. Verified against
advplyr/audiobookshelf.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 283826c952824e97233e46e057f37385ddc3054b)
* fix(abs): GET /me returns the real display username, not the userID
/me built its user object from the token claims and passed the numeric
userID as the username, so clients saw "98" instead of "puksthepirate".
Login gets the display name from the credential validator, but /me only has
the token, so it needs a lookup.
Add an optional UsernameResolver to the abs Dependencies; wire it from the
concrete SiloCredValidator (which holds the pgx pool) via a new
ResolveUsername method that mirrors Validate's display-name logic — the
profile name when a profile is set and named, else the account username.
handleMe uses it and falls back to the userID when unresolved.
abs package compiles + tests pass; the audiobooks package (service.go,
cred_validator.go) could not be linked locally (pre-existing bimg/libvips
pkg-config gap) and is validated at the Docker build.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 39ff3e3350aad087757f7fb0e94d5a7f10c08ae5)
* fix(abs): always emit AudioTrack keys + correct media.duration
Two item-detail issues that made Prologue report "Unable to load book
contents" (can't press Start Listening):
- AudioTrack used omitempty on chapters/metaTags/format/bitRate/codec/
metadata/etc, so empty values dropped those keys. Real ABS AudioFile/
AudioTrack always emit them; strict clients (Prologue, yaabsa) decode
tracks into a required-field model and throw keyNotFound on the missing
keys, failing the whole track decode. Removed omitempty and emit
chapters/metaTags as [] / {} (non-nil) in both track builders.
- media.duration used the item's Runtime, which is often stale/mis-scanned
(e.g. 222s for a 3.7h book) and desyncs the player scrubber. Now sum the
track durations (real ABS: sum of audio file durations), falling back to
Runtime only when there are no tracks.
Verified against advplyr/audiobookshelf models/Book.js (AudioFile/AudioTrack)
via a live packet capture of Prologue's item-detail decode failure. abs
suite green.
AI-use: implemented with Claude Code (Opus 4.8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8210ed2fc63e782f158b4168ef693e671ae19638)
* perf(abs): push down library browse filters + author counts MV
The ABS audiobook library-serving path was slow on large libraries
(~255k items): /libraries/{id}/items?filter=authors.{id} loaded and
hydrated the whole library into Go before filtering (~4.8s each), and
/libraries/{id}/authors ran a full GroupAggregate + COUNT(DISTINCT)
per page (~53s full sync) — slow enough to trip ABS client sync
timeouts (e.g. Prologue).
- Push author/series/narrator/no-series filters into indexed SQL
EXISTS predicates in ListAudiobooks; paginate + COUNT in SQL.
Semantically equivalent to the prior Go-side filter (kind=7 author,
kind=8 narrator, exact-case match, no-series sentinel).
- Add covering index media_items(content_id, type) so the count/list
type check runs index-only (CONCURRENTLY, NO TRANSACTION — no
write-lock on the live table).
- Serve /authors from a materialized view (abs_audiobook_author_counts)
refreshed every 15min, with a live-query fallback when the view is
empty/unrefreshed so the endpoint never blanks on a fresh deploy.
Conformance preserved: keeps authorObjectABS/seriesObjectABS shapes and
the limit&&page envelope decision; adds a regression test for the
bare {authors:[...]} envelope on limit-only requests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 0d55754051dd3ad016b3cec6a0921307071d3219)
* perf(abs): index-back audiobook search via trigram GIN
SearchAudiobooks matched the raw media_items.title with ILIKE '%q%'
OR'd with an author/narrator EXISTS. The un-indexed raw-title column
plus the OR forced a full seq scan of the ~255k-item library on every
search (~560ms on library 18).
Reshape into a UNION of two index-driven arms that reuse the search
infrastructure the rest of the catalog already relies on: the title arm
matches media_items.title_normalized (idx_media_items_title_normalized_trgm)
via the shared normalize_search_text(), the people arm matches people.name
(idx_people_name_trgm). GROUP BY content_id keeps the best rank when an
item matches both; a normalize_search_text($2) <> '' guard stops a
punctuation-only query from degenerating into ILIKE '%%'.
No new index or migration — the trigram indexes already existed and were
simply unused. ~560ms -> ~35ms, both indexes engaged, no seq scan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 98bbd1712719ecd03e4db87f840774cec788f177)
* perf(abs): index-ordered item paging + cached library count
The unfiltered /libraries/{id}/items path that ABS clients page through
to sync a library recomputed COUNT(*) over the whole library on every
page (~150ms each) and ordered by LOWER(sort_title), LOWER(title) — an
expression matching no index, forcing a full in-memory sort of all
~255k rows per page (~324ms shallow, ~543ms deep). A full sync is
thousands of pages, so both costs dominated indexing time.
- Order by lower(coalesce(nullif(btrim(sort_title),''), title)),
content_id so the page is served by an ordered index scan on the
existing idx_media_items_sort_key (~324ms -> ~1ms). content_id (PK)
is a stable tiebreaker, making sequential pagination deterministic —
the prior ordering could skip/repeat rows when sort keys collided.
- Memoize the per-page COUNT in a 60s TTL cache keyed on the fully
rendered count SQL + bound args, so it covers every input the WHERE
depends on (library, pushed-down filter, all access predicates) and
can't drift as access logic evolves. Expired entries swept on write.
No new index or migration — reuses idx_media_items_sort_key.
total may lag up to 60s during an active scan; clients re-sync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 32e26c2f99a1ffc071f600071c2ea7ddcd3397b4)
* fix(abs): address PR review — access-aware authors, offline progress create, cookie refresh, body limits
- media_store: ListLibraryAuthors bypassed per-item access when reading the
author materialized view (keyed by library only), leaking authors of books
hidden by a content-rating cap or excluded media types. Take the access-aware
live path whenever the filter carries an item-level predicate.
- session_local: offline sync used UPDATE-only UpdateProgressPosition, so a book
listened to entirely offline (no progress row yet) had its position silently
dropped while still reporting progressSynced. Create the row via UpsertProgress
when none exists; keep the monotonic update path for existing rows.
- login: handleRefresh never read the refresh_token cookie, so cookie-flow ABS
clients got 400 refreshToken required once the access token expired. Read the
cookie as a third source after header and body.
- session_local: cap /session/local and /session/local-all request bodies at
1 MiB via io.LimitReader, matching the rest of the package.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Quick
|
20e0dd7fe2 |
Fix Jellyfin parent browsing and release dedupe
- List seasons or episodes correctly for Jellyfin parent item queries - Deduplicate episode availability by logical episode identity - Improve audiobook title fallbacks and existing item updates |
||
|
|
9b111649f3 |
perf+fix(audiobooks): detail page, browse, sessions & scanner (#169)
* perf(catalog): fix audiobook detail N+1 + slow people facets Audiobook detail pages were slow in proportion to track count (up to 433 files/book). Root causes, found by EXPLAIN ANALYZE on the live DB: - effectiveAudioSelection ran 3-4 user-store queries (profile, audio pref, library pref) per file inside buildPlaybackInfo's loop, though the results are invariant across a request. Introduce a request-scoped audioPrefResolver that memoizes the store lookups (library prefs keyed by folder); a 400-file audiobook now issues each query once instead of per file. Selection logic is unchanged (audioPreference returns a copy so the original-language sentinel is still resolved per file). - buildAudiobookExtension ran its four independent related-content queries serially; run them concurrently so latency is the slowest, not the sum. - author/narrator browse facets did a full people-table scan; add a (kind, content_id, person_id) index so the facet resolves from an index-only scan of just that kind's credits (~112ms -> ~49ms on the live library). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(catalog): cache audiobook author/narrator group browse The Authors/Narrators audiobook pages were slow on cold load and slow again after a hard refresh (fast only while the React Query client cache was warm). Root cause (EXPLAIN ANALYZE on live, 31K-audiobook library): the grouped browse query is ~234ms/page, there are ~13K distinct authors, and the client pages through the entire list on every load (sequential 500-row requests). With no server-side cache each of the ~20 pages re-ran the full aggregation (COUNT(*) OVER() forces it), so a cold load was ~20x234ms. The client's 60s staleTime was the only thing making a warm revisit fast; a refresh wiped it. Fix: AudiobookGroupsCache caches the full sorted group list per (library, group_by, sort, viewer) for 60s (matching the client staleTime, so no extra staleness) and serves every page as an in-memory slice — one aggregation per window instead of one per page, and a refresh is a cache hit. Also raise the client page size 500->2000 so fewer sequential round-trips are needed now that a larger page is a cheap slice. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(settings): throttle per-request device last_seen upserts Device-setting reads (HandleGetDeviceSetting, HandleGetEffectiveSettings, HandleGetEffectiveSubtitleAppearance) each registered the request's device — an INSERT ... ON CONFLICT upsert of last_seen_at on a single per-device row. A page that fetches many settings fired hundreds of these concurrently; they serialized on that row's lock (observed 100-237ms each, ~250 per page load in the slow query log), taxing every settings fetch. Throttle device registration to one upsert per (profile, device) per 5 minutes via an in-process TTL cache, marking the device seen before the upsert so a concurrent burst collapses to a single write. last_seen_at stays fresh to within the window. Reads no longer issue a contended write on the hot path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf+fix(audiobooks): probe-repair, resume position, cache storm, groups reveal, hot-row + stats index From the full audiobook code review (EXPLAIN + slow-query trace on live): - #1 (P0, detail-page killer): NeedsCriticalProbeRepair required video codec/ resolution/tracks, which audio-only files never have, so PlaybackProbeEnsurer re-ran ffprobe per file on every detail/watch load (up to N serial spawns for an N-track book) and never converged. Gate video-field checks on the file actually having a video stream. TDD. - #3 (P0): abs session-sync rewound the resume cursor — UpdateProgressPosition did an unconditional SET with no monotonic guard, ignored its error, and no-op'd when no row existed (first-listen resume lost). Now a finish-preserving GREATEST upsert; caller logs failures. - #4 (P0 perf): progress reports fired every ~10s invalidated all of catalogKeys.all → refetched every active browse/detail query incl the 13k audiobook group lists. Scope invalidation to the reported item's detail. - #6 (P1 perf): Authors/Narrators page rendered all ~13k groups + cover images at once (main-thread freeze). Incremental reveal: render a capped window, grow on scroll via IntersectionObserver. - #10: throttle abs TouchToken last_seen upsert (one per token per 5min) — same hot-row contention class as the device fix. - #8: index abs_playback_sessions (user_id, profile_id, started_at) for the listening-stats aggregations. Deferred (need contract/validation): listening-time idempotency (client delta-vs- cumulative), scanner deleted-file reconcile, abs session retention job, abs list- handler batch fetch, scanner-output P2s (need re-backfill). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(audiobooks): batch-fetch abs list/shelf handlers (kill N+1) handleSimilarItems, handleItemsInProgress, and handleGetMyProgress called MediaStore.GetAudiobookByID once per row — up to ~500 single fetches (each a few queries) on app open. Add GetAudiobooksByIDs (one access-scoped fetch + people/series hydrated once for the whole set) and look results up from the returned map, preserving order. Underlying primitives (GetByIDsWithAccess, hydratePeople/Series) were already batch-capable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): reconcile deleted files on scan + prune session history #5: ScanAudiobookFolder only ever upserted — deleted/renamed books leaked media_items/media_files/memberships forever. Mirror the ebook reconcile: collect seenPaths during the walk, MarkMissing files no longer on disk, then reconcileLibraryMemberships. Safety mirrors ebooks/video: an inaccessible root (unmounted source) is skipped entirely, and a walk that saw zero files while the DB has rows only reconciles after operator cleanup confirmation (ebookEmptyCleanupAllowed) — so a flapping mount can't wipe the catalog. Soft mark only; the existing grace-period purge hard-deletes later. Reconcile runs only on a fully-completed (non-cancelled) scan. (#9 coarse case already handled: audiobookFolderShouldSkip skips unchanged folders; per-file reuse deferred.) #8-retention: abs_playback_sessions grew unbounded (one row per play-start, never deleted) and fed every listening-stats scan. Add an hourly sweep in SessionCleaner: close abandoned open sessions (no /close, stopped syncing >24h) and delete closed sessions older than 90 days. Mirrors the recommendation_cache / missing-files prune pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): address max-effort code-review findings From /code-review max on the pre-PR diff: - DATA RACE (P0): SessionCleaner.lastABSSessionPrune is read+written by both the 15s ticker goroutine and the shutdown-path CleanStale call (main.go defers Stop() to after that call). Guard the prune-due gate with a mutex. (CleanStale was stateless before this branch, so concurrent calls were previously safe.) - ScanAudiobookFolder hardcoded fullScan=true into the empty-walk cleanup guard, but it's also called from ScanSubtree (incremental scans). An empty subtree scan would wrongly consume the operator's one-shot empty-cleanup allowance and warn. Thread a real fullScan flag (true from ScanFolder, false from the two subtree call sites), mirroring the ebook path. - Revert UpdateProgressPosition to UPDATE-only (drop the INSERT-on-missing): keep the monotonic GREATEST + finish guard that fixes the resume rewind, but restore the no-op-on-missing contract so a stray sync tick can't resurrect just-cleared progress or create a zero-duration continue-listening row. - Clamp the audiobook-groups handler limit (paging moved into the cache, leaving the old 500/page bound stranded); also gofmt the Scanner struct. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(audiobooks): address review feedback for scanner and stats * fix(audiobooks): address review feedback * fix(audiobooks): retry failed session prune --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
e084cdd1d6 |
Add unified literary works for ebooks and audiobooks (#107)
* docs: add literary works design and plan * feat(literary): add work link schema * feat(literary): add work domain primitives * feat(literary): persist work links * feat(literary): score work matches * feat(catalog): include literary work summary on item detail * feat(literary): expose work detail API * feat(literary): assemble work detail * feat(literary): add admin work linking primitives * feat(catalog): group literary items by work * feat(literary): auto-link works during book scans * fix(literary): narrow work match candidates * fix(literary): address work merge blockers --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
bf54f040f9 |
fix(audiobooks): extract scan covers and harden dedupe (#103)
* fix(audiobooks): extract scan covers and harden dedupe * fix(audiobooks): address scan cover review --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
5855cebe75 |
fix(audiobooks): complete playback and catalog parity (#96)
* fix(audiobooks): complete playback and catalog parity * fix(web): allow podcast continue targets * fix(audiobooks): use folder sidecar covers during scan * fix(audiobooks): read nullable poster paths during cover scan * feat(home): split continue listening sections * fix(catalog): coalesce nullable media artwork fields * feat(playback): surface audiobook sessions in admin activity * fix(audiobooks): address playback review feedback --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
eb6024573e |
feat(audiobooks): make audiobook libraries first-class catalog items (#73)
* docs(audiobooks): design spec for plugin absorption Plan to absorb silo-plugin-audiobooks into silo-server as a first-party feature. Audiobooks land in silo's existing SPA; ABS clients connect directly. Hard constraints: reuse existing tables (media_items, media_files, user_watch_progress, user_playback_sessions, people, item_people, library_collections); only two new tables (abs_sessions, podcast_feeds) and at most one column add (media_libraries.kind); silo's main :8080 listener handles ABS Socket.io natively. Out of scope: audiobook requests flow, smart collections, share links, external recommender, custom metadata providers, separate audiobook SPA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 1 (discovery + schema) First of six sub-plans for the absorption. Six tasks: a discovery audit that resolves the spec's Risk questions, four idempotent SQL migrations (abs_sessions, podcast_feeds, media_libraries.kind, audiobooks.enabled feature flag), and an empty-but-compiling internal/audiobooks package scaffolded into cmd/silo. Lands as a strict no-op for users (feature flag defaults to false). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): discovery findings for absorption sub-plan 1 Locks schema/code decisions for migrations 139-142 and downstream sub-plans. Resolves open Risk questions from the absorption design spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 139 add abs_sessions table Parallel of jellycompat_sessions for Audiobookshelf-compatible clients. Lets ABS mobile/desktop apps maintain a device-bound session that silo's audiobooks/abs handlers will validate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): match codebase conventions in migration 139 Lowercases type keywords in the abs_sessions CREATE TABLE body to match neighboring migrations, fixes the client_version column alignment, and replaces the misleading "parallel to jellycompat_sessions" header comment with a more accurate description of the table's role. Cosmetic only — the running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 140 add podcast_feeds table Side table on media_items for RSS-subscribed podcasts. Holds feed URL, ETag/Last-Modified for conditional fetches, last-refresh timestamp, and the per-feed refresh interval consumed by the upcoming podcastfeed.Refresher scheduled task. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): uppercase PRIMARY KEY in migration 140 Aligns with the codebase convention (type keywords lowercase, constraint keywords uppercase) established in migration 139's post-style-fix form. Cosmetic only — running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(audiobooks): migration 141 no-op for media_folders.type Sub-plan 1 originally reserved migration 141 to add a 'kind' column to media_libraries discriminating audiobook/podcast libraries. Discovery audit (sub-plan 1 Task 1) found that the actual table is media_folders and it already has a type text NOT NULL column with no CHECK constraint or enum, so 'audiobooks' and 'podcasts' can be added as future values without DDL. Landing this migration as a documented no-op preserves the version numbering audit trail and pins the decision in git history. The matching down migration is also a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 142 add audiobooks.enabled flag Server-settings row that gates the absorbed audiobooks feature. Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent sub-plans branch on this flag and operators flip it to 'true' at cutover. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scaffold internal/audiobooks package Empty-but-compiling Service that reads the audiobooks.enabled feature flag from server_settings. Wired into cmd/silo so the package is referenced from the binary; no routes mounted, no scheduled tasks registered, no DB writes. Subsequent sub-plans hang scanner branches, ABS handlers, Socket.io, podcast refresher, and SPA pages off this Service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): cosmetic cleanups in scaffolded package Two pre-emptive cleanups flagged by code review before sub-plan 2 copies the patterns: 1. Sort the internal/audiobooks import after internal/adminjob in cmd/silo/main.go (alphabetical). 2. Drop the redundant "audiobooks: " prefix from the Enabled() error wrap; matches how every other top-level service package (watchstate, scanqueue, metadata, etc.) formats errors. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 2 (scanner) Second of six sub-plans. 10 tasks: PersonKind constants for Author and Narrator, audio-extension recognizer, library-type helpers, a walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter extraction via ffprobe, single-file and multi-file audiobook parsers, scanner write path producing media_items.type='audiobook', and a filesystem podcast parser (RSS deferred to sub-plan 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add Author and Narrator PersonKind constants Discovery audit confirmed item_people.kind is unconstrained smallint with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook people-links written by the upcoming scanner branches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add audio-extension recognizer for scanner Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by upcoming audiobook and podcast scanner branches to filter directory walks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library-type recognizers for scanner dispatch isAudiobookLibraryType and isPodcastLibraryType match singular and plural forms case-insensitively, mirroring isMovieLibraryType. Used by upcoming scanner walk branches (Task 4) that filter audio files into audiobook and podcast libraries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(scanner): replace movieLibrary bool with typed walkMode Lets walkLogicalTree dispatch on multiple library shapes (video, movie, audiobook, podcast) without proliferating boolean flags. Behavior for existing video and movie libraries is unchanged; audiobook and podcast modes will be consumed by the upcoming audiobook.go and podcast.go parsers in later tasks of this sub-plan. walkModeFor() derives the mode from a media_folders.type string; unknown types default to walkModeVideo to preserve prior behavior for any caller still passing a raw type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose ffprobe format tags on ProbeData The audiobook scanner needs format-level tags (title, artist, album, date) for media_items metadata; ffprobe already parses them in ffprobeFormat.Tags but ProbeData previously discarded them. Add FormatTags map[string]string to ProbeData, populate it in convertProbeData via a new normalizeFormatTags helper that lowercases keys and trims values. Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and format tags, and a test that verifies ProbeFile() returns both correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): parser for single-file audiobook folders parseAudiobookFolder reads tags + chapters via the existing ProbeFile (now that Task 5 exposes FormatTags on ProbeData) and produces a parsedAudiobook struct. Title falls back from "title" tag to "album"; author from "artist" -> "album_artist" -> "composer"; series from "album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools). Year parsed from "date" or "year" tags, tolerating ISO dates and parenthesized forms. Single-file case only; multi-file folders (one audio file per chapter) return a placeholder error and arrive in Task 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): multi-file audiobook folder support Folders containing N audio files (one per chapter/part) get one parsedAudiobookFile per file; each file's chapter list is synthesized as a single chapter with title = filename stem. Title/author/series/ year come from the first file's tags. Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in favor of the existing firstNonEmpty already in probe.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scanner write path produces audiobook media_items ScanAudiobookFolder walks an audiobooks-typed media folder and treats each immediate subdirectory as one audiobook. For each parsed audiobook it upserts: - one media_items row with type='audiobook' - one media_files row per audio file (with chapters JSONB) - author/narrator links in item_people (kind=7, kind=8) Adds itemRepo and personRepo to the Scanner struct, wired from fileRepo.Pool() in NewScanner — no constructor signature change needed. ScanFolder dispatches to this path when folder.Type='audiobooks', bypassing the per-file movie/TV pipeline because audiobooks are folder-scoped entities. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): filesystem podcast scanner ScanPodcastFolder walks a podcasts-typed media folder, treating each subdirectory as a podcast show and each audio file inside as an episode. Writes media_items.type='podcast' + episodes rows + media_files rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5; this task covers filesystem-only ingestion. ScanFolder dispatches to this path when folder.Type='podcasts'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 5 (podcasts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose audiobooks/podcasts library types in admin UI Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown in the admin libraries page so operators can flag a folder as an audiobook or podcast library. Extends contentLevelsForType() so the admin UI's downstream filtering treats those types correctly (audiobook -> ['audiobook'], podcasts -> ['podcast', 'podcast_episode']). Backend scanner branches for these types were already wired in sub-plan 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge origin/main adds 139_media_requests at the same number our local audiobook branch had used for abs_sessions. Renumber ours to 147 to free up 139 for the upstream migration. The schema_versions row is updated in lockstep on the running database so the migrator sees the abs_sessions migration as already applied at its new version. Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook feature flag, abs playback sessions, podcast episode guid, audiobook series, audiobook title cleanup) stay where they are — they don't collide with anything on origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge origin/main added 140_user_permissions at the same version this branch had used for podcast_feeds. Renumber ours to 157 (next free above the collections-unify migration at 156) so 140 is free for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees podcast_feeds as already applied at its new version. Same pattern as |