Files
silo-server/internal/scanner/audiobook_scan.go
881c96864b feat(playback): finalize platform-neutral protocol v3 (#567)
* docs(playback): add v3 neutral-contract finalization plan

Supersedes the wire-contract sections of the 2026-07-12 v3 plan: server-owned
attempt keys, delivery-keyed negotiation without Media3 engine names, tiered
capability evidence, neutral device/output context, track/quality replan
operations, audio-only planning, and coordinated no-back-compat rollout
across server, Android, Apple, and web.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): make v3 attempt keys server-owned and replace engines with deliveries

Contract core of the platform-neutral v3 finalization (plan sections 3.1
and 3.2), breaking on purpose — v3 is dark and all clients move together:

- Every PlanV3 now carries plan_attempt_key, an opaque server-computed
  token clients store and echo in attempted_plan_keys; ReplanRequestV3
  gains bounded local_mutations that the replan handler folds into the
  failed plan's key. Clients never hash anything.
- KotlinName() is deleted from DeliveryV3, StreamProtocolV3 and
  SubtitleModeV3; the attempt-key canonical string now uses lowercase
  wire tokens, and PlanRecipeVersionV3 bumps to v3.3 so no key or plan
  ID computed under the old canonicalization can collide.
- EngineV3 leaves the wire: ClientPlaybackContextV3.Engines (media3_*)
  becomes Deliveries keyed original_http|progressive|hls, with
  EngineCapabilityV3 renamed DeliveryCapabilityV3. PlanV3.Engine is
  removed; the planner, subtitle policy and quirk registry re-key on
  delivery class, and the media3_only feature token is deleted.
- Validated-claim strings drop the prefix: media3_h264_decode ->
  h264_decode, media3_audio_decode -> audio_decode.
- Golden fixtures in testdata/protocol_v3 are regenerated by Go and are
  now the cross-repo source of truth.

Part of the playback protocol v3 neutral-contract train (steps 2-3 of
docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): add v3 evidence tiers and neutral device/output context

Implement plan sections 3.3 and 3.4 of the v3 neutral-contract pass:

- ClientCodecCapabilitiesV3 gains required video_evidence and
  audio_evidence closed enums (exact | platform_attested | declared).
  Planner strictness follows the tier: exact keeps the strict decode-entry
  validation, platform_attested validates codec/resolution/bit-depth/
  frame-rate but skips profile/level matching, declared grants copy routes
  from the flat codec lists. Only exact audio evidence earns passthrough
  claims. The detailed_decode_capabilities feature token is deleted
  (subsumed by video_evidence=exact), and evidence-blocked direct routes
  carry the new evidence_insufficient_for_direct reason/warning.

- DeviceContextV3 is now platform/os_version/manufacturer/model plus a
  bounded platform_details map (<=16 entries, <=128 chars); the Android
  Build dump fields are gone. Fire TV quirks keep matching on
  manufacturer/model (brand fallback removed with the field).

- output_route_generation (int64, dual-location) becomes an optional
  opaque output_context_id string on the output context; the dual-location
  consistency validation is deleted. Attempt keys, plan invalidation,
  route events, and the planstore column follow (new Goose migration).

- Feature advertisement collapses to the top-level client_features list
  only; ClientPlaybackContextV3.Features is deleted and ReplanRequestV3
  gains an optional client_features refresh.

- PlanRecipeVersionV3 bumped v3.3 -> v3.4; fixtures re-keyed.

Part of the playback protocol v3 neutral-contract finalization plan
(docs/superpowers/plans/2026-07-30-playback-protocol-v3-neutral-contract.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(playback): add v3 intent replans, quality menu, and audio-only routes

Protocol v3 could only replan after a failure, so changing the audio track
or the quality still required the legacy audio PATCH and the client-recipe
transcode start — the two endpoints v3 is meant to replace. Clients also had
to own a resolution ladder to render a quality menu, and a source with no
video track was terminaled by the video/HDR gates, keeping audiobooks on the
legacy path.

Add track_change and quality_change replan operations. They carry no failure
classification and route through the existing replan transaction, so they
inherit its idempotency, capacity reservation, and staged-successor commit
for free. Because nothing failed, the previous route stays eligible: neither
the attempted-key history nor the failed-plan exclusion applies to them.

Publish the server ladder on the plan as available_qualities so the quality
menu is server-owned; the rungs come from the same resolutionLabelV3 and
ladderBitrateKbpsV3 helpers the planner itself uses, not a parallel table.

Plan audio-only sources through their own reduced route family: original_http
when the client decodes the codec, otherwise a progressive AAC conversion.
The plan advertises audio/mp4 for that remux and the transport now serves the
same value, because a declared-tier client probes the advertised MIME with
isTypeSupported before attaching a source buffer, and "video/mp4" on a stream
with no video track is exactly the mismatch that makes the probe lie.

Name the protocol's string vocabulary (dynamic ranges, transformations,
executors, validated claims, terminal reasons) as constants while touching
these lines, so the wire values have one definition.

Part of #135

* docs(playback): publish the v3 protocol contract and fix subtitle ordinals

Protocol v3 exists only as Go code today, so the Android and Apple ports have
no authority to implement against other than reading this repository. Publish
the contract as a normative document, machine-checkable schemas, and generated
golden fixtures, and fix the one place where the server's own wire output
disagreed with the ordinal space it publishes.

- docs/architecture/playback-protocol-v3.md is self-contained enough for a
  third-party client: endpoints and status codes, evidence tiers and their
  bound-matching rules, delivery classes, the timeline model, replan
  semantics, registries, track identity, plan identity, quality, and
  transformations.
- docs/design/schemas/playback-v3/ carries JSON Schemas for the five wire
  shapes plus valid and invalid fixtures, following the client-diagnostics
  layout. internal/playback/contract validates every fixture against its
  schema, so a schema that drifts from the Go types fails the Go suite.
- cmd/playbackfixtures generates internal/playback/testdata/protocol_v3 from
  the production planner. `make playback-fixtures` writes them and
  `make verify-playback-fixtures` (wired into CI) fails when they are stale.
  These files are what the client ports consume, so drift would otherwise
  surface as a playback bug on three platforms at once.

The subtitle fix: combined ordinals are one dense space over externals, then
embedded tracks, then downloaded ones, but the legacy URL builder skipped
burn-in-only tracks while assigning indices, so every track after a DVD/DVB
track was numbered one too low and resolved to its neighbour. Ordinal
assignment now lives in playback.BuildSubtitleInventoryV3 and both the plan
inventory and the legacy `subtitle_urls` shape project from it; the legacy
shape still filters burn-in-only entries but keeps each track's real index.

Part of #135

* feat(web): migrate the players to the neutral playback v3 contract

The web player was the last client still speaking the legacy start
protocol: it picked its own file version from a codec probe, posted an
ffmpeg recipe to start a transcode, PATCHed an endpoint to change audio
tracks, and derived its own quality ladder. None of that survives a
server-owned plan, and none of it produced telemetry the apps could be
compared against.

Video player: starts with a v3 request that advertises `declared`
evidence from `isTypeSupported` probes and the three delivery classes,
then consumes the returned plan for its URL, timeline, tracks and
warnings. Quality and track changes become replans (`quality_change`,
`track_change`), the quality menu renders `available_qualities` instead
of computing rungs, and playback failures emit `route-events` so web
failures land in the same diagnostics as Android and Apple. The
duration comes from `source.duration_seconds` rather than the playback
engine, and the "how was this delivered" overlay reads the plan's
delivery and server transformations instead of comparing codec strings.

Audiobook player: starts against the audio-only planner path with a
single `original` rung, and takes its seek anchor from
`timeline.player_start_seconds` so the progressive-remux route (which
anchors the stream and restarts the player clock at zero) does not seek
twice.

Server side, `disable_progress_persistence` left the wire, so the rule
it encoded is now derived. Resume state is keyed on the item, but every
part of a multipart presentation shares that key while carrying its own
file-local clock — persisting part 4's position would store "12 minutes
in" as the book's resume point. `PresentationPartTotal > 1` expresses
that directly and generalizes to multipart movies and split episodes,
and a client can no longer forget to ask or lie about it.

`useTranscodeQuality` and the legacy response types are deleted, and
`WEBTEST_KNOWN_FAILURES` loses the audiobook entry along with its fix.

Part of #135

* feat(playback)!: make v3 the only playback protocol

Protocol v3 shipped behind a flag, alongside the legacy start path it was
designed to replace. Running both meant every planner change had to be made
twice, in two shapes that disagree about who decides the route: the legacy
body carried a decision the client had already made, while v3 asks the server
to make it. This deletes the legacy half.

Removed:

- `handleStartPlaybackLegacy` and its request/response bodies. The
  `POST /playback/start` route stays, but the protocol-version dispatch
  envelope is now a strict v3 decode — a body that does not declare
  `protocol_version: 3` gets `426 client_upgrade_required` so an outdated app
  can render a clear "update required" state instead of misreading a plan.
  Deliberately not a `400`: the request may be well-formed for the protocol it
  was written against.
- `POST /playback/transcode/start`, superseded by the `quality_change` replan
  operation, and `PATCH /playback/{session_id}/audio`, superseded by
  `track_change`. Both mutated a session without re-planning.
- The shadow planner and both rollout settings rows. With v3 the only
  protocol, `playback.protocol_v3_enabled` would mean "no playback at all";
  `playback.protocol_v3_shadow_enabled` gated a comparison against a path that
  no longer exists. `409 protocol_disabled` on route-events goes with them, and
  capability `enabled` is now constant `true` (the field stays — clients
  feature-detect against it).
- Version-selection helpers in `internal/playback/resolver.go` that only legacy
  start reached. `Resolve`/`ClientCapabilities`/`PlayDecision` stay: downloads
  consumes them. `internal/jellycompat` has its own resolution surface and is
  untouched.

Behaviour the legacy handlers owned and v3 now owns explicitly: series version
and audio-track preferences are persisted on start and on a `track_change`
replan (not on failure recovery, whose forced route is not a user choice); an
omitted `start_position` resolves to the profile's saved resume point; and an
omitted audio track resolves through the series preference, the profile audio
language, then the library override. Both are settled before planning, because
the plan's timeline is cut at the start position. Spec §2.2 documents this as
"omission is a request, not a default".

The encode-target clamp that lived in the deleted transcode handler is already
enforced in the planner, twice — `availableQualitiesV3` omits rungs at or above
the source height, and the encode path clamps `targetHeight` to it.

Unchanged: progress, stop, HLS manifest and segment delivery, the realtime
control socket, stream tokens and restart reconstruction, watch together,
downloads, jellycompat.

Every removal is recorded in the pre-lock removals table in
docs/architecture/v1-scope.md.

Part of #135

* fix(scanner): stop recording embedded cover art as a video track

ffprobe reports embedded cover art as a video stream carrying
disposition.attached_pic. convertProbeData appended every "video" stream
to VideoTracks without consulting isMainVideoStream, the predicate that
already existed for duration decisions, so the picture was persisted as a
playable track. That misreports the file twice:

  - An audio file with a cover picks up a video track, so it no longer
    satisfies MediaFile.IsAudioOnly and the v3 planner routes an
    audiobook through the video path instead of planAudioOnlyV3.
  - When the picture is ordered ahead of the real stream, the flat
    codec_video/resolution/hdr columns describe the poster: a 954x720
    h264 episode was stored as mjpeg 480x480.

Filter attached_pic streams out of the track loop. The guard is the
disposition flag, not the codec name, so a genuine MJPEG video is still
probed as video — the library has one.

Already-probed rows self-heal on the next playback: NeedsCriticalProbeRepair
already reprobes tracks missing color_range, which covers 21 of the 23
affected rows, and applyProbeData overwrites VideoTracks wholesale. The
remaining two need a rescan; nothing persisted records attached_pic, and
keying repair off still-image codec names would reprobe the genuine MJPEG
file on every playback forever.

Part of the playback v3 neutral-contract work: it is what lets Android
drop AUDIOBOOK_COVER_ART_CODECS, which fabricated decode support the
client cannot honestly claim under video_evidence: "exact".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(playback): publish subtitle URLs even when playback starts with subtitles off

The v3 plan's subtitle inventory is the authoritative track list a client builds
its subtitle menu from, but the handler only rewrote it with session-scoped URLs
when a track was actually selected. A start or replan that resolved to
`subtitle.mode: "off"` therefore returned the planner's URL-less inventory, so a
client whose picker reads the inventory had a menu it could not fetch anything
from. The Cast path hits this every time: it starts with subtitles off and needs
the receiver's text tracks up front.

attachSubtitleArtifactV3 now scopes and publishes the inventory unconditionally
and gates only the artifact stamping on the selection. Spec §8 records that the
`url` on a sidecar entry does not depend on the current selection.

Part of the v3 neutral-contract finalization.

* chore(playback): reconcile neutral v3 with main

* fix(playback): preserve subtitle intent across replans

* fix(playback): retain subtitle inventory on adapted routes

* fix(playback): software-decode High10 AVC for QSV

* fix(playback): scale High10 frames before QSV upload

* fix(playback): preserve empty subtitle inventories

* fix(playback): freeze terminal attempt contract

* chore(playback): name fixture contract tokens

* fix(playback): close v3 conformance review gaps

* chore(playback): name conformance category

* fix(playback): complete v3 conformance contract

* fix(playback): keep schema fixtures generated

* fix(playback): emit schema-valid conformance arrays

* fix(playback): omit empty replan failures

* fix(web): omit empty replan failures

* fix(playback): close neutral v3 contract gaps

* fix(playback): harden v3 replan, transcode, and quality-ladder edge cases

Review remediation for the neutral v3 cutover, server side:

- A failed replan no longer overwrites the durable StartResponse with a
  terminal or advances the replan request ID; an idempotent start replay
  of a still-healthy session returns the original plan.
- SoftwareVideoDecode is now derived inside the transcode layer from
  source facts (codec/profile/bit depth) carried on TranscodeOpts, so
  jellycompat, downloads, recipe-card reconstruction, and transcode
  nodes get the High10 software-decode fix, not just the v3 handler.
  video_to_h264 recipe version bumps to 2 so mixed-version node pools
  that would silently drop the flag fail validation instead.
- Local transport startup shares the 30s ManifestStartupTimeout; a
  timeout with the process still running stays retryable and is no
  longer persisted as a durable terminal against the attempt.
- Sparse replan bodies (failure_recovery et al) no longer reset a
  user-selected quality preference to auto; the empty-value guard now
  covers every operation.
- availableQualitiesV3 publishes no fixed rungs when the source height
  is unknown, keeping the no-upscaling ladder contract.
- The proxy remux path serves audio-only fMP4 as audio/mp4 via a new
  additive AudioOnly token claim, matching the integrated path.
- Plain text subtitle sidecars accept any requested extension again
  (served as VTT), restoring the permissive v1 behavior; ASS and bitmap
  handling is unchanged.
- The 4K-disallowed terminal message discloses when a lower-resolution
  alternate exists but was pinned away by quality "original".

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(web): keep playback alive through failed replans and honest audio claims

Review remediation for the neutral v3 cutover, web player:

- A failed or refused replan no longer unmounts the player: the fatal
  error screen is reserved for loads with no adopted plan, and replan
  failures surface through the existing non-fatal replanError path.
- changeQuality rolls its optimistic preference back when the replan is
  refused or errors, so a failed switch is not silently applied by the
  next unrelated replan and the menu shows the real active rung.
- The capability probe now tests mp3/vorbis codecs and mp3/flac/ogg
  containers (MediaSource with a canPlayType fallback), restoring
  direct play for mp3 audiobooks instead of per-part AAC re-encodes.
- Reanchor seeks issued while a replan is in flight coalesce and run
  when it settles instead of being silently dropped with the scrubber
  pinned to a phantom position.
- Subtitle refresh/translation replans use the resume anchor while the
  media element has no metadata, so a subtitle_ready broadcast during
  startup no longer restarts a resumed stream at 0:00.
- An exhausted failure-recovery chain sets a visible error instead of
  returning silently.

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): accept video-only and VP9 probe metadata

Treat audio and video probe completeness independently so legitimate video-only assets converge without repeated ffprobe repair. Allow unknown codec profile/level metadata to fall through to server adaptation while preserving exact direct-decode constraints.

Fixes #574

* fix(playback): address protocol v3 review findings

* fix(playback): harden lease and probe repair decisions

* fix(playback): close remaining v3 review gaps

* fix(playback): recover failed transcode starts

* fix(playback): address remaining review-bot findings on v3 replan and audio planning

Server:
- The deferred replan lease release is bounded by a 3s timeout so a
  saturated pool or DB outage cannot wedge a handler goroutine that
  holds the per-session store lock on an uncancellable context.
- planAudioOnlyV3 honors the request bandwidth cap: an over-cap source
  skips the original_http direct route and converts to AAC with the
  same bandwidth_cap_applied warning and decision reason the video
  ladder uses. Unknown source bitrate never triggers the cap.
- A copy-audio progressive plan rejected only by a per-delivery
  audio_decode_codecs subset retries as an AAC conversion instead of
  returning adaptation_unavailable, and the AAC recipe respects the
  delivery's max_channels.

Web:
- failure_recovery replans issued while another replan is in flight
  queue (superseding a pending seek reanchor) instead of being
  silently dropped with the fatal overlay already suppressed.
- A terminal response to a fresh non-preserving start clears the
  previous plan and stops its session, so episode navigation cannot
  keep rendering the prior item under the new title.
- A refused recovery replan for a transport-dead plan surfaces the
  error and re-arms the plan failure key, so transient recovery
  failures no longer strand an endless spinner; the audiobook player
  gets the same guard reset.
- A track-less subtitle_translation_completed hands off to the
  refreshed persisted track once the inventory settles, clearing the
  live overlay, instead of pinning the synthetic live track forever.

Part of #135.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playback): reuse HLS transport for sidecar replans

* fix(playback): stabilize copy HLS remount timeline

* fix(playback): address v3 review findings

* fix(playback): satisfy player contract types

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 18:14:49 -04:00

1225 lines
37 KiB
Go

package scanner
import (
"context"
"errors"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/Silo-Server/silo-server/internal/idgen"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/titleutil"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type filesystemRootContentFinder interface {
FindContentIDByRootPath(ctx context.Context, folderID int, rootPath, preferredType string) (string, error)
}
type filesystemMediaItemWriter interface {
Upsert(ctx context.Context, item *models.MediaItem) error
}
type audiobookPosterExec interface {
Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error)
}
type audiobookPosterPathReader interface {
GetPosterPath(ctx context.Context, contentID string) (string, error)
}
const audiobookDuplicateCandidateSQL = `
SELECT mi.content_id, mi.title
FROM media_items mi
JOIN item_people ipa ON ipa.content_id = mi.content_id AND ipa.kind = 7
JOIN people pa ON pa.id = ipa.person_id AND LOWER(pa.name) = LOWER($1)
JOIN item_people ipn ON ipn.content_id = mi.content_id AND ipn.kind = 8
JOIN people pn ON pn.id = ipn.person_id AND LOWER(pn.name) = LOWER($2)
JOIN LATERAL (
SELECT COALESCE(SUM(mf.duration), 0) AS dur
FROM media_files mf WHERE mf.content_id = mi.content_id
) f ON TRUE
JOIN LATERAL (
SELECT btrim(regexp_replace(
regexp_replace(lower(mi.title), '[^a-z0-9]+', ' ', 'g'),
'[[:space:]]+', ' ', 'g'
)) AS normalized_title
) t ON TRUE
WHERE mi.type = 'audiobook'
AND mi.year = $3
AND f.dur > 0
AND ABS(f.dur - $4) <= $5
AND (
t.normalized_title = $6
OR t.normalized_title LIKE $6 || ' %'
OR $6 LIKE t.normalized_title || ' %'
)
ORDER BY ABS(f.dur - $4), LENGTH(mi.title), mi.content_id
LIMIT 25
`
// audiobookDiskFile is the on-disk projection used by audiobookFolderUnchanged.
// Path is the absolute file path; Size and ModTime come from os.Stat.
type audiobookDiskFile struct {
Path string
Size int64
ModTime time.Time
}
// audiobookFolderUnchanged reports whether the audio files on disk match the
// existing media_files rows for the same folder one-for-one on path, size,
// and mtime. A new file, removed file, or any byte-level / mtime drift returns
// false so the caller falls through to a full reconcile.
//
// Comparison uses sameFileModifiedAt for mtime to absorb sub-second precision
// differences between filesystem reads.
func audiobookFolderUnchanged(existing []*models.MediaFile, onDisk []audiobookDiskFile) bool {
if len(existing) != len(onDisk) {
return false
}
byPath := make(map[string]*models.MediaFile, len(existing))
for _, mf := range existing {
if mf == nil || mf.HasLegacyAttachedPictureVideo() {
return false
}
byPath[mf.FilePath] = mf
}
for _, d := range onDisk {
mf, ok := byPath[d.Path]
if !ok {
return false
}
if mf.FileSize != d.Size {
return false
}
if mf.FileModifiedAt == nil || !sameFileModifiedAt(mf.FileModifiedAt, d.ModTime) {
return false
}
}
return true
}
// audiobookFolderShouldSkip returns true when every audio file on disk in
// folderPath matches an existing media_files row by size + mtime AND the
// linked media_items row is in a non-unmatched status. False under any
// drift, missing row, or unmatched status — the caller then falls through
// to the full reconcile path.
//
// Errors are returned but the worker loop treats them as "do not skip".
func (s *Scanner) audiobookFolderShouldSkip(ctx context.Context, folder *models.MediaFolder, folderPath string) (string, bool, error) {
if s.fileRepo == nil || s.itemRepo == nil {
return "", false, nil
}
entries, err := os.ReadDir(folderPath)
if err != nil {
return "", false, fmt.Errorf("read folder: %w", err)
}
var onDisk []audiobookDiskFile
for _, e := range entries {
if e.IsDir() || !SupportsAudioFile(e.Name()) {
continue
}
full := filepath.Join(folderPath, e.Name())
info, statErr := os.Stat(full)
if statErr != nil {
return "", false, fmt.Errorf("stat %s: %w", full, statErr)
}
onDisk = append(onDisk, audiobookDiskFile{
Path: full,
Size: info.Size(),
ModTime: normalizeFileModifiedAt(info.ModTime()),
})
}
if len(onDisk) == 0 {
return "", false, nil
}
existing, err := s.fileRepo.ListByObservedRootPath(ctx, folder.ID, folderPath)
if err != nil {
return "", false, fmt.Errorf("list existing files: %w", err)
}
if !audiobookFolderUnchanged(existing, onDisk) {
return "", false, nil
}
contentID := existing[0].ContentID
if contentID == "" {
return "", false, nil
}
// All files must share the same content_id; fragmented folders need reconcile.
for _, mf := range existing[1:] {
if mf.ContentID != contentID {
return "", false, nil
}
}
items, err := s.itemRepo.GetByIDs(ctx, []string{contentID})
if err != nil {
return "", false, fmt.Errorf("get item for skip check: %w", err)
}
if len(items) == 0 || items[0] == nil {
return "", false, nil
}
if strings.TrimSpace(items[0].Title) == "" {
return "", false, nil
}
if strings.EqualFold(strings.TrimSpace(items[0].Status), "unmatched") {
return "", false, nil
}
return contentID, true, nil
}
// audiobookScanWorkers returns the configured number of parallel workers
// for audiobook reconciliation. Defaults to 8 — high enough to keep all
// cores busy on the ffprobe step (which dominates per-book wall time)
// without overwhelming a small server. Override with SILO_AUDIOBOOK_SCAN_WORKERS.
func audiobookScanWorkers() int {
if v := os.Getenv("SILO_AUDIOBOOK_SCAN_WORKERS"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
return n
}
}
return 8
}
type audiobookRootScan struct {
root string
candidates []string
seenPaths map[string]bool
rootErr error
walkFailures int
}
func (r *audiobookRootScan) failed() bool {
return r.rootErr != nil || r.walkFailures > 0
}
func collectAudiobookRootScans(ctx context.Context, folderID int, roots []string) ([]audiobookRootScan, error) {
scans := make([]audiobookRootScan, 0, len(roots))
for _, root := range roots {
if err := ctx.Err(); err != nil {
return nil, err
}
cleanRoot := filepath.Clean(strings.TrimSpace(root))
if cleanRoot == "" || cleanRoot == "." {
continue
}
scan := audiobookRootScan{
root: cleanRoot,
seenPaths: make(map[string]bool),
}
info, statErr := os.Stat(cleanRoot)
switch {
case statErr != nil:
scan.rootErr = fmt.Errorf("stat root: %w", statErr)
case !info.IsDir():
scan.rootErr = fmt.Errorf("root is not a directory after symlink resolution")
}
if statErr == nil && scan.rootErr == nil {
walkErr := filepath.WalkDir(cleanRoot, func(path string, d fs.DirEntry, walkErr error) error {
if err := ctx.Err(); err != nil {
return err
}
if walkErr != nil {
scan.walkFailures++
slog.WarnContext(ctx, "audiobook scan: walk error", "component", "scanner", "path", path, "error", walkErr)
return nil
}
if !d.IsDir() {
return nil
}
entries, err := os.ReadDir(path)
if err != nil {
scan.walkFailures++
slog.WarnContext(ctx, "audiobook scan: read dir failed", "component", "scanner", "path", path, "error", err)
return nil
}
hadAudio := false
for _, e := range entries {
if !e.IsDir() && SupportsAudioFile(e.Name()) {
scan.seenPaths[filepath.Join(path, e.Name())] = true
hadAudio = true
}
}
if hadAudio {
scan.candidates = append(scan.candidates, path)
return filepath.SkipDir
}
return nil
})
if walkErr != nil {
if errors.Is(walkErr, context.Canceled) || errors.Is(walkErr, context.DeadlineExceeded) {
return nil, walkErr
}
scan.rootErr = fmt.Errorf("walk root: %w", walkErr)
}
}
if scan.failed() {
slog.WarnContext(ctx, "audiobook scan: root walk incomplete; root excluded from missing-file reconciliation", "component", "scanner",
"folder_id", folderID,
"root", cleanRoot,
"walk_failures", scan.walkFailures,
"error", scan.rootErr,
)
}
scans = append(scans, scan)
}
return scans, nil
}
func splitAudiobookReconcileRoots(scans []audiobookRootScan) (roots []string, seenPaths map[string]bool, sawFiles bool) {
roots = make([]string, 0, len(scans))
seenPaths = make(map[string]bool)
for i := range scans {
scan := &scans[i]
if scan.failed() {
continue
}
roots = append(roots, scan.root)
if len(scan.seenPaths) > 0 {
sawFiles = true
for path := range scan.seenPaths {
seenPaths[path] = true
}
}
}
return roots, seenPaths, sawFiles
}
// ScanAudiobookFolder walks an audiobooks-typed media folder and writes
// one media_items row per subdirectory it can parse as an audiobook,
// plus the corresponding media_files rows and author/narrator links in
// item_people.
//
// Each immediate subdirectory of one of folder.Paths is treated as a
// single audiobook. Subdirectories that contain zero audio files are
// silently skipped (parseAudiobookFolder returns errFolderHasNoMedia).
//
// This bypasses the per-file movie/TV pipeline because audiobooks are
// inherently folder-scoped (one book = one item, possibly multi-file).
func (s *Scanner) ScanAudiobookFolder(ctx context.Context, folder *models.MediaFolder, fullScan bool) error {
if s == nil || folder == nil {
return fmt.Errorf("ScanAudiobookFolder: nil scanner or folder")
}
// Phase 1: walk the tree to collect candidate book folders. This is
// I/O-light (no ffprobe), and avoids holding the worker pool open
// for the duration of a 240k-folder scan.
scans, err := collectAudiobookRootScans(ctx, folder.ID, folder.Paths)
if err != nil {
return err
}
candidates := make([]string, 0)
for i := range scans {
candidates = append(candidates, scans[i].candidates...)
}
reconcileRoots, seenPaths, _ := splitAudiobookReconcileRoots(scans)
if len(candidates) == 0 {
// No books on disk. Still reconcile (guarded) so a legitimately-emptied
// library converges — but only when a root was readable, and the
// empty-walk guard requires operator confirmation before deleting.
if len(reconcileRoots) > 0 {
if err := s.reconcileAudiobookMissingFiles(ctx, folder, reconcileRoots, seenPaths, fullScan); err != nil {
slog.WarnContext(ctx, "audiobook scan: missing-file reconcile failed", "component", "scanner", "folder_id", folder.ID, "error", err)
}
} else if len(scans) > 0 {
slog.WarnContext(ctx, "audiobook scan: every root walk failed; skipping missing-file reconciliation", "component", "scanner",
"folder_id", folder.ID)
}
return nil
}
// Phase 2: reconcile in parallel. ffprobe dominates per-book wall time
// (hundreds of ms each), so single-threaded scans of a large library
// take days. Worker pool brings this to ~hours.
workers := audiobookScanWorkers()
slog.InfoContext(ctx, "audiobook scan: starting", "component", "scanner",
"folder_id", folder.ID,
"candidates", len(candidates),
"workers", workers,
)
ch := make(chan string, workers*2)
var (
wg sync.WaitGroup
processed int64
failed int64
skipped int64
cancelMu sync.Mutex
failures scanFailures
cancelErr error
)
start := time.Now()
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for path := range ch {
if ctx.Err() != nil {
return
}
if err := s.reconcileAudiobookFolder(ctx, folder, path, &skipped); err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
cancelMu.Lock()
if cancelErr == nil {
cancelErr = err
}
cancelMu.Unlock()
return
}
atomic.AddInt64(&failed, 1)
failures.addf("%s: %w", path, err)
slog.WarnContext(ctx, "audiobook scan: folder failed", "component", "scanner",
"folder_id", folder.ID,
"path", path,
"error", err,
)
}
n := atomic.AddInt64(&processed, 1)
if n%500 == 0 {
slog.InfoContext(ctx, "audiobook scan: progress", "component", "scanner",
"folder_id", folder.ID,
"processed", n,
"failed", atomic.LoadInt64(&failed),
"skipped", atomic.LoadInt64(&skipped),
"total", len(candidates),
"elapsed_sec", int(time.Since(start).Seconds()),
)
}
}
}()
}
for _, p := range candidates {
if ctx.Err() != nil {
break
}
ch <- p
}
close(ch)
wg.Wait()
if err := ctx.Err(); err != nil {
return err
}
if cancelErr != nil {
return cancelErr
}
slog.InfoContext(ctx, "audiobook scan: completed", "component", "scanner",
"folder_id", folder.ID,
"processed", atomic.LoadInt64(&processed),
"failed", atomic.LoadInt64(&failed),
"skipped", atomic.LoadInt64(&skipped),
"elapsed_sec", int(time.Since(start).Seconds()),
)
if processedCount := atomic.LoadInt64(&processed); processedCount > 0 {
failedCount := atomic.LoadInt64(&failed)
skippedCount := atomic.LoadInt64(&skipped)
if failedCount > 0 && skippedCount == 0 && failedCount == processedCount {
return fmt.Errorf("audiobook scan failed for every attempted folder_id=%d: %w", folder.ID, failures.join())
}
}
// Reconcile files that vanished from disk now that the full walk's
// seenPaths is known and the scan completed without cancellation.
if err := s.reconcileAudiobookMissingFiles(ctx, folder, reconcileRoots, seenPaths, fullScan); err != nil {
slog.WarnContext(ctx, "audiobook scan: missing-file reconcile failed", "component", "scanner", "folder_id", folder.ID, "error", err)
}
return nil
}
// reconcileAudiobookMissingFiles mirrors reconcileMissingEbookFiles: DB files
// under the scanned roots that were not seen on disk are marked missing, the
// folder trash is optionally emptied, and library memberships are reconciled so
// books with no remaining files are removed (a rename therefore converges on
// the newly indexed path instead of leaving a stale duplicate). A scan that saw
// zero files while the DB still has rows only reconciles when the operator has
// confirmed cleanup, so an unmounted source can't wipe the catalog.
func (s *Scanner) reconcileAudiobookMissingFiles(ctx context.Context, folder *models.MediaFolder, roots []string, seenPaths map[string]bool, fullScan bool) error {
if s.fileRepo == nil || s.libraryRepo == nil || len(roots) == 0 {
return nil
}
confirmedCleanup, blockAll, err := s.emptyCleanupDecision(
ctx, folder, roots, seenPaths, fullScan, "audiobook",
)
if err != nil {
return err
}
if blockAll {
return nil
}
now := time.Now().UTC()
missing := 0
for _, root := range roots {
existing, err := s.fileRepo.GetByFolderAndPathPrefix(ctx, folder.ID, root)
if err != nil {
return fmt.Errorf("listing existing audiobook files for %q: %w", root, err)
}
for _, mf := range existing {
if mf == nil || seenPaths[mf.FilePath] {
continue
}
if mf.MissingSince == nil {
if err := s.fileRepo.MarkMissing(ctx, mf.ID, now); err != nil {
slog.ErrorContext(ctx, "audiobook scan: failed to mark file missing", "component", "scanner",
"folder_id", folder.ID, "path", mf.FilePath, "error", err)
continue
}
}
missing++
}
}
trashed, removedMemberships, deletedItems, err := s.sweepMissingAndReconcile(ctx, folder, confirmedCleanup)
if trashed > 0 {
slog.InfoContext(ctx, "audiobook scan: emptied trash", "component", "scanner", "folder_id", folder.ID, "deleted", trashed)
}
if err != nil {
return err
}
if missing > 0 || removedMemberships > 0 || deletedItems > 0 {
slog.InfoContext(ctx, "audiobook scan: reconciled missing files", "component", "scanner",
"folder_id", folder.ID, "missing", missing,
"memberships_removed", removedMemberships, "items_deleted", deletedItems)
}
return nil
}
func (s *Scanner) reconcileAudiobookFolder(ctx context.Context, folder *models.MediaFolder, folderPath string, skipped *int64) error {
existingContentID, isUnchanged, skipErr := s.audiobookFolderShouldSkip(ctx, folder, folderPath)
if skipErr != nil {
slog.WarnContext(ctx, "audiobook scan: skip-check failed, falling through", "component", "scanner",
"folder_id", folder.ID,
"path", folderPath,
"error", skipErr,
)
} else if isUnchanged {
s.autoLinkLiteraryWork(ctx, existingContentID)
atomic.AddInt64(skipped, 1)
return nil
}
parsed, err := parseAudiobookFolder(ctx, s.ffprobePath, folderPath)
if err != nil {
if errors.Is(err, errFolderHasNoMedia) {
return nil
}
return fmt.Errorf("parse audiobook folder %s: %w", folderPath, err)
}
contentID, err := s.upsertAudiobookMediaItem(ctx, folder.ID, folderPath, parsed)
if err != nil {
return fmt.Errorf("upsert audiobook item: %w", err)
}
if err := s.upsertAudiobookMediaFiles(ctx, folder, contentID, folderPath, parsed); err != nil {
return fmt.Errorf("upsert audiobook files: %w", err)
}
if err := applyAudiobookSidecarCover(ctx, s.itemRepo, s.imageCacher, contentID, folderPath); err != nil {
slog.WarnContext(ctx, "audiobook scan: sidecar cover upload failed", "component", "scanner",
"folder_id", folder.ID,
"content_id", contentID,
"path", folderPath,
"error", err,
)
}
if len(parsed.Files) > 0 && s.fileRepo != nil {
if _, err := applyAudiobookEmbeddedCover(ctx, s.itemRepo, s.fileRepo.Pool(), FFmpegPathFromFFprobe(s.ffprobePath), s.imageCacher, parsed.Files[0].Path, contentID); err != nil {
slog.WarnContext(ctx, "audiobook scan: embedded cover failed", "component", "scanner",
"folder_id", folder.ID,
"content_id", contentID,
"path", parsed.Files[0].Path,
"error", err,
)
}
}
if err := s.upsertAudiobookPeople(ctx, contentID, parsed); err != nil {
return fmt.Errorf("upsert audiobook people: %w", err)
}
if err := s.upsertAudiobookSeries(ctx, contentID, parsed); err != nil {
return fmt.Errorf("upsert audiobook series: %w", err)
}
if _, err := s.fileRepo.Pool().Exec(ctx, `
INSERT INTO media_item_libraries (content_id, media_folder_id, first_seen_at)
VALUES ($1, $2, NOW())
ON CONFLICT (content_id, media_folder_id) DO NOTHING
`, contentID, folder.ID); err != nil {
return fmt.Errorf("upsert audiobook library membership: %w", err)
}
if parsed.ASIN != "" {
// Two unique constraints can conflict: (content_id, provider) is the
// PK, and (provider, provider_id, item_type) prevents two content
// rows from claiming the same external ID. We want to silently skip
// when either fires; ON CONFLICT DO NOTHING (no target) catches both.
if _, err := s.fileRepo.Pool().Exec(ctx, `
INSERT INTO media_item_provider_ids (content_id, provider, provider_id, item_type)
VALUES ($1, 'asin', $2, 'audiobook')
ON CONFLICT DO NOTHING
`, contentID, parsed.ASIN); err != nil {
return fmt.Errorf("upsert audiobook ASIN provider id: %w", err)
}
}
s.autoLinkLiteraryWork(ctx, contentID)
slog.InfoContext(ctx, "audiobook scan: indexed", "component", "scanner",
"folder_id", folder.ID,
"content_id", contentID,
"title", parsed.Title,
"author", parsed.Author,
"files", len(parsed.Files),
)
return nil
}
func applyAudiobookEmbeddedCover(
ctx context.Context,
reader audiobookPosterPathReader,
exec audiobookPosterExec,
ffmpegPath string,
cacher audiobookCoverCacher,
audioFilePath string,
contentID string,
) (bool, error) {
if exec == nil {
return false, nil
}
if reader != nil {
existingPosterPath, err := reader.GetPosterPath(ctx, contentID)
if err != nil {
return false, fmt.Errorf("get audiobook poster path for embedded cover: %w", err)
}
if strings.TrimSpace(existingPosterPath) != "" {
return false, nil
}
}
poster, thumb := ExtractAndUploadAudiobookCover(ctx, ffmpegPath, cacher, audioFilePath, contentID)
if poster == "" {
return false, nil
}
_, err := exec.Exec(ctx, `
UPDATE media_items
SET poster_path = $1, poster_thumbhash = $2, updated_at = NOW()
WHERE content_id = $3 AND (poster_path IS NULL OR poster_path = '')
`, poster, thumb, contentID)
if err != nil {
return false, fmt.Errorf("update audiobook embedded cover: %w", err)
}
return true, nil
}
// upsertAudiobookMediaItem reuses an item already linked to the same filesystem
// root, then falls back to the stricter ABS duplicate rule, or creates a new row.
// It intentionally avoids title/year-only dedupe because audiobooks often share
// titles, unknown years, or edition-specific credits. Returns the content_id used.
func (s *Scanner) upsertAudiobookMediaItem(ctx context.Context, folderID int, folderPath string, book *parsedAudiobook) (string, error) {
if s.itemRepo == nil {
return "", fmt.Errorf("itemRepo not configured on Scanner")
}
if s.fileRepo == nil {
return "", fmt.Errorf("fileRepo not configured on Scanner")
}
existingID, err := s.fileRepo.FindContentIDByRootPath(ctx, folderID, folderPath, "audiobook")
if err != nil {
return "", fmt.Errorf("find audiobook by root path: %w", err)
}
if existingID != "" {
if err := s.updateExistingAudiobookMediaItem(ctx, existingID, book); err != nil {
return "", err
}
return existingID, nil
}
cleanTitle := stripNarratorSuffix(book.Title)
if cleanTitle == "" {
cleanTitle = book.Title
}
if existing := s.findAudiobookByFilePath(ctx, folderID, book); existing != nil {
applyBookToMediaItem(existing, book)
if existing.SortTitle == "" {
existing.SortTitle = titleutil.DeriveDefaultSortTitle(existing.Title)
}
if err := s.itemRepo.Upsert(ctx, existing); err != nil {
return "", err
}
return existing.ContentID, nil
}
// Secondary dedup: catch books stored under two folders ("Foo" + "Foo
// Subtitle Version"). Same scan-time rule as the one-shot
// scripts/dedup_audiobooks.py — author + narrator + year + duration
// within tolerance + title is equal or an exact "X" / "X: subtitle"
// prefix relation. Attaches the new file to the existing row so we
// don't pile up duplicates as the scan progresses.
if existing := s.findAudiobookDuplicate(ctx, book, cleanTitle); existing != nil {
applyBookToMediaItem(existing, book)
if existing.SortTitle == "" {
existing.SortTitle = titleutil.DeriveDefaultSortTitle(existing.Title)
}
if err := s.itemRepo.Upsert(ctx, existing); err != nil {
return "", err
}
return existing.ContentID, nil
}
return createAudiobookMediaItem(ctx, s.itemRepo, book, cleanTitle)
}
func (s *Scanner) updateExistingAudiobookMediaItem(ctx context.Context, contentID string, book *parsedAudiobook) error {
items, err := s.itemRepo.GetByIDs(ctx, []string{contentID})
if err != nil {
return fmt.Errorf("get audiobook media item %s: %w", contentID, err)
}
if len(items) == 0 || items[0] == nil {
return fmt.Errorf("audiobook media item %s not found", contentID)
}
item := items[0]
applyBookToMediaItem(item, book)
if item.SortTitle == "" {
item.SortTitle = titleutil.DeriveDefaultSortTitle(item.Title)
}
return s.itemRepo.Upsert(ctx, item)
}
func resolveAudiobookMediaItem(
ctx context.Context,
rootFinder filesystemRootContentFinder,
itemWriter filesystemMediaItemWriter,
folderID int,
folderPath string,
book *parsedAudiobook,
) (string, error) {
if rootFinder == nil {
return "", fmt.Errorf("root content finder not configured")
}
if itemWriter == nil {
return "", fmt.Errorf("media item writer not configured")
}
existingID, err := rootFinder.FindContentIDByRootPath(ctx, folderID, folderPath, "audiobook")
if err != nil {
return "", fmt.Errorf("find audiobook by root path: %w", err)
}
if existingID != "" {
return existingID, nil
}
cleanTitle := stripNarratorSuffix(book.Title)
if cleanTitle == "" {
cleanTitle = book.Title
}
return createAudiobookMediaItem(ctx, itemWriter, book, cleanTitle)
}
func createAudiobookMediaItem(ctx context.Context, itemWriter filesystemMediaItemWriter, book *parsedAudiobook, cleanTitle string) (string, error) {
id, err := idgen.NextID()
if err != nil {
return "", fmt.Errorf("generate content_id: %w", err)
}
item := &models.MediaItem{
ContentID: id,
Type: "audiobook",
Title: cleanTitle,
SortTitle: titleutil.DeriveDefaultSortTitle(cleanTitle),
Year: book.Year,
}
applyBookToMediaItem(item, book)
if item.SortTitle == "" {
item.SortTitle = titleutil.DeriveDefaultSortTitle(item.Title)
}
if err := itemWriter.Upsert(ctx, item); err != nil {
return "", err
}
return id, nil
}
// findAudiobookDuplicate returns an existing audiobook media_items row that
// matches the parsed book on (author, narrator, year, duration ±0.5%/±10s,
// punctuation-normalized title). Used after the file-path lookup misses to
// detect the same recording stored under a different folder name. Returns nil
// when no match exists or any required attribute (author, narrator, year,
// duration) is missing — under-tagged files don't qualify for automatic dedup.
func (s *Scanner) findAudiobookDuplicate(ctx context.Context, book *parsedAudiobook, cleanTitle string) *models.MediaItem {
if s.fileRepo == nil {
return nil
}
if book.Author == "" || book.Narrator == "" || book.Year == 0 {
return nil
}
var totalDuration int
for _, f := range book.Files {
totalDuration += f.Duration
}
if totalDuration <= 0 {
return nil
}
tolerance := totalDuration / 200 // 0.5%
if tolerance < 10 {
tolerance = 10
}
titleKey := normalizeAudiobookDedupeTitle(cleanTitle)
if titleKey == "" {
return nil
}
rows, err := s.fileRepo.Pool().Query(ctx,
audiobookDuplicateCandidateSQL,
book.Author,
book.Narrator,
book.Year,
totalDuration,
tolerance,
titleKey,
)
if err != nil {
return nil
}
defer rows.Close()
var existingID string
for rows.Next() {
var candidateID, candidateTitle string
if err := rows.Scan(&candidateID, &candidateTitle); err != nil {
return nil
}
if audiobookDedupeTitlesMatch(candidateTitle, cleanTitle) {
existingID = candidateID
break
}
}
if rows.Err() != nil || existingID == "" {
return nil
}
items, err := s.itemRepo.GetByIDs(ctx, []string{existingID})
if err != nil || len(items) == 0 {
return nil
}
return items[0]
}
// findAudiobookByFilePath returns an existing audiobook media_items row
// whose media_files reference any of this book's audio file paths.
// Returns nil when no match exists (a fresh scan of a new folder).
func (s *Scanner) findAudiobookByFilePath(ctx context.Context, folderID int, book *parsedAudiobook) *models.MediaItem {
if s.fileRepo == nil || len(book.Files) == 0 {
return nil
}
paths := audiobookLookupPaths(book.Files)
if len(paths) == 0 {
return nil
}
var existingID string
err := s.fileRepo.Pool().QueryRow(ctx, `
SELECT mf.content_id
FROM media_files mf
JOIN media_items mi ON mi.content_id = mf.content_id
WHERE mf.media_folder_id = $2
AND (mf.file_path = ANY($1) OR LOWER(mf.file_path) = ANY($1))
AND mi.type = 'audiobook'
LIMIT 1
`, paths, folderID).Scan(&existingID)
if err != nil || existingID == "" {
return nil
}
items, err := s.itemRepo.GetByIDs(ctx, []string{existingID})
if err != nil || len(items) == 0 {
return nil
}
return items[0]
}
func audiobookLookupPaths(files []parsedAudiobookFile) []string {
seen := make(map[string]struct{}, len(files)*2)
paths := make([]string, 0, len(files)*2)
for _, f := range files {
path := strings.TrimSpace(f.Path)
if path == "" {
continue
}
for _, candidate := range []string{path, strings.ToLower(path)} {
if candidate == "" {
continue
}
if _, ok := seen[candidate]; ok {
continue
}
seen[candidate] = struct{}{}
paths = append(paths, candidate)
}
}
return paths
}
// applyBookToMediaItem copies parsed-audiobook tag fields onto the
// MediaItem. Used for both fresh inserts and re-scans of existing rows
// so manual edits to fields not driven by the file (e.g., poster_path
// set by the metadata enricher) survive.
//
// Audiobook titles are stripped of trailing narrator/edition suffixes
// (`Foo (Read by Bar)`, `Foo - read by Bar`, etc.) since that data is
// already captured as item_people kind=8. The raw tag value is preserved
// in OriginalTitle when it differs so the original is never lost.
func applyBookToMediaItem(item *models.MediaItem, book *parsedAudiobook) {
item.Type = "audiobook"
raw := strings.TrimSpace(book.Title)
cleaned := stripNarratorSuffix(raw)
if cleaned == "" {
cleaned = raw
}
item.Title = cleaned
if cleaned != raw && item.OriginalTitle == "" {
item.OriginalTitle = raw
}
item.Year = book.Year
if book.Overview != "" && item.Overview == "" {
item.Overview = book.Overview
}
if book.Publisher != "" {
item.Studios = mergeUniqueStrings(item.Studios, []string{book.Publisher})
}
if len(book.Genres) > 0 {
item.Genres = mergeUniqueStrings(item.Genres, book.Genres)
}
if rd := normalizeReleaseDateForSQL(book.ReleaseDate); rd != "" && (item.ReleaseDate == nil || *item.ReleaseDate == "") {
item.ReleaseDate = &rd
}
if book.Language != "" && item.OriginalLanguage == "" {
item.OriginalLanguage = book.Language
}
}
// normalizeReleaseDateForSQL coerces tag-derived date strings into the
// ISO YYYY-MM-DD shape media_items.release_date expects. Year-only
// tags ("2018") become "2018-01-01"; ISO already-correct values pass
// through; everything else returns "" (skip the column).
func normalizeReleaseDateForSQL(s string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
if len(s) >= 10 && s[4] == '-' && s[7] == '-' {
return s[:10]
}
if len(s) >= 4 {
y := s[:4]
ok := true
for _, c := range y {
if c < '0' || c > '9' {
ok = false
break
}
}
if ok {
return y + "-01-01"
}
}
return ""
}
func mergeUniqueStrings(existing, additions []string) []string {
seen := make(map[string]struct{}, len(existing)+len(additions))
out := make([]string, 0, len(existing)+len(additions))
for _, v := range existing {
k := strings.ToLower(strings.TrimSpace(v))
if k == "" {
continue
}
if _, ok := seen[k]; ok {
continue
}
seen[k] = struct{}{}
out = append(out, v)
}
for _, v := range additions {
k := strings.ToLower(strings.TrimSpace(v))
if k == "" {
continue
}
if _, ok := seen[k]; ok {
continue
}
seen[k] = struct{}{}
out = append(out, v)
}
return out
}
// upsertAudiobookMediaFiles writes one media_files row per audio file in the
// parsed audiobook. The content_id ties each file back to the media_items row.
// folderPath is used as the canonical_root_path / observed_root_path.
func (s *Scanner) upsertAudiobookMediaFiles(
ctx context.Context,
folder *models.MediaFolder,
contentID string,
folderPath string,
book *parsedAudiobook,
) error {
partTotal := len(book.Files)
for idx, af := range book.Files {
chapters := make([]models.MediaChapter, len(af.Chapters))
for i, ch := range af.Chapters {
chapters[i] = models.MediaChapter{
Index: ch.Index,
Title: ch.Title,
StartSeconds: ch.StartSeconds,
EndSeconds: ch.EndSeconds,
Source: ch.Source,
}
}
mf := models.MediaFile{
ContentID: contentID,
MediaFolderID: folder.ID,
CanonicalRootPath: folderPath,
ObservedRootPath: folderPath,
ContentGroupKey: contentID,
GroupKeyVersion: 1,
BaseTitle: book.Title,
BaseYear: book.Year,
BaseType: "audiobook",
IdentityConfidence: audiobookIdentityConfidence(book, af),
FilePath: af.Path,
Chapters: chapters,
ProbeSource: "local",
Duration: af.Duration,
Bitrate: af.Bitrate,
CodecAudio: af.CodecAudio,
Container: af.Container,
AudioChannels: af.AudioChannels,
}
if partTotal > 1 {
mf.PresentationKind = "multipart"
mf.PresentationGroupKey = contentID
mf.PresentationPartIndex = idx + 1
mf.PresentationPartTotal = partTotal
}
if _, err := s.fileRepo.Upsert(ctx, mf); err != nil {
return fmt.Errorf("upsert media file %s: %w", af.Path, err)
}
}
return nil
}
// audiobookCredit pairs a person name with a credit kind. Used to compare
// the desired-from-tags set against the existing item_people set without
// having to materialize a full []models.ItemPerson (which requires
// resolved person IDs).
type audiobookCredit struct {
Name string
Kind models.PersonKind
}
// audiobookPeopleCreditsEqual returns true when the existing item_people
// rows for an audiobook match the desired credit set one-for-one on
// (case-insensitive name, kind). Order is irrelevant because the upsert
// path orders by SortOrder.
//
// Case-insensitive comparison: audiobook tag casing drifts between rips
// and we don't want a stylistic re-cap to trigger DELETE+INSERT on every
// scan.
func audiobookPeopleCreditsEqual(existing []models.ItemPerson, desired []audiobookCredit) bool {
if len(existing) != len(desired) {
return false
}
type key struct {
name string
kind models.PersonKind
}
have := make(map[key]struct{}, len(existing))
for _, p := range existing {
have[key{strings.ToLower(strings.TrimSpace(p.Person.Name)), p.Kind}] = struct{}{}
}
for _, d := range desired {
k := key{strings.ToLower(strings.TrimSpace(d.Name)), d.Kind}
if _, ok := have[k]; !ok {
return false
}
}
return true
}
// upsertAudiobookPeople upserts author and narrator rows into item_people,
// using the PersonRepository to find-or-create each person by name. When the
// parsed credits differ, ReplacePeople makes scanner-derived credits authoritative.
// Skips the DELETE+INSERT entirely when the existing credit set already
// matches the desired set (case-insensitive on name, exact on kind).
func (s *Scanner) upsertAudiobookPeople(ctx context.Context, contentID string, book *parsedAudiobook) error {
if s.personRepo == nil {
return fmt.Errorf("personRepo not configured on Scanner")
}
var desired []audiobookCredit
if book.Author != "" {
desired = append(desired, audiobookCredit{Name: book.Author, Kind: models.PersonKindAuthor})
}
if book.Narrator != "" {
desired = append(desired, audiobookCredit{Name: book.Narrator, Kind: models.PersonKindNarrator})
}
if len(desired) == 0 {
return nil
}
existing, err := s.itemRepo.GetPeople(ctx, contentID)
if err == nil && audiobookPeopleCreditsEqual(existing, desired) {
return nil
}
people := make([]models.ItemPerson, 0, len(desired))
for i, c := range desired {
personID, err := s.personRepo.FindOrCreate(ctx, models.Person{Name: c.Name})
if err != nil {
return fmt.Errorf("find-or-create person %q: %w", c.Name, err)
}
people = append(people, models.ItemPerson{
Person: models.Person{ID: personID},
Kind: c.Kind,
SortOrder: i,
})
}
return s.itemRepo.ReplacePeople(ctx, contentID, people)
}
// upsertAudiobookSeries writes the parsed series_name and series_index into
// the audiobook_series table, overwriting any prior row (e.g. one populated
// by migration 145's title-pattern backfill). A blank book.Series clears
// the row so books explicitly retagged out of a series stop appearing in
// the "In this series" rail on the next scan.
func (s *Scanner) upsertAudiobookSeries(ctx context.Context, contentID string, book *parsedAudiobook) error {
if s.fileRepo == nil {
return fmt.Errorf("fileRepo not configured on Scanner")
}
desiredName := strings.TrimSpace(book.Series)
desiredIdx := parseSeriesIndex(book.SeriesPosition)
// Read current row.
var currentName *string
var currentIdx *float64
err := s.fileRepo.Pool().QueryRow(ctx, `
SELECT series_name, series_index FROM audiobook_series WHERE content_id = $1
`, contentID).Scan(&currentName, &currentIdx)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return fmt.Errorf("query audiobook_series: %w", err)
}
if desiredName == "" {
if currentName == nil {
return nil // already absent
}
if _, delErr := s.fileRepo.Pool().Exec(ctx,
`DELETE FROM audiobook_series WHERE content_id = $1`, contentID); delErr != nil {
return fmt.Errorf("delete audiobook_series row: %w", delErr)
}
return nil
}
if currentName != nil && *currentName == desiredName && floatPtrEqual(currentIdx, desiredIdx) {
return nil // identical row, skip the write
}
var idx any
if desiredIdx != nil {
idx = *desiredIdx
}
if _, err := s.fileRepo.Pool().Exec(ctx, `
INSERT INTO audiobook_series (content_id, series_name, series_index, updated_at)
VALUES ($1, $2, $3, NOW())
ON CONFLICT (content_id) DO UPDATE SET
series_name = EXCLUDED.series_name,
series_index = EXCLUDED.series_index,
updated_at = NOW()
`, contentID, desiredName, idx); err != nil {
return fmt.Errorf("upsert audiobook_series row: %w", err)
}
return nil
}
func audiobookIdentityConfidence(book *parsedAudiobook, file parsedAudiobookFile) string {
if book == nil {
return "low"
}
score := 0
if book.Title != "" {
score++
}
if book.Author != "" || book.Narrator != "" {
score++
}
if book.Year > 0 {
score++
}
for _, ch := range file.Chapters {
if ch.Title != "" && ch.EndSeconds > ch.StartSeconds {
score++
break
}
}
switch {
case score >= 4:
return "high"
case score > 0:
return "medium"
default:
return "low"
}
}
// floatPtrEqual returns true when two *float64 values represent the same
// state — both nil, or both non-nil and equal. Used to skip audiobook_series
// re-upserts when only the series_index needs to be NULL=NULL compared.
func floatPtrEqual(a, b *float64) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
// parseSeriesIndex extracts a leading numeric value from a freeform tag
// like "5", "1.5", "2 of 8", or "1a". Returns nil when no leading number
// is present so the audiobook_series.series_index column stays NULL rather
// than carrying a misleading zero.
func parseSeriesIndex(raw string) *float64 {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
end := 0
dot := false
for end < len(raw) {
c := raw[end]
if c >= '0' && c <= '9' {
end++
continue
}
if c == '.' && !dot {
dot = true
end++
continue
}
break
}
if end == 0 {
return nil
}
var v float64
if _, err := fmt.Sscanf(raw[:end], "%f", &v); err != nil {
return nil
}
return &v
}