Compare commits

..
Author SHA1 Message Date
QuickandClaude Fable 5 5b70dc7357 web: default subtitle background style to Box 75%
Matches the Apple clients' default so a profile with no saved appearance
renders the same on every client. (Android's default remains None and is
tracked separately.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146hwg1Q15YdeUhd9f37NWA
2026-07-04 10:51:33 -04:00
QuickandClaude Fable 5 ade3a3c5d4 docs: add GitHub Sponsors funding and restructure README for selfhosters
Add .github/FUNDING.yml to enable the Sponsor button, refocus the
README on selfhoster users with a Highlights section and a Supporting
Silo section, and move contributor-facing content (dev workflow, make
targets, migrations, tests, project structure) into DEVELOPMENT.md.

AI-use disclosure: written with Claude Code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:14:58 -04:00
ce06dc8ca5 fix(web): show Resume Reading on the manga CTA when the target volume is in progress (#280)
The series hero CTA only inspected chapter.read, so a volume with saved
partial progress (e.g. 18%) still rendered "Start Reading" — the button
contradicted the progress indicator on the same page. The resume target
with progress > 0 now renders "Resume Reading"; untouched series keep
"Start Reading" and mid-series targets keep "Continue".

Fixes #188

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:58:54 -04:00
604bbf1a0f feat(playback): unified restart-resilient playback (native + jellycompat) (#174)
* feat(playback): unified restart-resilient playback via shared TranscodeManager

Make direct, remux, and native HLS transcode sessions survive a server
restart through one shared flow instead of per-method paths. A missing
in-memory session becomes a reconstruct trigger, not a 404: the server
rebuilds the session from a tiny durable recipe card plus the position the
client re-supplies on its next request.

- internal/playback/transcode_manager.go: shared TranscodeManager owning the
  transcodes map, recipe-card lifecycle, reconstruct single-flight +
  concurrency cap, LoadOrReconstructSession front door, ReconstructSession /
  ReconstructTranscode, and orphan cleanup. ~90% is logic moved out of the
  native handler (no behavior change), not new surface.
- internal/playback/recipecard.go + recipecard_postgres.go: RecipeCard with a
  PlayMethod discriminator (direct/remux/transcode; empty decodes as transcode
  for back-compat) behind a swappable, nil-safe RecipeStore interface backed by
  transcode_recipes.
- internal/playback/session.go: RegisterReconstructed inserts a rebuilt Session
  under its existing id (no UUID mint, no limit double-count, race-yielding).
- internal/playback/transcode.go: CloseProcess keeps the output dir so a
  reconstruct winner keeps serving; Close removes it.
- internal/api/handlers: drain the transcode lifecycle into the manager; wire
  reconstruct into the stream/segment serve paths; re-bind ownership to the live
  caller (refuse userID==0/mismatch); card-aware orphan cleanup.
- migrations: add transcode_recipes (expires_at TTL, filter-on-read, indexed).

Ownership stays two-factor: an authenticated caller AND a session.UserID that
matches; the card stores no secrets and identity is re-resolved per request.

Tests: recipe-card round-trip/legacy-decode/disabled-noop, RegisterReconstructed
insert/race/concurrency, close-vs-close-process dir semantics, the
LoadOrReconstructSession status matrix, and the reconstruct concurrency cap.

AI-use: implemented with AI assistance (design, implementation, adversarial review).

* feat(jellycompat): reconstruct transcodes across restart via shared manager

Bring Jellyfin (jellycompat) HLS playback onto the same restart-resilient flow
as the native path. Previously jellycompat owned a separate PlaybackHandler with
a private transcodes map and a duplicated transcode lifecycle that never grew the
reconstruct half, so an in-flight Jellyfin transcode died on restart and the next
segment request 404'd.

- Embed the shared playback.TranscodeManager and delete the duplicate lifecycle,
  so jellycompat gets reconstruct, the concurrency cap, the node-affinity rule,
  and the card lifecycle for free.
- internal/jellycompat/playback_sessions_postgres.go: DurableCompatPlaybackStore,
  a write-through cache over jellycompat_playback_sessions behind the new
  CompatPlaybackStore interface (nil pool degrades to cache-only). This persists
  the load-bearing PlaySessionId -> UpstreamSessionID mapping (plus media sources,
  route item id, seek) so it survives a restart instead of vanishing with the map.
- Write a recipe card on compat transcode start keyed by the upstream session id,
  using the native StreamAppUserID so the ownership re-bind matches; reconstruct
  the upstream session and the transcode seeked to the requested seg_NNNNN.
- migrations: add jellycompat_playback_sessions (expires_at TTL + compat_token
  index, full PlaybackSession in data JSONB).

Auth is mapped to the native user id before reconstruct so the same two-factor
ownership check and userID==0/mismatch refusal apply unchanged.

Tests: DB-gated (SILO_TEST_DATABASE_URL) durable-store round-trip proving a
session written by one instance reloads in a fresh one (the restart case), plus
a nil-pool cache-only path; existing handler tests updated to the manager.

AI-use: implemented with AI assistance (design, implementation, adversarial review).

* docs(playback): consolidate unified playback reconstruction design

Replace the three overlapping playback docs (the native Postgres
restart-resilience spec, the jellycompat plan, and the unification spec) with a
single self-contained design at
docs/superpowers/specs/unified-playback-reconstruct.md.

The doc leads with the unified design — the one-idea reconstruct model, a strong
visual flow of a restart mid-playback, the shared TranscodeManager + recipe card,
the two swappable durable stores, security, the concurrency cap and node-affinity
constraint, preconditions, and verification. The design history and rationale
(reconstruct-not-rehydrate, phased delivery, Redis-vs-Postgres, token-as-
descriptor, failure analysis) move to an appendix. It references no other md file.

AI-use: written with AI assistance.

* fix(playback): address review on restart-resilient playback

Four fixes from PR review of the unified reconstruction work:

- Rewrite the recipe card on audio-track change. HandleChangeAudioTrack only
  updated the in-memory session/transcode, so after a restart reconstruct
  resumed with the stale AudioTrackIndex/TranscodeAudio (and stale play method)
  from the start-time card. Re-save the card (direct/remux/transcode) with the
  switched state, mirroring the start-card pattern.
- Guard nil TranscodeManager in LoadOrReconstructSession and ReconstructSession.
  StreamHandler.TM is documented optional (tests/minimal setups); a missing
  session previously panicked in recipeEnabled instead of returning
  SessionMissing. ReconstructTranscode already guarded nil; make the two
  siblings consistent.
- Reject direct/remux cards in doReconstructTranscode before spawning ffmpeg, so
  a non-transcode card id can never enter the HLS reconstruction path.
- Log a non-success status from the remote transcode-node DELETE in
  CloseTranscodeSession; a 401/404/500 was previously silent.

AI-use: implemented with AI assistance.

* fix(playback): harden restart-resilient compat sessions

* feat(playback): token-carried reconstruction across restarts

Build on the shared TranscodeManager (introduced earlier in this branch) so a
playback session survives an API-server or transcode-node restart without the
client re-negotiating, and retire the Postgres transcode_recipes store in favor
of a recipe carried inside the signed stream token.

- RecipeCard encodes the byte-affecting encode parameters and rides inside the
  stream token; LoadOrReconstructSession rebuilds the in-memory Session (and,
  for integrated transcodes, the ffmpeg process) on a cold miss, single-flighted
  per session and paced by a spawn semaphore. Removes recipecard_postgres.go and
  the 20260617233705_add_transcode_recipes migration.
- transcodenode reconstructs a lost ffmpeg node-side from the forwarded token.
- TR-lease: proxy/streamauth enforce a revocation deny-marker on every served
  segment, with a 500ms Redis timeout, a bounded per-session "allowed" cache
  (3s TTL, expiry-first graceful eviction), and a degraded-fail-open counter.

Review hardening folded in:
- Manifest/segment handlers do the in-memory session lookup first and only
  verify the stream token on a reconstruct miss (token HMAC was per-segment).
- Copy-mode reconstruct never applies the encoded-only seg*dur seek, at spawn
  time or via the recovery path: RestartSeekTarget reports "unresolved" for a
  copy session whose manifest cannot yet map the segment, so the client retries
  instead of seeking to a fabricated source time.
- Crash teardown is a compare-and-delete (CloseTranscodeSessionIf returns
  whether it matched); the crash closure tears down the playback session only
  when it matched, so a session reconstructed under the same id is not killed.
- Reconstruct enforces the same per-user stream/transcode caps as a fresh start
  (RegisterReconstructedWithLimits), closing a token-replay slot bypass.

AI-use disclosure: implemented with AI assistance (Claude Code), including a
two-round multi-agent adversarial review whose findings drove the hardening.

* feat(jellycompat): node-side transcode reconstruct via shared recipe store

Make Jellyfin-compat playback sessions survive a server or transcode-node
restart by reusing the shared TranscodeManager reconstruct path and a durable
recipe store, on top of the durable compat session store added earlier in this
branch.

- Node-side transcode reconstruct goes through the shared recipe store; the
  recipe is persisted to the control-plane store (Redis) when a dedicated
  transcode node is used so the node can rebuild ffmpeg after its own restart.
- Adopt the shared manager's API (3-arg OnFFmpegCrash carrying the dead session,
  guarded CloseTranscodeSessionIf, RegisterReconstructedWithLimits).

Review hardening folded in:
- Recipe lifecycle: noderecipe.Store gains Delete, called on deliberate
  teardown (stop, method-switch discard, node stop/force-reload) so a stopped
  session cannot be resurrected by a buffered request after a node restart;
  crash paths intentionally keep the recipe so a resume can reconstruct.
- Crash closure tears down the upstream session only when the guarded transcode
  close matched, so a reconstructed successor is never left orphaned.
- Copy-mode segment recovery surfaces a retryable not-found instead of a
  wrong-position restart, matching the native and node paths.
- Durable Update is now a SELECT ... FOR UPDATE transaction, removing the
  lost-update clobber that could silently drop a transcode recipe.
- Empty-token route resolution no longer falls back to an unbounded full-table
  scan; DB expiry filters bind the injected clock; the redundant re-Get is gone.

AI-use disclosure: implemented with AI assistance (Claude Code), including a
two-round multi-agent adversarial review whose findings drove the hardening.

* docs(playback): consolidate restart-resilient playback design

Replace the superpowers spec with a single architecture record describing the
token-carried recipe card, the shared TranscodeManager reconstruct path for
direct/remux/transcode, the jellycompat durable session + node recipe store, and
the revocation-lease model with its fail-open tradeoff.

AI-use disclosure: written with AI assistance (Claude Code).

* docs(playback): correct jellycompat node-recipe rationale in comments

The noderecipe / transcode-node / jellycompat comments justified the Redis
recipe store with "a Jellyfin client cannot round-trip a token". The real
reason: the node-hop token is server-minted and could carry the recipe, but the
recipe is mutated in place under a stable session id (a /Sessions/Playing/Progress
audio switch restarts ffmpeg without re-minting the client's token) and a
third-party Jellyfin client cannot be driven to refresh a stale token, so the
node must reconstruct from a server-authoritative, node-reachable store.

Aligns the comments with docs/architecture/restart-resilient-playback.md §10.
Comment-only; no behavior change.

* refactor(playback): remove deny-lease revocation, defer to future PR

The deny-lease stream-revocation mechanism (the internal/streamauth
package, its silo:streamauth:<sid> Redis markers, the proxy Allowed()
enforcement, and the admin Stop/Terminate deny write) only ever enforced
on the offload-proxy topology and was a silent no-op on the integrated
single box and the dedicated transcode node. Rather than ship a partial
revocation feature that looks complete but isn't, remove it wholesale and
defer a uniform cross-topology revocation design to a dedicated follow-up.

Removed: internal/streamauth (package + tests); the LeaseDenier field,
StreamLeaseDenier interface, and denyStreamLease helper in playback.go;
the admin deny write; the router/main wiring; and the proxy verifyToken
Allowed() gate. The unified-reconstruct core (recipe-token,
LoadOrReconstructSession) is orthogonal and untouched.

Known limitation (now on every topology): admin Terminate and user Stop
tear down the live in-memory session and ffmpeg producer, but a still-valid
stream token can reconstruct the session until its 24h TTL expires. No
node-side byte-withholding ships in this PR.

docs/architecture/restart-resilient-playback.md is updated to mark the
revocation/deny-lease sections as deferred and to drop the overstated
"instant revocation on admin kill" claim.

* fix(playback): allow zero-caller bearer on transcode reconstruct

The authless HLS transcode delivery routes (master.m3u8 / segment) treat
the session UUID as the bearer credential, so a real request carries
requestUserID == 0. The live serve path already allows this, but
ReconstructSession hard-rejected a zero caller, so a request that worked
before a restart became SessionMissing -> 404 after the in-memory session
was gone, breaking the restart resilience these routes advertise.

Match the live-path contract in LoadOrReconstructSession: allow a zero
caller (UUID-as-bearer) and refuse only a non-zero caller that mismatches
the card owner. The reconstructed session is bound to card.UserID either
way. Adds TestReconstructSession_Ownership covering both cases.

* fix(jellycompat): re-persist recipe on local audio switch

A Jellyfin client switching audio on an integrated/local compat transcode
restarted live ffmpeg with the new track but did not re-persist
PlaybackSession.Recipe. The remote branch already re-persists via
startRemoteTranscode -> persistTranscodeRecipe. After a central restart,
reconstruct rebuilt ffmpeg from the stale Recipe.AudioTrackIndex, so the
integrated session resumed on the original audio track.

Persist the updated recipe (best-effort) after a successful Restart in the
local branch, mirroring the remote branch, so the durable
Recipe.AudioTrackIndex tracks live ffmpeg. Adds a regression test.

* fix(playback): strip stream token from proxied transcode-node URL

proxyToTranscodeNode appended the client's raw query string to the internal
transcode-node URL and logged that URL on transport failure. When a remote
transcode runs without a separate proxy node, that query carries
?st=<signed JWT> — a 24h bearer reconstruction descriptor exposing the
media path and recipe claims — placing the token into internal requests and
error logs.

Strip the "st" param before building targetURL, preserving any other query
params. The token is neither forwarded to the node nor present in the
logged URL. Header-forwarding of the token (so the node can reconstruct) is
a separate follow-up (#6).

* fix(playback): fail open on transient limit-provider error in reconstruct

During the reconstruct wave right after a restart (Postgres under peak
load), a transient limit-provider DB error was collapsed into a hard 404,
permanently stopping playback for a user within their limits. limitsForUser
wrapped any provider error, RegisterReconstructedWithLimits propagated it,
and ReconstructSession mapped every error to SessionMissing -> 404 -
indistinguishable from a genuine over-cap rejection.

Distinguish the two: tag provider errors with a new ErrLimitProviderUnavailable
sentinel and, during reconstruct, fail OPEN on a provider error (admit via
RegisterReconstructed + log a degraded warning) rather than refuse - mirroring
the reliability-first fail-open-on-dependency-error philosophy. A genuine
ErrTooManyStreams / ErrTooManyTranscodes over-cap still refuses. Adds tests
for both the fail-open and still-refused paths.

* fix(playback): forward stream token to transcode node as header

The dedicated transcode node's reconstruct path reads the stream token only
from the X-Silo-Stream-Token header, but proxyToTranscodeNode forwarded only
the node-API bearer token (and #5 now strips st from the URL). So when the
central API proxied to the node and the node self-restarted, it could not
reconstruct from the recipe-complete native token -> 404.

Capture st before stripping it from the URL, verify it at the API boundary
(streamtoken.Verify + SessionID match, mirroring the node's own check), and
forward it as X-Silo-Stream-Token. Best-effort: a missing/invalid token never
blocks the live proxy, and the token is still kept out of the forwarded URL
and logs.

* fix(playback): restart node ffmpeg on native remote audio switch

A native audio-track switch on an offloaded/remote transcode was a no-op at
the node yet returned 200 with a fresh URL: HandleChangeAudioTrack restarted
ffmpeg only when the API owned a LOCAL TranscodeSession, so for an offloaded
transcode the node kept serving the OLD audio (the node consults the token
only on a session miss). The replacement URL was also minted from identity-
only claims, so a later node restart 404'd.

For the offloaded transcode case (detected via session.TranscodeNodeURL),
POST a fresh /transcode/start to the node with the new AudioTrackIndex
(handleStart tears down and restarts ffmpeg) and mint the replacement proxy
URL from a full RecipeCard so reconstruct survives a node restart. The encode
recipe is derived from the durable session target fields plus the file,
mirroring HandleStartTranscode. A concrete SegmentDuration
(playback.DefaultSegmentDuration) is embedded rather than 0: the node's token
completeness gate treats SegmentDuration<=0 as incomplete and falls back to a
recipe store the native path never populates, which would 404 on a node
restart - the exact resilience this path provides. A failed node POST now
surfaces 502 rather than a false 200. Remux and non-offloaded (local)
transcode paths keep their prior identity-claim URLs unchanged.

Known limitation: Session does not persist the original SegmentDuration or
SubtitleTrackIndex/SubtitleBurnIn, so a remote audio switch resets subtitle
selection to none and assumes the default segment length; a client that
started with a non-default segment length will resegment on switch. Making
that state durable on the session is a follow-up.

* docs(playback): scrub stale deny-lease/revalidator comments

The deny-lease revocation mechanism and its "central revalidator" were removed
earlier in this branch, but four comments still described them as live
(transcode_manager.go, noderecipe/store.go, streamtoken/token.go,
proxy/server.go). Reword them to match the shipped behavior: ownership claims
are re-resolved at reconstruct, the noderecipe store shares Redis only with the
node-session tracker, and a sub-TTL hard cut depends on a node-side revocation
mechanism that is deferred to a future PR.

* fix(jellycompat): surface durable playback-session write failures

DurableCompatPlaybackStore.Update applied the in-memory mutation and then
swallowed every Postgres commit-failure path, returning nil. Callers that
promise restart resilience (persistTranscodeRecipe's recipe write, the
upstream-session binds in streams.go) were told the session was durably
persisted when only the cache held it, so a transient DB hiccup could leave
the next restart reloading a stale row (wrong audio track) or 404ing.

updateDB now returns the genuine DB round-trip error (begin/query/unmarshal/
marshal/exec/commit); Update propagates it while still applying the in-memory
mutation so live state stays correct. A nil pool and a genuinely absent/expired
row remain best-effort (return nil) — only real infrastructure failures
propagate, so existing rollback paths fire exactly when durability is lost.

Part of #174

* fix(playback): re-inject stream token into proxied transcode manifests

API-proxied remote transcode manifests dropped the reconstruct token from
their segment URLs, so playback died after a node or API restart. When a
remote transcode has no separate proxy node, the client loads its manifest via
the API-local path; proxyToTranscodeNode strips the signed token ("st") from
the forwarded URL (keeping it off node URLs and logs, forwarded only as the
X-Silo-Stream-Token header), and the node builds relative segment URIs from
that token-less query. The segment URLs the client received carried no token,
and the proxy only re-attached the header when an incoming segment request
already had "st" — which it never did — so a restart made those segments
non-reconstructable and they 404'd.

proxyToTranscodeNode now rewrites the manifest body at the boundary: every
segment and #EXT-X-MAP init URI gets the client-facing, API-verified token
re-appended (new playback.AppendManifestQueryParam helper), so the client's
later segment fetches carry "st" again and reconstruct after a restart. The
token still never reaches the node URL or its logs. Only 200 .m3u8 responses
are rewritten (Content-Length corrected); segments stream through untouched.

Part of #174

* fix(playback): preserve subtitle/cadence recipe across offloaded audio switch

Switching audio on a remote (offloaded) transcode with burned-in subtitles
silently dropped them, and reset a non-default segment cadence. The offloaded
audio-switch restart rebuilt the node start request from Session state, but
Session/SessionStreamState retained no subtitle or segment-duration state
(only the live local ts.Opts() and the RecipeCard did), so the branch
hard-coded SubtitleTrackIndex:-1, SubtitleBurnIn:false and
SegmentDuration:Default — signing that altered recipe into the replacement
stream token. An audio switch then changed bytes beyond audio selection, and
any later reconstruct kept the wrong no-subtitle/wrong-cadence recipe.

Persist the byte-affecting recipe on the session: SubtitleTrackIndex,
SubtitleBurnIn and SegmentDuration are added to Session/SessionStreamState,
populated at start (finalizeTranscodeStart) and on post-restart reconstruct
(ReconstructSession from the card), carried forward on every audio-switch
state update, and read back when rebuilding the offloaded node request and its
recipe card. The restart now reproduces the exact live stream. Also resolves
the M-4b non-default segment_duration reset.

Part of #174

* fix(playback): serialize transcode spawn paths with a per-session lock

Reconstruct was single-flighted only against other reconstructs, so a
restart-driven segment reconstruct racing a quality/seek/audio fresh start
could spawn two ffmpeg processes writing the same output directory at once —
segment corruption, partial-write closes, orphaned processes, and skewed
active-job accounting. The atomic register-after-spawn (GetOrRegister / the
reconstruct compare-on-register) prevented a map leak but not the concurrent
disk writers, because the losing path had already spawned. The dedicated
transcode node had the same split between handleStart and spawnReconstruct.

Add a refcounted per-session lifecycle lock to both TranscodeManager and the
node Server, held across "check existing -> spawn -> register":
- reconstruct (doReconstructTranscode / spawnReconstruct) re-checks under the
  lock and yields to any live session instead of spawning a duplicate;
- the native and jellycompat fresh-start paths take the lock around their
  spawn+register (the native path also closes any session a reconstruct rebuilt
  in the meantime so its fresh ffmpeg is the sole writer);
- the node handleStart holds it across teardown+spawn+register.
The refcount drops the map entry once no path holds/waits, keeping it bounded.
GetOrRegisterTranscodeSession is removed — the lock supersedes it and keeping a
register-after-spawn primitive would invite reintroducing the race.

Part of #174

* fix(playback): serialize restart re-spawn under the session lifecycle lock

TranscodeSession.Restart() releases s.mu across cancel -> wait-for-done ->
re-exec and spawns ffmpeg into opts.OutputDir without holding the per-session
lifecycle lock. LockSessionLifecycle's contract (fresh start, restart,
reconstruct) requires restart to hold it too, but all five callers invoked
Restart unlocked: native audio-switch and segment-recovery, compat
audio-switch and segment-recovery, and the transcode-node segment-recovery.

A restart racing another restart (audio-switch vs segment-recovery) or a
fresh-start/reconstruct could land two ffmpeg processes writing the same
segment directory -- mixed timelines, init.mp4/segment mismatch, and an
orphaned-but-still-writing ffmpeg -- the exact concurrent-writer corruption
the lifecycle lock exists to prevent.

Add RestartSessionLocked (TranscodeManager) and restartSessionLocked (node
Server) that hold LockSessionLifecycle only across the cancel->respawn
transition, re-check that the handle is still the live mapped session under
the lock, and return ErrSessionSuperseded rather than re-spawning a stale
handle. Route all five call sites through them. The lock is released before
callers wait on segments so recovery latency is unchanged.

Tests: gating (restart blocks until the lifecycle lock frees, then spawns),
concurrent-restart serialization, and superseded re-check on both the manager
(covers native + compat) and node lock owners.

---------

Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-07-02 14:23:14 -04:00
631d95530a fix(catalogseed): cast v.id to bigint in bulkUpdatePeople (#274)
A standalone VALUES clause gives Postgres no target column to infer
parameter types from, so v.id defaulted to text and the UPDATE ... FROM
join predicate became bigint = text, failing seed imports with SQLSTATE
42883 during the people-update phase.

Fixes #267

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:08:15 -04:00
d590eda22a fix(libraries): exclude manga chapters from admin Unmatched queue (#275)
* fix(libraries): exclude manga chapters from admin Unmatched queue

Manga chapter rows are internal sub-units resolved through their series
and intentionally stay 'pending' with type='ebook', so they flooded the
admin Unmatched Items view even after the series matched successfully.
Apply the shared MangaChapterExclusionWhere guard to both the count and
list queries, matching every other catalog listing surface.

Fixes #204

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(libraries): scope unmatched-queue regression test with a unique per-run tag

The regression test seeded fixed literal titles and asserted an exact
total for q=Unmatched+Test, which is collision-prone against a shared
SILO_TEST_DATABASE_URL database: leftover or concurrently seeded rows
matching the literal query would skew the count and flake the test.

Embed a unique per-run tag (issue204-<unixnano>) in every seeded title
and scope the search query to that tag (URL-encoded), so the exact
total assertion only ever sees this run's rows. Assertions are
otherwise unchanged: the manga chapter stays excluded, the plain
pending ebook stays present, and total == 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-07-02 14:08:02 -04:00
c75c519e3d fix(metadata): stop manual rematch from resurrecting recorded stale IDs (#276)
* fix(metadata): stop manual rematch from resurrecting recorded stale IDs

The Apply Match flow (ModeIdentify) re-injected durable provider IDs into
the identify request without checking stale_media_ids, so a known-dead
tmdb ID rode along, 404ed again during the Phase-2 fetch, and was
re-recorded with a fresh last_seen_at — the item never left the Stale
External IDs list and jumped back to the top after every rematch.

Filter recorded-stale IDs out of the injected durable set in
prepareProcessRequest. Caller-supplied IDs are untouched, so an admin
deliberately re-selecting a previously-stale ID still retries it (which
is also why the ModeIdentify suppression guard in processInternal stays).

Fixes #268

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(metadata): normalize provider-id keys so stale-ID suppression can't be bypassed by casing

Review on PR #276 flagged that suppressRecordedStaleProviderIDs lowercases
and trims the stored stale row's provider before looking it up in the
incoming map, while the map keys are used verbatim, and that
HandleApplyItemMatch passes req.ProviderIDs from the JSON body straight
into metadata.Process without the normalization the search endpoint
applies. A caller-supplied key like "TMDB" or " tmdb " therefore defeated
the suppression. The same normalization gap was previously flagged on
PR #182.

Fix both layers:
- HandleApplyItemMatch now runs req.ProviderIDs through
  normalizeMatchProviderIDs (same semantics as the search endpoint) and
  returns 400 when no non-blank entries remain, mirroring the existing
  empty-map rejection.
- suppressRecordedStaleProviderIDs now indexes the incoming map by
  normalized key and deletes the matching original keys, so suppression
  is robust regardless of caller casing or padding.

Adds regression tests at both layers: a metadata-level case where the
durable row arrives as "TMDB " while the stale row records "tmdb", and
handler-level cases asserting apply normalizes keys/values and rejects
all-blank provider-id maps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-07-02 14:07:34 -04:00
012006e7a7 test(access): add ListProgressSince to stubStore so tests compile (#277)
PR #258 added ListProgressSince to the userstore.UserStore interface but
did not update the access resolver test stub, breaking test compilation
for internal/access on main.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:07:28 -04:00
f983c54e87 fix(metadata): re-fetch titles/overviews when a library's metadata language changes (#278)
* fix(metadata): re-fetch titles/overviews when a library's metadata language changes

An item's default_metadata_language was stamped once at first match and
never updated: the canonical-language pin in mergeAndPersist routed any
refresh in a different language into the localization tables, the
upserts' COALESCE kept the old stamp forever, quick-mode library refresh
skipped complete items entirely, and changing the language in
HandleUpdateLibrary triggered nothing. Items stayed in the old language
no matter how often the admin refreshed (#211).

Four coupled changes:
- ProcessRequest.AdoptLanguage: folder-scoped manual refreshes adopt the
  library's language as the item's new canonical language when it
  differs from the stamp, rewriting the base row instead of localizing
  to the side. Only ModeManualRefresh adopts — scheduled refreshes merge
  fill-empty and would restamp without rewriting the text.
- Upsert language pins inverted (media_items, seasons, episodes): prefer
  the incoming non-empty default_metadata_language over the existing
  stamp. All existing callers send the unchanged stamp or empty, so
  behavior is unchanged outside adoption; the restamp is atomic with the
  canonical write.
- Quick-mode refresh lister now includes complete items whose stamp
  differs from the library's configured language.
- HandleUpdateLibrary enqueues a quick library metadata refresh when the
  metadata language changes, mirroring the paths-change rescan trigger.

Fixes #211

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(metadata): gate language adoption on field locks and library agreement

Two hardening fixes for AdoptLanguage from review of #278:

- Locked fields defeat the restamp: title/overview merges honor per-field
  locks, but the restamp was unconditional. An item with both language-
  bearing fields locked kept its old-language text yet got stamped the
  new language, so the quick-refresh mismatch predicate never flagged it
  again. mergeAndPersist now skips adoption when both name and overview
  are locked, falling back to the non-adopting behavior: the stamp stays
  put, isCanonicalWrite goes false, and the fetch routes to the
  localization tables exactly like a non-adopting refresh in that
  language does today. One locked field still adopts — the other is
  actually rewritten.

- Multi-library flip-flop: an item in libraries with different metadata
  languages had its canonical base row rewritten to whichever library
  refreshed last, oscillating forever. Process now requires every
  library containing the item (media_item_libraries) to resolve to the
  adoption target before setting AdoptLanguage, via the existing
  GetDistinctMetadataLanguagesForItem (which applies the same empty→en
  default as resolveFolderLanguage). Disagreement or a lookup failure
  keeps the current stamp — stable beats flip-flopping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-07-02 14:07:21 -04:00
a4402d4621 fix(playback): stop seek-restart churn and premature paused-session reaping (#279)
* fix(playback): stop seek-restart churn and premature paused-session reaping

Two latent defects (present since the initial migration) explain the
cross-client freezes on resume-after-pause, intro-skip, and manual seek
reported in #243:

1. Dueling transcode restarts. TranscodeSession.Restart had no
   concurrency guard, and SegmentRecoveryDecision told concurrent
   segment requests to restart again while a restart was already in
   flight (the restart window runs with running=false, so requests hit
   the restart-happy transcode_not_running branch before the Restarting
   check). Pipelined HLS segment requests therefore spawned restarts
   that kept killing the ffmpeg the player was waiting on — a 30s stall
   and another restart, surfacing as buffer-then-freeze. Restart is now
   single-flight and the recovery decision waits out an in-flight
   restart (WaitForSegment already polls politely through one).

2. Paused sessions were reaped after 2 minutes, killing the transcode
   with no revival path — every resume request then 404s. Clients that
   stop progress reporting while paused (tvOS; backgrounded web tabs)
   froze on play after a >5 minute pause. Paused grace is now 30
   minutes in both the in-memory session manager and the DB session
   cleaner.

Also unifies post-restart re-arming: the throttler and exit monitor are
re-armed via a restart hook installed at session creation and fired by
Restart itself, so the jellycompat seek path (which previously re-armed
nothing) and the audio-switch path (which re-armed only the throttler)
now behave identically to web segment recovery.

Follow-up (not in this PR): lazily revive a reaped session's transcode
on resume instead of 404ing, so even multi-hour pauses recover.

Fixes #243

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(playback): scope the restart-hook comment to handler-created sessions

The comment above SetRestartHook claimed the hook covers restarts from
'jellycompat seek', but jellycompat creates its own TranscodeSessions via
playback.StartTranscode in a separate registry and calls Restart directly;
those sessions never get the hook and never had throttler/exit-monitor
wiring to re-arm. Reword the comment (and the matching test comment) to
scope the guarantee to handler-created sessions.

Also fix TestRestartInvokesRestartHook to resolve `true` via PATH instead
of hardcoding /bin/true, which does not exist on macOS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
2026-07-02 14:06:25 -04:00
866392fecd feat(notifications): announce new audiobooks and ebooks on server channels (#260)
Audiobook and ebook libraries previously never entered the Recently
Added pipeline: availability detection only ran for TV/movie/mixed
libraries and release_events only knew episode/movie kinds, so server
channels (Discord/generic webhooks) could not announce new audiobooks
or ebooks.

Generalize the movie path into a flat-item-kind registry
(internal/notifications/item_kind.go) driving availability detection,
recording, channel toggles, payload rendering, test fixtures, and the
admin backfill seeder. New kinds share a kind-discriminated
item_availability table; movie_availability stays as-is. Channels gain
notify_new_audiobooks/notify_new_ebooks toggles (default on, additive
API fields) and embeds carry the author from item_people. Flood-safe by
construction: existing libraries seed silently on their first
post-upgrade full scan.

Extract internal/librarykind to replace the is*LibraryType helper
copies that had drifted across scanner, libraryingest, and metadata
(metadata's movie check silently included mixed; now spelled
explicitly).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:42:28 -04:00
287ff3658f fix(jellycompat): keep still-playing direct-play sessions in the activity view (#262)
* test(jellycompat): implement ListProgressSince on userstore test fake

PR #258 added ListProgressSince to the userstore.UserStore interface but
missed the progressCountingStore stub, leaving the jellycompat test build
broken on main. Stub it with the same panic("unused") the sibling methods
use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(jellycompat): keep still-playing direct-play sessions in the activity view

An Infuse Static=true direct play skips PlaybackInfo, so its playback
reports carry a client-generated PlaySessionId. handlePlaybackReport only
resolved reports by exact PlaySessionId, so every report silently no-op'd:
the admin Activity View position froze, nothing refreshed the upstream
session's activity, and stale cleanup reaped the session ~45s in while the
client kept playing. Once reaped, ensureUpstreamPlayback early-returned the
dangling session id forever, so the stream card never came back.

Fix all three legs:
- handlePlaybackReport falls back to the same token-scoped route lookup the
  stream path uses (ItemId, then MediaSourceId) when the PlaySessionId is
  unknown, and revives a reaped upstream session when a progress report
  proves the client is still playing.
- HandleVideoStream marks an in-flight media transport around direct-play
  and remux serving, mirroring the native stream handler, so long-lived
  range transfers keep the session alive without progress reports.
- ensureUpstreamPlayback verifies the upstream session still exists before
  reusing it, recreating it under the same play session when it was reaped.

FindByRoute now compares RouteItemID with UUID normalization so report ids
match regardless of client dash/case formatting.

Closes #244

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(jellycompat): harden report fallback and upstream revive per adversarial review

Address three review findings on the issue #244 fix:

- Deterministic report binding: the stream path now records the client's
  own PlaySessionId (Static=true direct play never learns the server id)
  as an alias on the play session it binds, and playback reports resolve
  by that alias before falling back to the ambiguous item/source route
  scan. A Stopped report that only matched by bare route no longer tears
  the session down (it may not own it when the same item plays twice
  under one token); stale cleanup owns that path.
- Same-method revive closes any transcode still keyed to the reaped
  upstream id before recreating the session, so a second ffmpeg cannot
  start alongside an orphaned one.
- Attaching a new upstream session is now a compare-and-swap on the
  observed upstream id: the loser of a concurrent revive/range-request
  race stops its session and adopts the winner instead of leaving an
  orphan counted against the user's stream limits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(jellycompat): guard alias ambiguity and method switches in upstream attach

Second-round adversarial findings on the issue #244 fix:

- A client PlaySessionId alias now only resolves when it identifies
  exactly one live session for the token, and the candidate must agree
  with the report's ItemId/MediaSourceId. A reused or stale client id
  degrades to the route scan (progress still lands on the right item)
  instead of binding an arbitrary session, and an ambiguous Stopped
  report tears nothing down.
- A CAS loser only adopts the concurrent winner when it serves the same
  play method; a concurrent method switch surfaces 409 Conflict instead
  of continuing with mismatched transcode bookkeeping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(jellycompat): key audio-selection updates by the resolved play session id

Review follow-up (PR #262): after an alias or route fallback,
req.PlaySessionID is the client's own generated id and not a playback
store key, so the audio-track-change branch in handlePlaybackReport
silently dropped selection updates for Static direct-play sessions.
Use the resolved playSession.ID for the store mutation and logging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:37:32 -04:00
b250dbb59b fix(watchtogether): harden realtime sync, room lifecycle, and lobby UX (#273)
* fix(watchtogether): harden realtime sync, room lifecycle, and lobby UX

Remediates all findings from a deep review of the Watch Together feature.

Server:
- Serialize every websocket write (pong/error replies bypassed the write
  mutex, racing broadcasts on the same gorilla conn)
- Send room_closed with a reason on terminal connect failures so clients
  stop reconnecting to dead rooms
- Persist room state outside the service-wide mutex via a shared
  generation-CAS helper; drop ~450 lines of dead duplicated methods
- Measure transport latency from server-side ping/pong RTT (was one-way
  client-clock delta, poisoned by clock skew) and clamp the lead time
- Re-evaluate readiness when a waiting participant disconnects and add a
  30s waiting deadline that skips stragglers (activates ignoreWait)
- Guard the host-disconnect close timer against reconnect races
- Clamp buffering-report anchor moves; clear stale member sessions on
  selection change
- Janitor: evict empty live rooms and close rooms idle >24h
- Snapshot gains an additive members list with profile display names

Web:
- Surface terminal room errors (REST 404/410/403 and WS error codes) as
  closedReason instead of reconnecting forever on "Connecting..."
- Memoize the playback-sync hook and narrow VideoPlayer's video-listener
  effect deps to stop re-subscribing 13 listeners on every render
- Preserve invite-link destination through login/profile guards
- Lobby: terminal ended/missing-token states with CTAs, End-room confirm
  dialog, toast feedback via shared action helpers (dedup with player),
  participant list with guest Leave, mobile-visible connection status,
  document title, a11y labels/focus reveal, unified status dot component
- Delete dead useWatchTogetherRoom hook (345 lines, zero importers)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(watchtogether): guard indexed access in join-page keyboard nav for noUncheckedIndexedAccess

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(watchtogether): reconcile CAS conflicts without regressing newer state; roll back unpersisted waiting-resume

Addresses Codex review on PR #273:
- persistRoomChangeLocked now undoes the failed writer's optimistic
  generation increment and only adopts the refreshed database row when it
  is at least as new as the local copy, so a stale conflict refresh can
  no longer overwrite a concurrent writer's newer in-memory state (and a
  failed write can no longer leave a phantom generation)
- maybeResumeFromWaitingLocked restores the waiting state and re-arms the
  deadline when the resume transition fails to persist, instead of
  broadcasting a resume the database never recorded

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:34:02 -04:00
26089950af fix(player): update jassub so ASS subtitles stop rendering corrupted at original resolution (#263)
* fix(player): update jassub so ASS subtitles stop rendering corrupted at original resolution

At original resolution the web player renders ASS subtitles client-side via
JASSUB (lower resolutions transcode with server-side burn-in, which is why
dropping the quality worked around the bug). jassub 2.4.2's WebGL2 shaders
declare mediump precision, so on GPUs where mediump is fp16 the fragment
shader's texel coordinates quantize on large canvases and glyphs come out
sheared/banded (issue #209). Upstream fixed this in 2.5.5 ("always use highp
on webgl2"); 2.4.3–2.5.6 also fix wasm loading on current WebKit/Chromium,
subs not clearing between repaints, and zero-size resize handling.

The bump needs two companions:

- jassub >= 2.5.4 no longer ships its built-in default font file but still
  points at it, which would silently blank every track that relies on the
  default (we disable queryFonts). Vendor Liberation Sans (SIL OFL 1.1, the
  same file 2.4.2 shipped) and pass it via availableFonts so the missing
  asset is never referenced.
- Exclude jassub from Vite's dep optimizer (its module worker otherwise 404s
  in dev) and prebundle its two CJS deps for ESM interop.

Verified with a standalone harness (Playwright Chromium + WebKit 26.5):
2.4.2 fails to even instantiate its wasm on those engines, while 2.5.6
initializes and renders correctly with the vendored default font.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(player): colocate SIL OFL license with vendored Liberation Sans

Adversarial review flagged the redistributed font binary shipping without
its license text; copy the repo's existing Liberation license (same one
used for the pdfjs standard fonts) next to the asset and record provenance
in the import comment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:12:27 -04:00
d1fd7c236f fix(jellycompat): sync activity dashboard immediately on compat playback start/stop (#261)
* fix(jellycompat): sync activity dashboard immediately on compat playback start/stop

Compat (Infuse/Jellyfin) sessions were only reconciled into
playback_sessions_sync on the periodic 15s reconciler tick, so stopped
streams lingered in the admin activity dashboard and overlapped with
newly started ones as ghost sessions. Native playback handlers already
trigger an immediate SessionSyncer.SyncNow on start/stop; wire the same
syncer into the jellycompat playback handler and flush after
teardownPlaySession (Stopped report + ActiveEncodings teardown) and
after a new upstream session starts.

The stop-path sync detaches from request cancellation
(context.WithoutCancel) because clients often drop the connection right
after reporting a stop.

Also adds the ListProgressSince method to the jellycompat test fake so
the package's tests compile again after the downloads-v2 interface
change.

Part of #205

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(worker): serialize SyncNow and bound compat request-path session sync

Adversarial review follow-ups: SyncNow snapshots could commit out of
order when request-path syncs raced the periodic tick (an older
snapshot committing last would resurrect stopped sessions or drop
fresh ones), and the detached stop-path sync had no deadline, letting
a stalled DB pin request goroutines. Serialize snapshot capture +
reconcile under one lock and cap request-path syncs at 5s so failures
degrade to the periodic tick.

Part of #205

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(worker): coalesce immediate session syncs instead of queueing on a lock

Review iteration 2: a stop-triggered sync queued on a bare mutex could
expire its 5s deadline waiting behind a slow tick and fail before
removing the stopped row. Coalesce instead: one owner reconciles at a
time, callers that arrive mid-flight return immediately after flagging
a follow-up pass, and the owner re-captures a fresh (post-change)
snapshot afterwards. Request goroutines never block behind another
sync, snapshots still commit in capture order, and an expired-context
owner leaves the queued pass for the next tick instead of burning it.

Part of #205

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:07:09 -04:00
18e9856227 fix(web): detect HDR via video-dynamic-range media query for Firefox (#259)
* fix(web): detect HDR via video-dynamic-range media query for Firefox

Firefox's (dynamic-range: high) query reflects the browser canvas and
reports standard even on HDR-enabled displays, so the player capability
check always sent hdr=false. Firefox 116+ exposes the video plane via
(video-dynamic-range: high); accept either query. Unknown media features
parse as non-matching, so the extra query is safe in other browsers.

Closes #206

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(web): keep Window receiver when passing matchMedia to HDR helper

Invoking a detached matchMedia reference throws (Illegal invocation /
TypeError) in some browsers, crashing capability detection instead of
reporting HDR. Wrap it in an arrow so the Window receiver is preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 00:13:51 -04:00
57fbc1e0b4 perf(literaryworks): cut Postgres load on ebook↔audiobook auto-linking (#253)
* perf(literaryworks): cut Postgres load on ebook↔audiobook auto-linking

Rescans and candidate matching were driving high Postgres CPU on book
libraries.

- AutoLinkContent now checks literary_work_items with a cheap indexed
  lookup before GetMatchItem, so unchanged already-linked books skip the
  heavy triple-lateral query on every rescan.
- ListMatchCandidates is split into an indexable ID-selection phase
  (title / provider EXISTS / series EXISTS) and a hydration phase, so the
  per-row lateral aggregates run only for the LIMIT candidates kept rather
  than for every opposite-format book.
- Add a partial LOWER(title) index scoped to ebook/audiobook so the OR of
  title/provider/series filters can be driven entirely by indexes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(migrations): drop invalid leftover index before concurrent rebuild

An interrupted CREATE INDEX CONCURRENTLY (cancel/restart) can leave an
INVALID idx_media_items_books_title_lower; IF NOT EXISTS would then skip
the rebuild while Goose records success. Add the preflight invalid-index
drop used by the repo's other concurrent-index migrations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 23:21:25 -04:00
9cae868a27 feat(downloads): offline sync for mobile — downloads v2 (#258)
* feat(downloads): offline sync for mobile (downloads v2)

Replace internal/download with a unified internal/downloads package and add
fully-offline download + watch-sync support for mobile clients, across five
independently-shippable phases:

- Phase 0: reshape the downloads table and the /downloads contract to be
  device- and format-aware; add GET /downloads/capability; extend
  DownloadConfig (default-off keys); update the web download hooks/components in
  lockstep. This is the one approved pre-lock exception to the additive-only
  /api/v1 rule (the web app is the only consumer and is updated together).
- Phase 1: managed device-library entries (create/list/PATCH/delete/serve),
  keyed on the X-Silo-Device-Id header.
- Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that
  strip every presigned URL (inline thumbhashes + authenticated proxies).
- Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable,
  leased artifact queue with startup recovery, hosted on the task manager;
  playback.PrepareFile emits one +faststart MP4. Adds the admin transcode
  toggle and per-artifact LRU cleanup.
- Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a
  server-assigned synced_seq cursor on watch_progress; an optional clamped
  updated_at on POST /sync/progress and an opaque ?since= cursor on
  GET /progress (additive; existing callers unaffected).

Security & reliability invariants, each with an acceptance test:
1. Server-owned sync ordering: ?since= delta delivery is driven only by the
   server-assigned synced_seq; the client clock is bounded (event_at, clamped
   to now+skew) and used only for last-write-wins on the caller's own profile.
2. Full profile+device authorization on every managed endpoint, with a
   per-profile content/library access re-check before serving any bytes/assets.
3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP
   LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no
   crash strands a download in preparing and concurrent workers never
   double-encode.

Migrations are timestamped Goose files: reshape downloads (device/format);
download_artifacts (durable queue); watch_progress event_at/synced_seq.

DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI;
the invariant-1 progress test also runs against the real SQLite backend locally.

Client repos (silo-android, silo-apple) consume the reshaped /downloads/*
contract and the updated_at/?since= progress fields and require coordinated
follow-up.

Implements the maintainer-approved v1 capability proposal for offline sync
(downloads v2).

AI-use disclosure: implemented by Claude (Claude Code) from the approved design
doc under docs/superpowers/specs, with human review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(downloads): series & season downloads + client-pull monitoring

Build season downloads and a "monitor a series" capability on top of the
downloads v2 (offline sync for mobile) work.

Season downloads:
- POST /downloads accepts season_number (with series:true) to download one
  season. CreateSeries/CreateSeason share one body via a listEpisodes closure
  and register managed entries under a shared batch_id (original-only). Episode
  files are resolved in a single batched query.

Series monitoring (auto-download), client-driven:
- New device-scoped download_subscriptions table with a Sonarr-style mode
  (all | future | latest_season | specific_seasons), a client-enforced
  delete_watched flag, and a max_storage_bytes cap. The server never deletes
  on-device files; retention and the hard cap are the client's, the server
  only soft-gates registration.
- The client calls POST /downloads/subscriptions/sync on open / background
  refresh; the server registers the in-scope, not-yet-downloaded episodes
  (idempotent via the managed-entry unique index) and the device pulls them on
  its own schedule. No background worker and no dependency on the notifications
  subsystem. latest_season follows new seasons (>= subscribe-time season);
  future excludes the back catalog via air date.
- Subscription CRUD + sync are profile+device authorized (device id from the
  X-Silo-Device-Id header only) with a per-request content-access re-check. The
  capability endpoint advertises season_download / series_monitoring /
  monitoring_modes.

Also lands the downloads-v2 work already present in the tree: durable artifact
(remux/transcode) preparation and offline watch-progress reconciliation, plus
the design-spec updates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync

* test(downloads): fix deterministic ID collision in reconcile test

Artifact IDs are time-sortable, so two artifacts created in the same
moment share their first 8 chars; combined with a captured timestamp the
two preparing-download IDs collided on downloads_pkey. Use the full
artifact ID, which is unique per row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): support sqlite userdb backend for managed downloads

With the sqlite userdb backend, profiles live only in per-user SQLite
stores and public.user_profiles stays empty, so user_devices'
profile FK made every managed create/subscription/offline-sync request
fail with an FK violation. Drop the FK (shared Postgres tables must not
FK profile tables — same rule as notifications) and replace the lost
cascade with an app-level purge on profile deletion, wired through
ProfileHandler for both backends. DB-backed regression tests cover the
no-Postgres-profile-row path and the purge cascade.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): dispatch encode kick asynchronously

triggerDrain invoked the kick inline, and the kick (taskmanager RunTask)
executes the encode task on the caller's goroutine — so a POST
/api/v1/downloads with a bitrate quality blocked the HTTP request on the
entire queue drain, ffmpeg encodes included, delaying the 202 by minutes
on an idle queue. Dispatch the kick on a goroutine; the task manager
already serializes concurrent runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): enforce per-user quota on the encode pipeline

Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared
downloads: artifact-backed rows are created in 'preparing' (never
'queued'/'downloading'), which CountActiveByUser didn't count, and
createArtifactDownload enqueued the encode job before limiter.Check, so
even a 429-rejected request left a job the worker would transcode.
Count 'preparing' as active and check the limiter before Ensure; managed
replacements stay quota-exempt since they don't add a row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): protect ephemeral artifact links from LRU eviction

HasActiveLink only counted managed (device_id IS NOT NULL) rows, so
under a byte budget Cleanup could delete an artifact still referenced by
a ready-but-unfetched ephemeral web download — permanently 404ing a row
the API kept listing as ready (the artifact row is gone, so recovery
can't re-queue it). Any non-terminal link now protects the artifact;
only artifacts whose links are all cancelled/failed/revoked are
evictable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): batch manifests skip bad entries instead of failing whole batch

One deleted or access-filtered episode made GET
/downloads/batches/{id}/manifests 404 for the entire season, so a
client could no longer fetch manifests for the still-valid entries.
Report unbuildable entries in a skipped[] array (revoked | not_found |
error) alongside the delivered manifests, mirroring the create path's
skip idiom. Also cut the batch cost: the shared series detail is
resolved once per batch instead of once per episode, and buildSubtitles
reuses the already-loaded media file instead of re-querying it per
manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(migrations): wrap DO block in StatementBegin/End markers

Under NO TRANSACTION goose splits statements on semicolons, so the
dollar-quoted DO block failed every fresh install with 'unterminated
dollar-quoted string' (SQLSTATE 42601). Already-applied databases are
unaffected. Same fix is being applied to main; identical content merges
cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api): allow season 0 (Specials) in season downloads

season_number was a plain int dispatched with '> 0', so requesting the
Specials season was indistinguishable from omitting the field and
silently broadened to a full-series download. Dispatch on pointer
presence, treat 0 as the Specials season, and reject negatives with 400.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): capability quality_presets is never JSON null

PresetsFor returned a nil slice when downloads are disabled or the user
lacks the permission, and Capability's []string{} initialization was
immediately overwritten by it — so GET /downloads/capability serialized
"quality_presets": null where the contract documents an array.
Normalize at the source so every caller inherits the guarantee.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): subscription sync correctness + batched registration

Three subscription fixes:

- A paused subscription no longer syncs: PATCHing scope (or pausing and
  changing scope in one request) registered episodes for a monitor the
  user had just stopped, inconsistently with SyncSubscriptions' guard.
- SubModeFuture compares calendar days (UTC): air_date is date-only, so
  the strict instant comparison permanently excluded episodes airing the
  same day the user subscribed; episodes with no air date now fall back
  to their ingest time instead of never registering.
- Registration is one batched fetch (GetManagedEntriesByKeys) plus one
  batched INSERT ... ON CONFLICT DO NOTHING RETURNING
  (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode —
  a 300-episode series cost ~600 sequential round trips per request and
  every no-op sync re-walked the full set. RETURNING yields exactly the
  new rows, so the sync response's 'registered' count now honestly
  reports 0 in the steady state instead of the full in-scope count on
  every app open. The now-unused InsertManagedEntryIfAbsent is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(userstore): stamp triggers own the event_at LWW key

MarkProgressBatch (jellycompat series mark-played) advanced updated_at
but never event_at, and both stamp triggers only defaulted event_at when
NULL — so a queued offline event with a client time between the row's
old event_at and the mark could win SetProgressIfNewer and resurrect a
stale resume position that then re-synced to every device.

Make the triggers authoritative instead of adding a tenth hand-written
SET clause: whenever an UPDATE changes updated_at without explicitly
changing event_at, the trigger advances the LWW key; writes that do set
event_at (offline sync's clamped client event time) keep their value.
Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb
migration that drops and reinstalls the trigger bodies (CREATE TRIGGER
IF NOT EXISTS never replaces). Conformance tests cover both batch paths,
the preserved-client-time invariant, and the v11→v12 upgrade.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps

Migrations: fold the 20260621 corrective migration back into the base
Downloads V2 migrations (its columns/constraints already exist there)
and fix the reshape Down, which re-added the narrow status CHECK without
collapsing managed-lifecycle rows first — rollback aborted on any DB
with preparing/ready/revoked rows; validated against a live row. Branch
databases that applied the corrective migration need its version row
removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459.

Code: drop the dead 'registered' status (nothing ever wrote it; the
lifecycle is preparing -> ready; 'revoked' stays reserved for the
planned admin revoke flow) along with unused KindDirect and
ErrInvalidFormat.

Sweeps: Cleanup now runs an age-based hygiene pass independent of the
byte budget — cold terminally-failed artifacts (with .part leftovers),
orphaned ready artifacts no download row references, and ephemeral web
rows older than their convenience-record lifetime (also unpinning their
artifacts and bounding GET /downloads growth). The byte budget remains
the disk quota per the limits & restrictions design.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff

Document the contract changes from the review fixes: batch-manifest
skipped[] shape, honest subscription 'registered' semantics, season 0 =
Specials, always-array quality_presets, bytes_sent actual behavior,
ephemeral 7-day retention, header-pairing requirement, progress-delta
deletion caveat, and the ready/failed push event schema (new §9.4).
Add an Android client handoff section (§11) mirroring the Apple one,
register HEAD on /downloads/{id}/file for download stacks that probe
before ranged GETs, and add season_number to the web create-request
type. Flag the /direct-download session-token-in-URL tradeoff; a
short-lived download-scoped URL is a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: consolidate download/progress helpers, prune dead code, gate sweeps

Behavior-preserving consolidation from the Downloads V2 review:

- appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection,
  shared by the HLS builder and the single-file prepare builder (the
  drift pattern that already bit tone-mapping once).
- userstore.ResolveProgressState: one home for the min-resume/watched
  threshold rule, replacing five identical copies across both store
  backends and the offline-sync ingest.
- Download file selection ranks resolutions via access.CompareQuality
  (adds 4320p, agrees with playback) instead of a private switch.
- writeSubtitle uses the shared subtitles.SubtitleContentType mapping.
- config.DefaultTranscodeDir replaces three '/tmp/silo-transcode'
  literals.
- Read-side quality/revision defaulting helpers removed: insertArgs plus
  the NOT NULL/CHECK schema already guarantee the invariant.
- Dead code removed: Repository.ListByUser, SubscriptionRepository.
  ListActiveBySeries, and the stale auto-register-worker comments (the
  design is client-pull; no worker exists).
- Redundant left-prefix indexes dropped from the base migrations (their
  unique indexes serve the same prefixes).
- recover()'s disk-presence sweep and the stale-row hygiene sweep run on
  startup then hourly instead of every 30s tick (both are O(cache
  size)).
- gofmt/prettier fixes for pre-existing drift in handlers/playback.go
  and pages/Profiles.tsx.

Deferred (noted for follow-ups): quality-ladder preset table collides
with the drafted download limits & restrictions design, which specifies
its own ladder helper; Download-literal construction consolidation and
the managed-identity value object remain open.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(downloads): draft download limits & restrictions design

Design input for the follow-up v1 capability proposal (quality ceiling,
batch size cap, per-user quantity/bandwidth overrides). Committed with
downloads v2 because the remediation work explicitly defers the quality
ladder refactor and revocation wiring to this spec.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(progress): reject malformed updated_at; clamp negative progress inputs

Review findings on #258:

- A malformed (non-RFC3339) updated_at in POST /sync/progress previously
  parsed to the zero time, which clampEventAt treated as "now" — letting a
  stale offline event win LWW as a fresh server-time write. The item is now
  rejected with a per-item error instead.
- ResolveProgressState now clamps negative position/duration before
  classification so no backend can persist negative progress through
  UpdateProgress/SetProgress.
- The online-write event_at invariant test is table-driven over both
  SetProgress and UpdateProgress, which share the same contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests

Review findings on #258:

- UpdateSubscription now applies the same feature/DownloadAllowed gate as
  CreateSubscription and SyncSubscriptions; a PATCH could previously
  re-activate or widen a monitor and register managed rows after an admin
  disabled downloads or revoked the user.
- Serving download bytes (managed and ephemeral) and /direct-download now
  mirror playback's per-file authorization via catalog.FileAllowedByAccess:
  library scope and the profile's max playback quality are re-checked at
  serve time, with artifact-backed rows checked against the artifact's
  resolution (a 720p transcode of a 4K source stays servable under a 1080p
  ceiling).
- Offline manifests for remux/transcode entries now describe the prepared
  artifact (container, codecs, resolution, single selected audio track)
  instead of the catalog source file the client never receives.
- ArtifactRepository.Requeue reports ErrNotFound when the row was
  concurrently swept; ArtifactManager.Ensure recreates the job in that case
  instead of linking downloads to a dead artifact id.
- "No downloadable episodes" is a sentinel (mapped to 404
  no_downloadable_episodes) rather than a bare error that surfaced as 500.
- Subscription season_numbers are bounds-checked (0–9999) before the int32
  narrowing in the repo could silently wrap them.
- HandlePatchDownload reuses requireManaged instead of hand-rolling the
  same managed-identity checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 22:05:36 -04:00
1e8f79fe18 fix(api): surface ListDownloadedSubtitles errors instead of masking as 404 (#256)
The web subtitle handler wrapped the downloaded-subtitle lookup in
`if err == nil { ... }`, so a real DB/backing-store failure from
ListDownloadedSubtitles was silently swallowed and control fell through
to a generic 404 ("Subtitle track not found") with no server-side signal.
This made genuine internal failures look like an intermittent client-side
"subtitles won't render" bug and left nothing in the logs to diagnose.

On error, log at ERROR (with file_id/track/error, matching the sibling
font-extraction path) and return 500 — mirroring the neighbouring error
handling in the same branch (S3 GetObject -> 502, ConvertToVTT -> 500).
The genuine not-found fall-through is preserved for the case where the
listing succeeds but the requested index is out of range.

Closes #248

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 18:13:46 -04:00
QuickandClaude Fable 5 9c6252dace chore: gitignore docker-compose.local.yml dev-deploy override
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 18:11:57 -04:00
8b3ebc78b6 fix(api): show symlinked directories in folder browser (#257)
os.ReadDir reports lstat semantics, so a symlink pointing at a directory
has IsDir() == false and was silently dropped by HandleBrowse. Library
folders reached through directory symlinks (e.g. a Docker bind mount whose
entries link into a FUSE mount) never appeared in the folder picker even
though the targets were valid and browsable.

Resolve non-directory entries with os.Stat to follow the link and include
them when the target is a directory; entries whose target cannot be stat'd
(broken or looping links) are skipped rather than erroring the request.
Regular directories keep the lstat fast path and take no extra syscall.

Closes #208

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 18:10:56 -04:00
2486345679 Reduce metadata image cache R2 churn (#249)
* Reduce metadata image cache R2 churn

* Preserve image cache failure cooldowns

---------

Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
2026-07-01 17:37:06 -04:00
eb35141ecc fix(watchsync): correct Trakt TV episode sync payload (#254)
* fix(watchsync): omit empty Trakt episode ids so series mark-watched targets the right show

omitempty on a struct value is a no-op in encoding/json, so the history
export sent all-zero episode ids ({tmdb:0,tvdb:0}); Trakt matched the
degenerate id to one default show, mis-recording every watched series.
Make episode IDs a *traktIDs pointer and attach it only when a real id
exists, else use the show + season/number fallback (mirroring scrobble).
Adds a debug log on the show-fallback path and payload tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(watchsync): sync TV episodes to Trakt via valid nested shows payload

Episode history exports were dropped by Trakt because they were emitted
into the flat episodes[] array with a bogus sibling show object plus
season/number keys — a shape the Trakt API does not accept, so it
silently discarded them (200/201 with no history recorded). Movies were
unaffected since they always carry their own external IDs.

Two coordinated changes:

- watchstate/identity.go: ResolveHistoryIdentity now carries the
  episode's own imdb/tmdb/tvdb IDs (already stored on the episodes
  table) so episodes with real IDs export via the flat episodes[].ids
  form, matching how movies work.

- watchsync/providers/trakt/provider.go: episodes without their own ID
  now serialize into the correct nested shows[].seasons[].episodes[]
  structure keyed by the show's IDs, merging episodes by show/season.
  Same fix applied to the history-remove payload. Empty-payload guards
  account for the new shows[] list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(watchstate): keep episode identity when episode has its own IDs

Address CodeRabbit review on PR #254: the episode identity builder
dropped the whole identity whenever series IDs were empty, so episodes
with a valid episode IMDb/TMDB/TVDB ID but no series IDs never reached
the flat episodes[].ids Trakt path. Only require series IDs when the
episode has no IDs of its own (nested show fallback).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 17:36:47 -04:00
cf0db385f3 Add Apple push notifications support (#255)
* Add push notifications support

* fix(notifications): address push notification review findings

- Gate the capability endpoint's apple_push availability on the admin
  delivery toggle, matching web push: Available now means setup will
  actually deliver.
- Reject direct admin writes to push_relay_deployment_id/api_key; the
  relay issues them as a pair during registration and a lone write
  desyncs them (and poisons the next rotation request).
- Purge a device's registrations under other profiles when it
  re-registers, so a profile switch on a shared device stops the old
  profile's pushes (attempts cascade); adds a DB-backed test.
- Extract the shared channelDispatcher core + retry sweep and rebuild
  the webhook/web push/Apple push dispatchers on it instead of keeping
  three copies of the worker-pool/retry loop.
- Deduplicate relay URL validation (admin setting + register flow) and
  the push outbox attempt-building loops behind shared helpers.
- Cap free-text decline reasons in notification display bodies.
- Fix TestHandleApplePushDisplayDB expectations to match the shared
  display copy (test previously failed under SILO_TEST_DATABASE_URL).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(notifications): route push relay URL writes through registration only

Direct writes to notifications.push_relay_url via the admin settings
endpoint bypassed the relay registration flow, letting the stored URL
drift out of sync with the deployment id / API key pair the relay
minted for it. Reject the URL alongside the deployment id and API key
in the settings handler; POST /admin/notifications/push/relay/register
remains the only path that persists all three together.

The admin UI's Relay URL field now edits local draft state and is
applied by the Register/Rotate action instead of the settings save.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 17:25:16 -04:00
eafa22a3c3 fix(jellycompat): improve audio track selection and display titles (#240)
- preferredAudioStreamIndex: when default track is TrueHD/MLP and device
  profile doesn't explicitly list that codec, falls back to best compat
  track (EAC3 > AC3 > DTS > AAC…), same language preferred. Prevents
  Android TV clients receiving undecoded lossless passthrough streams.
- audioTrackDisplayTitle: structured display title (e.g. "English - EAC3 5.1")
  replacing the raw firstNonEmpty fallback. Suppresses generic embedded
  labels (Stereo, Surround) that add no information.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-01 09:06:38 -04:00
CoffeeKnyteandGitHub 652b5d1e9f perf(jellycompat): fast-paths for slow Jellyfin-compat queries (#247)
* perf(jellycompat): fast-paths for slow Jellyfin-compat queries

Rebuilds the jellycompat slow-query fast-paths cleanly on main, without
the hub-sections virtual-library feature (which is intentionally absent
on this branch). Each path is bounded so behavior stays predictable
under load.

Fast-paths:
- Episode catalog: add the load-bearing `si.type = 'series'` hydration
  guard plus a matching entry-scan series-parent guard, so the page and
  count scans stay aligned with hydration (no under-filled pages, no
  inflated TotalRecordCount from podcast/audiobook episode rows).
- /Shows/{id}/Episodes AdjacentTo: serve a bounded prev/self/next
  neighbor window with series-membership validation and a cross-series
  id guard, falling back to the full listing.
- Resume: serve Continue Watching through the capped native
  continue-watching fetcher (next-up suppressed, in-progress only)
  instead of loadProgressPage's unbounded scan. Wires only the
  sections read-time fetcher (StoreProvider) into ItemsHandler; no
  hub-section/virtual-library exposure.
- Played-history: push the type + library filter down into SQL.
- recently_added: fan a no-parentId browse out across libraries with an
  app-side dedup merge, clamped to BrowseFilters.MaxLimit, and exclude
  compat-hidden (audiobook/podcast) libraries from the fan-out.
- Detail fields: batch per-item GetItemDetail in the Latest/browse
  detail-field paths via GetItemDetailsByIDs, with per-item fallback.
- Search: term-length and result-count guards for aux search.

* perf(jellycompat): clamp Resume fast-path fetch to maxResumeItems

A large client-supplied StartIndex on /UserItems/Resume inflated
loadResumeViaSections' fetchLimit (pageSize + StartIndex) and fed it
straight into ResolvedSection.ItemLimit, letting the continue-watching
fetcher scan up to its continueProgressMaxScanned hard cap (1000 rows)
instead of stopping near maxResumeItems. Clamp fetchLimit to
maxResumeItems and short-circuit to an empty page when StartIndex is at
or beyond the cap (no visible row can land there), keeping the capped
fetcher's scan bounded and predictable under load.

Addresses CodeRabbit review finding on PR #247.
2026-07-01 09:06:24 -04:00
CoffeeKnyteandGitHub 4fb8f6a711 fix(metadata): merge duplicate people on refresh instead of looping on 23505 (#250)
* fix(metadata): merge duplicate people on refresh instead of looping on 23505

The background person-refresh worker re-selected the same people every cycle.
When a refresh resolved an external id (tmdb_id/imdb_id) already held by another
people row, the UPDATE violated a partial-unique index and raised SQLSTATE
23505; the tx rolled back so updated_at never advanced and FindCandidates
re-qualified the row forever. The underlying cause is two people rows for the
same human created from credits ingested with disjoint id sets (one tmdb-only,
one imdb-only) that BatchFindOrCreate never reconciled.

PersonRepository.Update now reconciles the collision instead of failing:

- The common no-collision path stays a single plain transaction (no added cost).
- On a 23505, updateResolvingConflicts runs the whole reconciliation in ONE
  transaction, retrying the write via savepoints so it commits atomically: it
  locks both rows FOR UPDATE in id order, then either merges the partner into
  the survivor (repoint item_people skipping duplicate credits, fold the
  partner's ids/fields onto the survivor, delete the partner) or, when the rows
  are not confidently the same human, drops just the conflicting id.
- canMergePeople requires compatible ids AND matching names, so a provider that
  hands the same id to two genuinely different people cannot trigger a
  destructive delete; that case falls to the non-destructive drop.
- A row merged away concurrently surfaces as pgx.ErrNoRows, which the refresh
  service maps to ErrPersonNotFound.

Existing stuck rows self-heal: they are still re-selected each cycle and now
merge (or drop) instead of looping, draining the warning population to zero.

Adds unit tests for the merge-decision logic (guard, compatibility,
field-folding, constraint mapping).

AI-use disclosure: implemented and adversarially reviewed with AI assistance.

* fix(metadata): preserve survivor's existing id when declining a person merge

The non-mergeable branch of resolveExternalIDConflict blanked the
conflicting external-id field before retrying the write. Because
execPersonUpdate is a full-row UPDATE, the retry persisted an empty
string and returned success, silently dropping a previously-valid
provider id (e.g. the admin PATCH path that mutates an existing id into
a colliding value) instead of leaving the row unchanged as the 23505
did.

Restore the locked survivor's currently-persisted value for the field so
the retried write is a no-op on that column: it commits without looping,
without deleting a possibly-distinct person, and without blanking an id
the survivor already held. Writing a row's own current value back can
never violate the unique index, so the field will not re-trigger the
conflict. The refresh-worker path is unchanged (survivor value is empty).

Convert clearExternalIDField into a general setExternalIDField setter and
extend its unit test to cover set-to-value and set-to-empty.
2026-07-01 09:05:58 -04:00
CoffeeKnyteandGitHub 1dbcf2cd9a fix(jellycompat): guard aux search paths and index-back person search (#252)
* fix(jellycompat): guard aux search paths and index-back person search

Short, recursive type-ahead terms (e.g. a single "a") against the
PostgreSQL people index and the in-memory collection/box-set filter
produced multi-second scans that pegged the server when a client fired
one search per keystroke. Every jellycompat search path except the
Meilisearch-backed /Items media search now rejects a provided SearchTerm
shorter than 3 runes without touching any backend, and caps results at
20 regardless of the client-requested Limit:
- /Persons        (PostgreSQL people scan)
- /Search/Hints   (catalog search)
- /Items BoxSet   (in-memory collection filter)

Shared policy + helpers live in search_guard.go (auxSearchMinTermLen=2,
i.e. reject 1-2 runes / allow 3+, and auxSearchMaxResults=20) with unit
coverage. The 3-rune floor is deliberate: 3 runes is the point where a
pg_trgm trigram index becomes usable, so the gate lines up with the index
and still lets legitimate short titles/names ("300", "Saw", 3-letter
actors) and 3-char type-ahead hints through.

PersonRepository.Search filters with `name ILIKE '%'||$1||'%'` rather
than `LOWER(name) LIKE '%'||LOWER($1)||'%'`. The old expression filtered
on LOWER(name), which the trigram GIN index idx_people_name_trgm (built
on name) could not serve, so every search fell back to an ordered index
scan on idx_people_name that walked the whole table for rare terms
(~300-400ms on the 889k-row production people table). ILIKE on name lets
pg_trgm serve rare 3+ char terms from the trigram index, while the planner
still picks the ordered btree scan with early termination for common
terms. ILIKE is case-insensitive, so behavior is preserved (including the
pre-existing treatment of % and _ in the term as LIKE wildcards).

Verified on production silo-postgres (889,302 people), parameterized query
under both custom and generic plan cache modes:
  rare 3-char 'qzx':  304ms -> 1-8ms   (trgm bitmap index)
  rare 4-char 'zzzz': 329ms -> 1-2ms   (trgm bitmap index)
  common 3-char 'ann':  15ms -> 7-74ms (btree early-stop / trgm bitmap)
New worst case across all terms is ~83ms (generic-plan common 3-char).

* fix(jellycompat): stop gating meilisearch hints, clamp box-set search

Two review follow-ups on the aux-search guards:

- HandleSearchHints is served by the catalog (Meilisearch-backed) search
  provider, which already bounds and short-term-handles its own results.
  Gating it with auxSearchTermTooShort contradicted the guard's own
  "non-Meilisearch paths only" policy and hid valid 1-2 char titles
  ("Up", "It") from global type-ahead. Drop the too-short gate there;
  keep the empty-term check and the result clamp.

- handleBoxSetsList gated short terms but passed query.limit straight to
  slicePage, which treats <=0 as no cap, so a box-set *search* was
  uncapped unlike Persons/Hints. Clamp the limit when a search term is
  present; empty-term browse keeps its client paging window.
2026-07-01 09:05:41 -04:00
Quick 75cf5e3b11 fix(nextup): include completed or partially watched items in NextUp query 2026-06-29 10:46:22 -04:00
QuickandGitHub 8afe783277 fix(jellycompat): honor StartItemId episode queues (#242)
* fix(jellycompat): honor StartItemId episode queues

* fix(jellycompat): compact episode queue models
2026-06-27 22:02:54 -04:00
QuickandGitHub 9cea2bb4f0 fix(metadata): enable AI translation on season and episode pages (#238) 2026-06-27 21:44:28 -04:00
QuickandGitHub a0fbe2cdda [codex] feat(jellycompat): use catalog search provider (#236)
* feat(jellycompat): use catalog search provider

* fix(jellycompat): route video search buckets to provider
2026-06-27 20:10:48 -04:00
290 changed files with 31725 additions and 4052 deletions
+1
View File
@@ -0,0 +1 @@
github: quick104
+3
View File
@@ -64,10 +64,13 @@ logs/
.playwright-mcp/
demo/
docker-compose.dev.yml
docker-compose.local.yml
# Local-only deployment override (unpublishes bundled redis/postgres host
# ports via `ports: !override []`). Kept out of git so a rebase from main
# never disturbs it and it never lands in a PR; its absence once exposed
# Redis to the internet, so it must persist on the box.
docker-compose.override.yml
# Local compose customization layered onto docker-compose.yml by make dev-deploy.
docker-compose.local.yml
.playwright-cli/
output/
+112
View File
@@ -0,0 +1,112 @@
# Developing Silo
This document covers building, running, and contributing to the Silo server. If you just want to run Silo, see the [README](README.md).
See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.
## Prerequisites
- **Go** 1.24+
- **Bun** 1.0+
- **PostgreSQL** 18+
- **FFmpeg** (for transcoding support)
## Local Development
Local development remains intentionally separate from the deploy-oriented compose setup. Use [docker-compose.yml](docker-compose.yml) for local services and the source-build workflow below.
```sh
# Start local PostgreSQL and Redis
docker compose up -d postgres redis
# Run the frontend dev server (hot reload, proxies API to :8090)
make dev-frontend
# Run the Go backend
make dev-backend
```
The main compose file expects `MEDIA_ROOT` to be set even if you only want the bundled PostgreSQL and Redis services, so set that in `.env` first (`cp .env.example .env`).
If you are developing `Silo` and `silo-plugin-sdk` together, keep using the local [`go.work`](go.work) workspace. That workspace is a developer convenience only. CI and release builds run with `GOWORK=off`, so any new SDK helper used here must be pushed and tagged in `silo-plugin-sdk` before this repo can merge or release the change.
Plugin authors should start with [docs/architecture/plugin-development.md](docs/architecture/plugin-development.md), which covers the RPC plugin package format, generated proto workflow, SDK import paths, route and asset exposure, and auth or user-config integration points.
## Make Targets
| Target | Description |
|---|---|
| `make build` | Build frontend + Go binary |
| `make frontend` | Build frontend only |
| `make dev-frontend` | Vite dev server with HMR |
| `make dev-backend` | Run Go backend (integrated mode) |
| `make dev-proxy` | Run a standalone proxy node |
| `make dev-transcode` | Run a standalone transcode node |
| `make migrate-create NAME=add_thing` | Create a timestamped Goose SQL migration |
| `make migrate-validate` | Validate Goose migration files without touching a database |
| `make migrate-status` | Show Goose migration status using Silo's bootstrapping runner |
| `make migrate-up` | Apply pending Goose migrations using Silo's bootstrapping runner |
| `make clean` | Remove build artifacts |
## Database Migrations
PostgreSQL schema migrations are managed by Goose. Migration SQL files live in
`migrations/sql/` and use Goose annotations. Converted legacy migrations keep
their original numeric versions so existing `schema_versions` rows can bootstrap
cleanly into Goose without replaying old SQL. New migrations should be created
with timestamped filenames:
```sh
make migrate-create NAME=add_thing
make migrate-validate
```
Do not run `goose fix`; timestamped migrations are the repository policy because
they avoid version collisions across parallel PRs. The existing `001`-style
files are historical compatibility records, not the naming pattern for new work.
Runtime migrations are applied by the integrated/API server only. Proxy and
transcode modes never mutate schema.
For existing installs, use `make migrate-status` and `make migrate-up` rather
than invoking the Goose CLI directly; those targets copy legacy
`schema_versions` rows into `public.goose_db_version` under the migration lock
before reading or applying migrations. Set `ENV_FILE=path/to/.env` when the
database URL should be read from a non-default env file.
## Running Tests
```sh
# Go tests (uses testcontainers — Docker must be running)
go test ./...
# Frontend tests
cd web && bun test
```
## Linting
```sh
# Go
golangci-lint run
# Frontend
cd web && bun run lint
cd web && bun run format:check
```
## Project Structure
```
cmd/silo/ Entry point
internal/
api/ HTTP router, handlers, middleware
auth/ JWT authentication and sessions
catalog/ Media item, episode, season repositories
config/ YAML + env var configuration
jellycompat/ Jellyfin/Emby protocol compatibility
metadata/ Plugin-driven metadata matching and enrichment
playback/ Direct play, remux, transcode session management
scanner/ Media file discovery and FFProbe
worker/ Background jobs (scan, match, reconcile)
web/ React + TypeScript frontend (Vite, Tailwind, shadcn/ui)
migrations/sql/ Goose-managed PostgreSQL schema migrations
```
+100 -177
View File
@@ -1,8 +1,17 @@
# Silo
A self-hosted media streaming server with a React frontend and Go backend. Supports direct play, remuxing, and hardware-accelerated transcoding. Includes optional Jellyfin/Emby-compatible app support for clients such as VidHub and Findroid.
Silo is a self-hosted media streaming server for your movies, shows, music, and books. Point it at your media folders and stream to your devices — at home or away — with direct play, remuxing, and hardware-accelerated transcoding handled automatically.
Join the community on [Discord](https://discord.gg/4RxuUQAEnW).
Join the community on [Discord](https://discord.gg/4RxuUQAEnW). If Silo is useful to you, consider [sponsoring the project](https://github.com/sponsors/quick104) — see [Supporting Silo](#supporting-silo).
## Highlights
- **Plays your media, your way** — direct play when the device supports it, remux or hardware-accelerated transcode (including NVENC) when it doesn't.
- **Web app included** — a full-featured web client and admin interface ship with the server.
- **Works with apps you already use** — optional Jellyfin/Emby-compatible API supports clients such as VidHub, Findroid, and Infuse.
- **Household profiles** — multiple profiles per account, with per-profile watch state and parental controls.
- **Plugin-driven metadata** — match and enrich your libraries with providers like TMDB and TVDB, installed as plugins.
- **Fast setup** — one `docker compose up -d` brings up the whole stack; everything else is configured in the admin UI.
## Deploy with Docker (recommended)
@@ -105,165 +114,6 @@ Silo is externally stateful by default rather than fully stateless. Durable appl
Migrating an existing Continuum Docker install should be done with the preflight
helper and cutover guide in [docs/continuum-to-silo-docker-migration.md](docs/continuum-to-silo-docker-migration.md).
## Build from Source
### Prerequisites
- **Go** 1.24+
- **Bun** 1.0+
- **PostgreSQL** 18+
- **FFmpeg** (for transcoding support)
### Quick Start
1. **Start PostgreSQL** (skip if you already have one running)
```sh
docker compose up -d postgres redis
```
The main compose file still expects `MEDIA_ROOT` to be set even if you only want the bundled PostgreSQL and Redis services, so set that in `.env` first.
2. **Configure the database connection**
```sh
cp .env.example .env
```
Edit `.env` and set `DATABASE_URL` to point to your PostgreSQL instance.
3. **Build and run**
```sh
make build
./silo
```
The server starts at `http://localhost:8080` by default. All other settings are configured through the admin UI.
## Development
Local development remains intentionally separate from the deploy-oriented compose setup. Use [docker-compose.yml](docker-compose.yml) and the existing source-build workflow for local development.
```sh
# Run the frontend dev server (hot reload, proxies API to :8090)
make dev-frontend
# Run the Go backend
make dev-backend
```
If you are developing `Silo` and `silo-plugin-sdk` together, keep using the local [`go.work`](go.work) workspace. That workspace is a developer convenience only. CI and release builds run with `GOWORK=off`, so any new SDK helper used here must be pushed and tagged in `silo-plugin-sdk` before this repo can merge or release the change.
See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.
Plugin authors should start with [docs/architecture/plugin-development.md](docs/architecture/plugin-development.md), which covers the RPC plugin package format, generated proto workflow, SDK import paths, route and asset exposure, and auth or user-config integration points.
## Reporting Issues
If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.
Please include:
- What you were trying to do
- Exact steps you took
- What you expected to happen
- What actually happened
- What exact action is slow or broken (`save`, `scan`, `browse`, `import`, `playback`, etc.)
- Whether it happens every time or only sometimes
- The library, media type, filter, setting, or value involved
- Version, branch, commit, and deployment details if you know them
- Screenshots, recordings, or log snippets if relevant
If you used Claude/Codex for debugging, put that under `Technical notes` at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.
Use this template:
```text
Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:
```
### Make Targets
| Target | Description |
|---|---|
| `make build` | Build frontend + Go binary |
| `make frontend` | Build frontend only |
| `make dev-frontend` | Vite dev server with HMR |
| `make dev-backend` | Run Go backend (integrated mode) |
| `make dev-proxy` | Run a standalone proxy node |
| `make dev-transcode` | Run a standalone transcode node |
| `make migrate-create NAME=add_thing` | Create a timestamped Goose SQL migration |
| `make migrate-validate` | Validate Goose migration files without touching a database |
| `make migrate-status` | Show Goose migration status using Silo's bootstrapping runner |
| `make migrate-up` | Apply pending Goose migrations using Silo's bootstrapping runner |
| `make clean` | Remove build artifacts |
### Database Migrations
PostgreSQL schema migrations are managed by Goose. Migration SQL files live in
`migrations/sql/` and use Goose annotations. Converted legacy migrations keep
their original numeric versions so existing `schema_versions` rows can bootstrap
cleanly into Goose without replaying old SQL. New migrations should be created
with timestamped filenames:
```sh
make migrate-create NAME=add_thing
make migrate-validate
```
Do not run `goose fix`; timestamped migrations are the repository policy because
they avoid version collisions across parallel PRs. The existing `001`-style
files are historical compatibility records, not the naming pattern for new work.
Runtime migrations are applied by the integrated/API server only. Proxy and
transcode modes never mutate schema.
For existing installs, use `make migrate-status` and `make migrate-up` rather
than invoking the Goose CLI directly; those targets copy legacy
`schema_versions` rows into `public.goose_db_version` under the migration lock
before reading or applying migrations. Set `ENV_FILE=path/to/.env` when the
database URL should be read from a non-default env file.
### Running Tests
```sh
# Go tests (uses testcontainers — Docker must be running)
go test ./...
# Frontend tests
cd web && bun test
```
### Linting
```sh
# Go
golangci-lint run
# Frontend
cd web && bun run lint
cd web && bun run format:check
```
## License & Trademarks
Silo's source code is licensed under the **GNU Affero General Public License
v3.0 or later** (`AGPL-3.0-or-later`) — see [LICENSE](LICENSE).
The **Silo name, logo, and wordmark are trademarks of Silo Media L.L.C.** and
are **not** covered by the AGPL. You're free to fork and redistribute the code,
but forks and redistributions must not use the Silo brand as their identity and
must remove or replace the brand assets. Publishing a Silo-branded app to an app
store requires written permission. See [TRADEMARK.md](TRADEMARK.md) for what's
permitted — including referential use like "compatible with Silo."
## Configuration
Silo requires only a `DATABASE_URL` when running from source or against external infrastructure. In the default Docker Compose path, the stack wires the database and Redis URLs for you. All other settings — libraries, metadata providers, transcoding, users — are managed through the admin UI after first launch.
@@ -277,7 +127,7 @@ Silo requires only a `DATABASE_URL` when running from source or against external
| `proxy` | Stream proxy node that connects to the shared deployment database and Redis |
| `transcode` | HLS transcode worker node that connects to the shared deployment database and Redis |
## PostgreSQL Auto-Tuning
### PostgreSQL Auto-Tuning
The default Docker Compose stack does not require a checked-in `postgresql.conf`.
It enables Silo's [pgtune](https://github.com/le0pard/pgtune)-style OLTP tuning
@@ -329,20 +179,93 @@ change PostgreSQL server settings. Settings already written with `ALTER SYSTEM`
remain in `postgresql.auto.conf`; reset those PostgreSQL parameters if you later
move fully to a custom `postgresql.conf`.
## Project Structure
## Build from Source
If you prefer running Silo without Docker:
1. **Install prerequisites**: Go 1.24+, Bun 1.0+, PostgreSQL 18+, and FFmpeg.
2. **Start PostgreSQL and Redis** (skip if you already have them running)
```sh
docker compose up -d postgres redis
```
The main compose file still expects `MEDIA_ROOT` to be set even if you only want the bundled PostgreSQL and Redis services, so set that in `.env` first.
3. **Configure the database connection**
```sh
cp .env.example .env
```
Edit `.env` and set `DATABASE_URL` to point to your PostgreSQL instance.
4. **Build and run**
```sh
make build
./silo
```
The server starts at `http://localhost:8080` by default. All other settings are configured through the admin UI.
## Reporting Issues
If you are reporting a bug, install problem, or performance issue, start with the admin workflow and reproduction steps, not Claude/Codex analysis.
Please include:
- What you were trying to do
- Exact steps you took
- What you expected to happen
- What actually happened
- What exact action is slow or broken (`save`, `scan`, `browse`, `import`, `playback`, etc.)
- Whether it happens every time or only sometimes
- The library, media type, filter, setting, or value involved
- Version, branch, commit, and deployment details if you know them
- Screenshots, recordings, or log snippets if relevant
If you used Claude/Codex for debugging, put that under `Technical notes` at the end. Suspected files, SQL output, stack traces, and root-cause theories can be helpful, but only after the workflow and repro steps are clear.
Use this template:
```text
Goal:
Steps:
Expected:
Actual:
What is slow/broken:
Scope:
Version/branch:
Deployment:
Technical notes:
```
cmd/silo/ Entry point
internal/
api/ HTTP router, handlers, middleware
auth/ JWT authentication and sessions
catalog/ Media item, episode, season repositories
config/ YAML + env var configuration
jellycompat/ Jellyfin/Emby protocol compatibility
metadata/ Plugin-driven metadata matching and enrichment
playback/ Direct play, remux, transcode session management
scanner/ Media file discovery and FFProbe
worker/ Background jobs (scan, match, reconcile)
web/ React + TypeScript frontend (Vite, Tailwind, shadcn/ui)
migrations/sql/ Goose-managed PostgreSQL schema migrations
```
## Contributing & Development
Silo is open source and contributions are welcome. See [DEVELOPMENT.md](DEVELOPMENT.md) for building from source in a dev workflow, running tests, database migrations, and project layout, and [CONTRIBUTING.md](CONTRIBUTING.md) for contribution expectations, merge request guidance, and the policy for AI-assisted submissions.
## Supporting Silo
Silo is an open-source hobby project, developed in spare time and funded out of pocket. If you'd like to support development, you can sponsor via [GitHub Sponsors](https://github.com/sponsors/quick104).
Donations go directly toward the costs of building and running the project:
- AI development tooling subscriptions (Claude, Codex) used to build and maintain Silo
- Push notification relay infrastructure
- Future development costs
Sponsoring is entirely optional — Silo is and will remain free and open source. Bug reports, contributions, and feedback are just as valuable.
## License & Trademarks
Silo's source code is licensed under the **GNU Affero General Public License
v3.0 or later** (`AGPL-3.0-or-later`) — see [LICENSE](LICENSE).
The **Silo name, logo, and wordmark are trademarks of Silo Media L.L.C.** and
are **not** covered by the AGPL. You're free to fork and redistribute the code,
but forks and redistributions must not use the Silo brand as their identity and
must remove or replace the brand assets. Publishing a Silo-branded app to an app
store requires written permission. See [TRADEMARK.md](TRADEMARK.md) for what's
permitted — including referential use like "compatible with Silo."
+57
View File
@@ -50,6 +50,7 @@ import (
"github.com/Silo-Server/silo-server/internal/clientip"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/database"
"github.com/Silo-Server/silo-server/internal/downloads"
"github.com/Silo-Server/silo-server/internal/ebooks"
evt "github.com/Silo-Server/silo-server/internal/events"
"github.com/Silo-Server/silo-server/internal/historyimport"
@@ -68,6 +69,7 @@ import (
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/nodeconfig"
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/noderecipe"
"github.com/Silo-Server/silo-server/internal/nodesessions"
"github.com/Silo-Server/silo-server/internal/notifications"
"github.com/Silo-Server/silo-server/internal/opslog"
@@ -615,6 +617,10 @@ func main() {
} else {
srv := transcodenode.NewServer(watcher, tracker)
srv.SetFFmpegLogSink(playback.NewSlogFFmpegLogSink(slog.Default(), nodeID))
// Read jellycompat reconstruction recipes central wrote at transcode
// start, so this node can rebuild a Jellyfin transcode after its own
// restart (the node hop token is recipe-less). Shares the offload Redis.
srv.SetRecipeStore(noderecipe.NewStore(redisClient, 0))
handler = srv.Handler()
}
@@ -1398,6 +1404,7 @@ func main() {
)
userStoreProvider = notifications.WrapUserStoreProvider(userStoreProvider, notificationSystem)
deps.Notifications = notificationSystem
if libraryIngestExecutor != nil {
libraryIngestExecutor.SetAvailabilityDetector(notificationSystem.Detector)
}
@@ -1734,6 +1741,41 @@ func main() {
}
taskMgr.Register(tasks.NewActivityLogCleanupTask(deps.DB, settingsRepo, activityPM))
taskMgr.Register(tasks.NewOperationalLogCleanupTask(deps.DB, settingsRepo, opsPM))
if deps.FileRepo != nil {
// Download prepare-to-file pipeline (Phase 3): a durable, leased encode
// queue hosted on the task manager. Built here (before Start) and shared
// with the API via deps so the download service can enqueue jobs.
artifactMgr := downloads.NewArtifactManager(
downloads.NewArtifactRepository(deps.DB),
downloads.NewRepository(deps.DB),
deps.FileRepo,
downloads.NewPlaybackPreparer(),
deps.NodeID,
func() *config.Config {
if deps.LiveConfig != nil {
if c := deps.LiveConfig(); c != nil {
return c
}
}
return deps.Config
},
func(ctx context.Context, d *downloads.Download) {
if deps.EventsHub == nil {
return
}
_ = deps.EventsHub.PublishJSON(ctx, evt.ChannelUserState, "download", map[string]any{
"download_id": d.ID,
"status": d.Status,
"media_item_id": d.ContentID,
"format": d.Format,
}, evt.PublishOptions{UserID: d.UserID, ProfileID: d.ProfileID})
},
)
encodeTask := tasks.NewEncodeDownloadArtifactsTask(artifactMgr)
artifactMgr.SetKick(func() { _ = taskMgr.RunTask(appCtx, encodeTask.Key()) })
taskMgr.Register(encodeTask)
deps.ArtifactManager = artifactMgr
}
if notificationSystem != nil {
taskMgr.Register(tasks.NewSeedContentAvailabilityTask(notificationSystem))
taskMgr.Register(tasks.NewRebuildReleaseInterestTask(notificationSystem))
@@ -2147,6 +2189,10 @@ func main() {
JWTSecret: cfg.Auth.JWTSecret,
RecWorker: recWorker,
FrontendFS: deps.FrontendFS,
// Hand remote-transcode recipes to the shared recipe store so a dedicated
// transcode node that restarts can rebuild a jellycompat session.
RecipeNodeStore: noderecipe.NewStore(apiRedisClient, 0),
SessionSyncer: deps.SessionSyncer,
}
// Wire direct dependencies when DB is available.
@@ -2185,6 +2231,17 @@ func main() {
compatDeps.WatchCompletionObserver = deps.WatchCompletionObserver
compatDeps.SettingsRepo = settingsRepo
compatDeps.PersonRepo = personRepo
compatSearchService := catalog.NewCatalogSearchService(
appCtx,
settingsRepo,
itemRepo,
catalog.NewSearchIndexEventRepository(deps.DB),
deps.CatalogSearchVectorizer,
)
if compatSearchService != nil {
compatSearchService.StartCoverageRefresh(appCtx)
compatDeps.CatalogSearchProvider = compatSearchService.Provider()
}
if deps.S3Public != nil {
compatDeps.PosterPresigner = deps.S3Public
@@ -0,0 +1,787 @@
# Restart-resilient playback (TR-lease)
How Silo keeps playback alive across a server restart, reconnect, or partial
stream — and how the design arrived at **TR-lease** (token-carried
reconstruction + a server-side session-deny marker for revocation).
> **Status: the revocation half is deferred to a future PR.** Only the
> token-carried **reconstruction** core shipped in this PR. The **session-deny /
> stream-revocation** mechanism described throughout (the `silo:streamauth:<sid>`
> deny marker, the proxy's `Allowed()` enforcement, and the admin
> Stop/Terminate deny write) is **not present in the current implementation**.
> Admin Terminate and user Stop tear down the live in-memory session and the
> ffmpeg producer, but they do **not** prevent a still-valid stream token from
> reconstructing the session until its 24h TTL expires. Sections describing that
> mechanism are kept for design continuity and are individually flagged as
> deferred below.
This document is the consolidated design record. It folds together four earlier
working notes so a future engineer can see the full evolution, the options
weighed, the issues raised across review rounds, the variables traded off, and
why the final shape was chosen. The superseded notes were:
- *unified-playback-reconstruct* — the reconstruct core (PR #174), "RC".
- *token-carried-playback-reconstruct* — the storage evolution, "TR".
- *async-recipe-card-playback* — a monitoring-preserving middle path, "RC-async".
- *playback-reconstruct-options-comparison* — the goals/options matrix.
Paths are repository-relative; assume the repository root is the cwd.
---
## 1. The problem
Before this work, a missing in-memory playback session was a `404`. A server
restart (deploy, crash, OOM) wiped the in-memory session map, so every in-flight
stream died: the client's next manifest/segment/byte-range request hit a session
that no longer existed and playback stopped hard. The same gap appeared on any
front-end that had never held the session (horizontal scale) and on a dedicated
transcode node reboot.
Core priorities for the fix (from the repo guidelines): **performance first,
reliability first, predictable behavior under load and during failures**. The
hot path (per-segment serve) must stay off central/Postgres; bytes must stay
offloaded to nodes; correctness and robustness win over convenience.
---
## 2. The core insight: reconstruct, don't rehydrate
A missing session is a **reconstruct trigger, not a 404**. The client always
re-requests the next chunk after an outage, and its request already states where
it is (an HTTP `Range`, a `?seek=`, or a `seg_NNNNN` number). So the server
rebuilds the session just-in-time from two things:
1. a small **durable descriptor** (identity, ownership, and the byte-affecting
encode parameters — the "recipe"), and
2. the **position the client supplies** on the triggering request.
Nothing live is serialized (no ffmpeg process, context, channels, or log sink).
This is the contribution of PR #174 ("RC"), which also unified the native and
Jellyfin-compat paths behind one `playback.TranscodeManager` so the
card-lifetime rules, the reconstruct concurrency cap, and the node-affinity rule
live in exactly one place. The reconstruct skeleton —
`LoadOrReconstructSession`, `ReconstructSession` / `ReconstructTranscode`, the
single-flight + `reconstructSem` cap that paces the post-restart thundering
herd, `RegisterReconstructed`, the `CloseProcess`/`Close` split — is the
foundation every later option reused. **It was kept; only the descriptor storage
and the revocation model evolved.**
The six routes that must survive a restart:
| # | Route | Tier | How it is rebuilt |
|---|-------|:----:|-------------------|
| 1 | native direct | 1 | client `Range` re-serve; descriptor = identity |
| 2 | native remux | 1 | re-spawn ffmpeg at `?seek`; descriptor = id + audio |
| 3 | native transcode | 2 | rebuild `Session` + ffmpeg seeked to the requested segment; descriptor = full encode opts |
| 4 | jellycompat direct | 1 | compat store + `Range` re-serve |
| 5 | jellycompat remux | 1 | compat store + `?seek` |
| 6 | jellycompat transcode | 2 | compat store + shared-manager reconstruct |
Tier 1 = stateless re-serve (client re-supplies position). Tier 2 = rebuild
ffmpeg. Routes 4–6 always retain a durable central compat store
(`jellycompat_playback_sessions`): third-party Jellyfin clients cannot
round-trip a native descriptor, so the compat layer keeps the full
`PlaybackSession` (route + media source resolved before native reconstruct
runs).
---
## 3. The two axes the options differ on
Every option is a point on a 2-D grid: **where the descriptor lives** (vertical)
× **how revocation is bounded** (horizontal).
```text
AUTHORITY / REVOCATION MODEL ----------------------->
re-resolve at long token short TTL +
reconstruct (24h, expire) refresh checkpoint
D central +----------------+----------------+----------------+
E Postgres row | RC (#174) | (n/a) | RC-async |
S +----------------+----------------+----------------+
C client/node- | (n/a) | TR-24h | TR + re-mint |
R carried token | | | |
I +----------------+----------------+----------------+
P
T top row = descriptor stays central (node serves bytes but cannot
| self-reconstruct → node-restart gap)
| bottom = recipe travels to the node (node self-sufficient, no central record)
```
**TR-lease is a fourth point off this grid:** bottom row (token-carried, 24h),
but the revocation column is replaced by a *server-side deny marker* — on an
admin kill, central would write a per-session deny to Redis that the node
enforces by withholding bytes. Revocation is decoupled from the token TTL and
needs no client refresh. (An earlier iteration polled and re-resolved every
stream on a timer; that was cut to the event-driven session-deny — see §7 and
the §8 design note.)
> **Status: deferred to a future PR.** The deny-marker column described above is
> the design target, not the current shipped behavior. As shipped, the
> token-carried row offers no revocation tighter than the 24h token TTL on the
> node path; admin Terminate and user Stop only tear down the live session and
> ffmpeg producer.
---
## 4. Goals weighed
| ID | Goal |
|----|------|
| G1 | Hot path (per-segment serve) stays off central/Postgres |
| G2 | Bytes offloaded to nodes (served from disk, signature-verified, never central) |
| G3 | Restart resiliency across all 6 routes |
| G4 | Two-factor ownership (auth caller + ownership rebind; refuse `userID==0`/mismatch) |
| G5 | Revocation latency (how fast a banned user / pulled access stops playing) |
| G6 | Single-box: zero external store required for reconstruction |
| G7 | Restart runway (descriptor stays valid longer than the outage) |
| G8 | Multi-front-end / verify-anywhere (no integrated-transcode split-brain) |
| G9 | Observability — durable, identity-rich active-stream record |
| G10 | Low operational surface (tables, migrations, stores, endpoints) |
| G11 | Node-restart recovery |
| G12 | No new client coordination required |
| G13 | Small signing-key blast radius |
| G14 | Cleanup correctness (never wipe a live session's segment dir) |
A note on G9 (observability): the *live* "who is streaming now" view does **not**
depend on the descriptor strategy. Every offload node calls the Redis-backed
node-session tracker (`internal/nodesessions/tracker.go`) on every serve, tied to
actual byte-serving (so a non-cooperative client or replayed token still shows
up) and surviving a central restart. G9 scores only the *durable, identity-rich*
(who + what + recipe) slice on top of that.
---
## 5. The options, and the comparison
Legend: ✓ meets · ✓\* minor asterisk · ~ partial · ✗ fails · ✓✓ best-in-class
| Goal | `main` | RC #174 | TR-24h | TR + re-mint | RC-async | **TR-lease** |
|------|:------:|:-------:|:------:|:------------:|:--------:|:------------:|
| G1 hot path free of central DB | ✓ | ✓\* | ✓ | ✓ | ✓ | ✓ |
| G2 bytes offloaded | ✓ | ✓ | ✓✓ | ✓✓ | ✓✓ | ✓✓ |
| G3 restart resiliency (6/6) | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ |
| G4 two-factor ownership | ✓ | ✓ | ~ | ~ | ~ | ✓~ |
| G5 revocation latency | ~ | ~ | ✗ (≤24h) | ✓ (≤TTL) | ✓✓ | ✓✓ |
| G6 single-box, no external store | ✓ | ✗ | ✓✓ | ~ | ~ | ~ |
| G7 restart runway | n/a | ✓ (30m) | ✓✓ (24h) | ~ | ~ | ✓✓ (24h) |
| G8 multi-front-end | ✗ | ~ | ✓~ | ✓~ | ✓~ | ✓~ |
| G9 durable identity-rich record | ✗ | ✓✓ | ✗ | ✗ | ✓ | ~ |
| G10 low operational surface | ✓✓ | ~ | ✓ | ~ | ✗ | ~ |
| G11 node-restart recovery | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ |
| G12 no new client coordination | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| G13 small signing-key blast radius | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |
| G14 cleanup correctness | ✓ | ✓✓ | ~ | ~ | ~ | ~ |
The TTL/revocation strategies head-to-head. **Note: the session-deny column is
the deferred design target — see §7; as shipped, the node path has no revocation
tighter than the 24h token TTL.**
| Strategy | Revocation latency | Restart runway | Client refresh? | Central on refresh? |
|----------|--------------------|----------------|-----------------|---------------------|
| Re-resolve at reconstruct (RC) | on reconstruct; node hop ≤24h | 30 min | no | only on reconstruct |
| Leave 24h, expire only (TR-24h) | up to 24h | 24h | no | never |
| Short TTL + refresh + denylist (RC-async / TR item B) | ≤~5 min | ~90 s grace | **yes** | every ~3.5 min/session |
| 24h token + session-deny marker (TR-lease — **deferred**) | *design target:* admin kill cuts node bytes on next serve; passive ban = next-play / ≤24h. *As shipped:* node path bounded by ≤24h token only | 24h | **no** | nothing on the serve path; one write per admin-kill event |
---
## 6. Why each non-final option was set aside
- **RC (PR #174, the recipe card in Postgres).** Correct and the basis of
everything here, but it couples reconstruction to a shared per-session Postgres
store: a write on every start, per audio/quality change, and a ≤1/min refresh,
with resilience tied to the DB being reachable. It does not give the single-box
zero-dependency property, and the node hop's 24h token stays unrevocable
anyway. RC's reconstruct *core* was kept; its *storage* was the thing to evolve.
- **TR-24h (token-carried, leave the token to expire).** Wins the single-box case
(zero external dependency) and removes the per-start DB write, but revocation is
bounded by *nothing tighter than the 24h token TTL* on the node path — a banned
user keeps node-served direct/remux for up to 24h. This is actually the
*status quo* for the node path even pre-branch (the proxy verifies by signature
alone, no DB), so TR-24h only adds identity claims to the token without
improving revocation. Not acceptable as the endpoint.
- **TR + re-mint (short TTL + authenticated playlist refresh + denylist), and
RC-async (the same revocation idea but keeping an async-written Postgres mirror
for observability).** Both bound revocation to ~5 min by having the client
refetch a stable `RequireAuth` playlist that re-checks access and re-mints
short-lived segment URLs. The cost: a **hard client-coordination requirement**
(silo-android / silo-apple / Jellyfin clients must implement the refresh flow),
a shrunken restart runway (~90 s grace instead of 30 min — if central is down
past the grace, playback stalls), and a new authenticated re-mint transport
that does not exist on the supported proxy/node topology. RC-async additionally
keeps a write pipeline + denylist + the central row (the highest operational
surface of all). They solved revocation by pushing work onto the client and
shortening the runway — the opposite of the reliability priorities.
The open item across TR and RC-async was always **revocation enforcement** ("item
B"): bounding revocation needs *either* a per-segment token check on the hot path
*or* an authenticated re-mint checkpoint the client drives. TR-lease removes that
dilemma.
---
## 7. TR-lease: the chosen design
TR-lease keeps TR's 24h token (so the restart runway is untouched and no client
refresh flow is needed) and adds a server-side **session-deny marker** for the
one revocation case the node path cannot otherwise cover — no client
participation, no steady-state cost.
**Reconstruction (token-carried).** The signed stream token *is* the durable
descriptor. Its claims carry the full byte-affecting recipe plus `uid/pid/mfid`
ownership lookup keys. A front-end that lost its in-memory session rebuilds
ffmpeg from the token the client re-presents — no shared per-session store, zero
external dependency for reconstruction on a single box. `HWAccel`/`HWDevice` are
deliberately *not* carried; they are re-resolved from live config so an operator
config change applies to reconstructed sessions too. Ownership re-binds to the
live authenticated caller and refuses `userID==0` / mismatch — the token's `uid`
is never trusted alone (a leaked URL is useless without the owner's auth session
on the native path).
**Revocation (session-deny marker).**
> **Status: deferred to a future PR.** Everything in this subsection (the
> `silo:streamauth:<sid>` deny marker, the admin Stop/Terminate deny write, and
> the node-side `Allowed()` enforcement) describes the design target and is **not
> present in the current implementation**. As shipped, admin Terminate and user
> Stop tear down the live in-memory session and the ffmpeg producer, but a
> still-valid stream token can reconstruct the session until its 24h TTL expires;
> there is no node-side byte-withholding and no guaranteed/instant revocation on
> the node path. The design below is retained for when revocation lands.
The revocation surface was deliberately
kept to the *one* case the offloaded topology cannot otherwise cover, rather than
a periodic re-check of every stream. An admin Stop/Terminate would write
`silo:streamauth:<sid> = deny` (TTL ≥ the 24h token lifetime); the offload node
would read it (a node-local Redis GET, sub-ms) before serving any bytes and
return `403` with no bytes on a hit. Enforcement is **byte-withholding, not
client cooperation** — a revoked client that ignores the 403 keeps hitting a
wall.
Why this would be sufficient, and what it deliberately does *not* do:
- **New playback is already blocked at central** — `/playback/start` is
`RequireAuth` + access check, so a ban/scope change always takes effect on the
next play, everywhere, with no marker needed.
- **Cooperative clients already stop** — central holds the realtime WebSocket and
can drop it + send a stop on a ban.
- **The only thing needing a hard node-side cut** is the *explicit, session-scoped
admin kill* of a *non-cooperative* client holding a valid token on a
node-served direct/remux stream (no producer to kill). That is exactly what the
(deferred) session-deny marker would cover; until it lands this case is bounded
by the ≤24h token like the passive bans below.
- **Not covered (accepted limitation):** a *passive* ban or *partial* access
change (lost one library, lowered rating) does NOT hard-revoke an
already-running, non-cooperative node stream. It is enforced at next play and
otherwise bounded by the ≤24h token. Closing this would require either a
periodic re-resolver (a poll over every active stream — rejected as constant
DB load for a rare case) or per-content deny enumeration; both were judged not
worth the machinery for a self-hosted server. A user-scoped deny was rejected
too: it over-blocks content the viewer still legitimately has access to.
**Fail-open on absence (deferred).** In the design, the normal case is *no*
marker, which serves; a Redis error also serves; only a present deny withholds
bytes. This is a deliberate reliability-first choice: availability over
revocation latency.
**Why this shape.** It keeps token-carried reconstruction's single-box and
verify-anywhere properties and would add revocation with no client coordination
(G12) and essentially no steady-state cost — the marker is written only on the
rare admin-kill event, never on a timer. It is, in effect, **TR-24 plus a single
session-scoped kill switch**. The kill switch is **deferred**; what shipped is
TR-24 (token-carried reconstruction, ≤24h token, no node-side revocation).
---
## 8. What was implemented, and where
This PR ships **Commit 1 only** (token-carried reconstruction). **Commit 2 (the
node session-deny marker) is deferred to a future PR** — the `internal/streamauth`
package, the proxy `Allowed()` guard, and the admin deny write described below
are **not present in the current implementation**.
**Commit 1 — token-carried reconstruction; retire `transcode_recipes`. (shipped)**
- `streamtoken.Claims` gains the recipe + `uid/pid/mfid`
(`internal/streamtoken/token.go`).
- `playback.RecipeCard` projects to/from claims (`ToClaims` /
`RecipeCardFromClaims`); the `Reconstruct*` front door takes the decoded recipe
instead of reading Postgres (`internal/playback/recipecard.go`,
`transcode_manager.go`).
- Native serve URLs carry the token as `?st=`; the manifest rewriter already
appends `RawQuery` to segment URIs, so segments inherit it with no route change
(`internal/api/handlers/playback.go`, `stream.go`). The integrated server is
hit directly, so the query-stripping-proxy concern that motivates a path
segment does not apply there; the proxy/node path keeps the token in the URL
path as before.
- Jellycompat folds the recipe into its durable compat store
(`PlaybackSession.Recipe`) since Jellyfin clients cannot round-trip a native
token (`internal/jellycompat/...`).
- Segment-dir cleanup switches from the card index to in-memory liveness (live
map + in-flight reconstruct set + age past the max token TTL); a dedicated node
keeps its boot-time full wipe (`internal/playback/transcode_cleanup.go`,
`transcode_manager.go`).
- `PostgresRecipeStore`, the `RecipeStore` interface, and the unmerged
`transcode_recipes` migration are deleted.
**Commit 2 — node session-deny marker. (DEFERRED — not in this PR)**
> The items below describe the planned revocation commit. They were removed from
> this PR and deferred to a future one; none of these symbols exist in the
> current implementation. The admin Stop/Terminate path still tears down the live
> session and ffmpeg producer, but writes no deny marker, so a valid token can
> reconstruct until its 24h TTL expires.
- `internal/streamauth`: a deny-only Redis `Store` — `Deny(sid)` (TTL ≥ token
lifetime) and a fail-open `Allowed(sid)` reader. No poller, no allow-leases, no
access re-resolver.
- The node guard in `internal/proxy/server.go` (`verifyToken` consults
`Allowed`).
- An admin Stop/Terminate writes the deny marker
(`internal/api/handlers/admin_playback_control.go` → `PlaybackHandler.LeaseDenier`).
- `nodesessions.SessionInfo` carries the numeric ownership keys
(`auth_user_id`/`profile_id`/`media_file_id`), populated by the node from the
verified token, to enrich the live admin "active streams" view (who/what, not
just session id) — see §10 monitoring.
- A single integrated box reads no marker and needs none — the producer-kill +
session removal on Terminate is the stop there.
> Design note (about the deferred commit 2): an earlier iteration added a central
> *revalidator* that re-resolved access for every active stream every ~2 min and
> wrote allow/deny leases. It was dropped in favour of the event-driven
> session-deny above: the poll re-derived, at a constant all-streams DB cost,
> signals the
> system already emits (new playback is gated at `/playback/start`; admin kill is
> an explicit event), and its only unique coverage — sub-24h *passive* revocation
> of an in-flight node stream — was judged not worth the machinery (see §7).
---
## 9. Issues raised across review, and their disposition
From two adversarial review rounds on the reconstruct core and the storage
evolution:
- **Integrated-transcode split-brain across front-ends (P-1 / A).** The
`SessionManager` + reconstruct single-flight are per-process and the descriptor
carries no owning-node id, so without sticky affinity two front-ends can run
divergent ffmpeg. **Disposition: deferred (owner-identity claim).** Supported
topology is dedicated `--mode=transcode` nodes (routed via `tnode`, no
split-brain) + single-box / LB-session-affinity integrated transcode. The
caveat is documented, not silently assumed.
- **24h node-path token is unrevocable (P-2).** The (deferred, see §7–§8)
session-deny marker would let the explicit admin kill cut node bytes on the next
serve. **Until that ships, this remains open:** the node-path token is
unrevocable, so admin kill and passive bans / partial access changes are all
bounded by next-play + the ≤24h token.
- **Dedicated transcode-node restart (P-3 / tr-10). Fixed for both paths.**
*Native:* the proxy forwards the verified stream token to the transcode node
(`X-Silo-Stream-Token`); on a manifest/segment miss the node re-verifies the
token, decodes the recipe, and self-reconstructs ffmpeg seeked to the requested
segment — single-flighted and concurrency-capped exactly like the integrated
reconstruct. *Jellycompat:* the node-hop token is server-minted and *could* carry the
recipe like native, but it deliberately does not — the recipe is mutated in
place under a stable session id (a Jellyfin audio/subtitle switch can arrive as
a `/Sessions/Playing/Progress` report that restarts ffmpeg without re-minting
the client's token), and a third-party Jellyfin client cannot be driven to
refresh a stale token, so a token *snapshot* could reconstruct a stale
rendition. Central instead writes the recipe to a shared Redis recipe store
(`internal/noderecipe`) keyed by upstream session id and overwrites it on every
switch, and the node (which cannot reach Postgres) reads the *current* recipe on
the reconstruct miss — over the same Redis the session tracker already uses (and
the deferred deny-lease would use), so no central URL is needed. The boot-time
full segment-dir wipe
stands; reconstruct re-transcodes from the requested segment. See §10 for the
full rationale (and why this is *not* a token-carryable case).
- **`userID==0` tolerance / "auth optional" (P-5).** Reconstruct hard-rejects
`userID==0` and a card whose session id does not match the URL.
- **Mutable recipe under a stable id (D).** A parameter change re-mints a new
manifest + token and the client reloads, so the common case self-heals. The
only residual is a restart *during* the brief switch transition with a prior
token in flight: it reconstructs the prior *rendition* (access is re-resolved,
only the rendition can be stale). TTL-bounded, self-healing on reload; minor,
accepted.
- **Cleanup liveness (F / tr-9).** Each process owns its `TranscodeDir`, so the
liveness signal is the in-process live map + the in-flight reconstruct set +
age-past-max-TTL; a dir is reaped only when absent from both and older than any
surviving token. No cross-process enumeration-failure mode for the supported
topology.
- **Signing-key blast radius (E / tr-7, G13).** A single `JWTSecret` signs native
auth, the stream token, and the node bearer, so a node-secret leak can forge
native logins. **Accepted risk** on the trusted-node precondition. **Deferred
hardening:** split the stream-token secret from the native-auth secret, then
asymmetric stream-token signing before any untrusted/edge node.
---
## 10. Residuals and follow-ups
- **Revocation of an in-flight node stream (all cases, until deny-lease ships).**
Because the session-deny marker is deferred (§7–§8), *no* ban — admin kill,
passive ban, or partial access change — hard-cuts an already-running,
non-cooperative node-served stream; all are enforced at next play, bounded by
the ≤24h token. The future deny-lease closes the *admin-kill* case; passive
bans would still need a per-content deny enumeration on the access-change event,
or accepting it. See §7.
- **Monitoring.** Two distinct questions:
- *Live "who is watching what right now"* — covered. Every node serve writes a
record to the Redis tracker (`silo:sessions:*`), now enriched with
`auth_user_id` / `media_file_id` from the verified token, so the admin
active-streams view answers who/what/where, tied to real byte-serving and
surviving a central restart. Integrated (no-node) sessions show in central's
in-memory session list.
- *Durable historical/audit ("who watched what last week")* — NOT provided by
this subsystem (there is no longer a durable per-session recipe record, by
design — G9 was the deliberate trade for token-carried reconstruction). That
history lives in the separate watch-progress / scrobble system.
- **Multiple viewers on one stream token.** The token is per playback session, so
a shared/leaked stream URL lets several clients pull bytes under one session id.
They **collapse to a single entry** in the tracker (one `sid`) — monitoring
cannot distinguish or count them — and the (deferred) session-deny would cut
them as a group, not individually. Detecting/limiting this needs a per-account
concurrency cap or
per-device session ids (device binding); deferred. Do not bind to client IP
(mobile / CGNAT).
- **Token in logs (tr-1).** Scrub the `?st=` token / path token from request
loggers (the jellycompat logger logs `RawQuery`; the native logger omits it).
Short TTL + signature bound the exposure.
- **Owner-identity (A).** Required before multi-front-end *integrated* transcode
without sticky affinity; defines owner election, self-URL discovery, a
peer-forward endpoint, and failover.
- **Node-side reconstruct (P-3). Done** for native (token-forwarded) and
jellycompat (Redis recipe handoff via `internal/noderecipe`: central writes the
recipe at remote-transcode start, the node reads it on a reconstruct miss).
Precondition: central and the nodes share the same Redis (already required for
the session tracker, and for the deferred deny-lease). Residual: the recipe
store is consulted
only for a jellycompat token, so a node restart still depends on the recipe key
surviving (24h TTL, ≥ token lifetime).
- *Why a server-side store and not the token here (the real rationale).* It is
tempting to delete `internal/noderecipe` and let the jellycompat node-hop
token carry the recipe like native — the token is server-minted and opaque to
the Jellyfin client (it follows it only as a redirect target), so it *could*.
That does **not** work, and the reason is **not** the often-stated "a Jellyfin
client can't round-trip a token." The reason is that the recipe is **mutated
in place under a stable session id**: a Jellyfin audio/subtitle switch can
arrive as a `/Sessions/Playing/Progress` report
(`restartCompatTranscodeForAudioSelection`) that restarts ffmpeg in place
**without re-minting the client's token**. A third-party Jellyfin client
cannot be driven to refresh the now-stale token — its VOD/`ENDLIST` manifest
is fetched once, and the compat WebSocket (`HandleSocket`) is a receive-only
keep-alive stub with no server→client command to force a manifest re-fetch. A
token is a snapshot at mint time, so after such a switch it would reconstruct
the **stale** rendition (old audio/subtitle) until the client happens to
re-fetch the manifest (e.g. on a seek), which for a VOD stream may be never.
Because the node also cannot reach Postgres (where the authoritative compat
recipe lives), the only way the node rebuilds the *current* recipe is a
node-reachable, server-authoritative store that central overwrites on every
switch — i.e. `internal/noderecipe`. Native has no equivalent path: every
native audio switch re-mints the manifest + token, so the native client's
token is never stale, which is exactly why native needs no such store. The
store is **load-bearing, not redundant**.
- **Key split / asymmetric signing (E).** Closes the account-takeover path; a
precondition for untrusted edge nodes.
---
## 11. Preconditions
- **Persistent `TranscodeDir` (recommended).** With persistence, reconstruct
serves surviving segments; with an ephemeral dir it re-transcodes from the
descriptor (which survives a wiped dir). A node *restart* still loses the
session regardless (P-3).
- **Re-transcode of one segment beats the buffer runway** on worst-case hardware
(software decode, subtitle burn-in). Measure.
- **Client retry/backoff exists.** Clients retry segment/manifest GETs on
5xx/connection error and reconnect the WebSocket. The token rides the URL, so no
extra client logic is needed to carry it; a client that does not retry sees the
failed request as fatal.
---
## 12. Diagrams
The `mermaid` blocks render in GitHub / VS Code / most Markdown viewers; the
ASCII blocks render anywhere.
### 12.1 Topology — where state lives
> **Deferred:** the `silo:streamauth:<sid> = deny` marker, the "Admin
> Stop/Terminate → write session-deny marker" arrow, and the node's "deny GET"
> are the planned revocation path and are **not** in the current implementation.
> The tracker (`silo:sessions:*`) and the recipe handoff are shipped.
```text
┌─────────────────────────── CENTRAL (mode=server) ───────────────────────────┐
Client │ RequireAuth /playback/start /playback/transcode/start (mint token) │
(web / android / │ native serve: /playback/transcode/{sid}/master.m3u8?st= /stream/{sid}?st= │
apple / jellyfin) │ Admin Stop/Terminate → write session-deny marker (event, not a timer) │
└───────────┬──────────────────────────┬───────────────────────┬──────────────┘
│ Postgres │ Redis │
▼ ▼ │ (multi-node only)
┌───────────────────────┐ ┌──────────────────────────┐ ▼
│ catalog / access │ │ silo:sessions:* (tracker)│ ┌────────────────────────┐
│ jellycompat_playback_ │ │ silo:streamauth:<sid> │ │ Proxy node (mode=proxy) │
│ sessions (compat │ │ = deny (session kill) │◀─│ verifyToken + deny GET │
│ store, carries Recipe)│ └──────────────────────────┘ │ serves direct/remux │
└───────────────────────┘ ▲ │ proxies transcode ──────┼──▶ Transcode node
✗ NO transcode_recipes table (deleted this branch) │ node-local GET └────────────────────────┘ (mode=transcode)
└── sub-ms, off Postgres ffmpeg /transcode/{sid}/…
```
Descriptor location by route (the core change). The "lease (node)" revocation
column is the **deferred** design; as shipped, the node path has no revocation
tighter than the ≤24h token:
| route family | descriptor (how it reconstructs) | revocation |
|--------------|----------------------------------|------------|
| native (1–3) | the signed token the client re-presents | *deferred:* lease (node) / shipped: producer-kill (integrated), ≤24h token (node) |
| jellycompat (4–6) | `jellycompat_playback_sessions.data.Recipe` (token can't round-trip) | *deferred:* lease (node) / shipped: producer-kill, ≤24h token (node) |
| ~~all~~ | ~~`transcode_recipes` Postgres row~~ — removed | — |
### 12.2 Where the token rides
```text
NATIVE (integrated server is hit directly → query param; segments inherit it)
manifest : GET /playback/transcode/{sid}/master.m3u8?st=<JWT>
segment : GET /playback/transcode/{sid}/segment/seg_00042.m4s?st=<JWT> ← RawQuery propagated by the manifest rewriter
direct : GET /stream/{sid}?st=<JWT>
PROXY/NODE (capability URL → path segment; survives a query-stripping hop)
GET {proxy}/stream/transcode/{JWT}/master.m3u8 → relative segment URIs inherit {JWT}
GET {proxy}/stream/direct/{JWT} GET {proxy}/stream/remux/{JWT}
JELLYCOMPAT → recipe is NOT in the token; it lives in the compat store.
the path token is used only for the node hop.
```
### 12.3 Normal playback — native, integrated box (no nodes)
```mermaid
sequenceDiagram
participant C as Client
participant S as Central server (integrated)
participant PG as Postgres (access)
Note over C,S: transcode start
C->>S: POST /playback/transcode/start (RequireAuth)
S->>PG: resolve file + access
S->>S: StartTranscode (ffmpeg) + RegisterTranscodeSession
S->>S: card = NewRecipeCard(opts); token = Sign(card.ToClaims(), 24h)
S-->>C: 202 manifestURL = /playback/transcode/{sid}/master.m3u8?st=TOKEN
Note over C,S: hot path — NO Postgres, NO lease (live session in memory)
C->>S: GET .../master.m3u8?st=TOKEN
S->>S: LoadOrReconstructSession → live hit → BuildPlaybackManifest (segments carry ?st)
S-->>C: manifest
loop every segment
C->>S: GET .../segment/seg_N.m4s?st=TOKEN
S->>S: live session → GetSegment → ServeFile
S-->>C: bytes
end
```
The hot path touches neither Postgres nor the lease — it is an in-memory map
hit. `?st` is verified only to decode a card if reconstruct is needed; a live
session ignores it.
### 12.4 Normal playback — native, multi-node (offloaded)
> **Deferred:** the `GET silo:streamauth:{sid}` lease guard and its `deny → 403`
> branch are the planned revocation path, **not** in the current implementation.
> As shipped the node verifies the token, tracks the session, and serves — there
> is no deny check.
```mermaid
sequenceDiagram
participant C as Client
participant S as Central server
participant N as Proxy node
participant T as Transcode node
participant R as Redis
C->>S: POST /playback/transcode/start (RequireAuth)
S->>S: card(full recipe, tnode=T); token = Sign(card.ToClaims())
S-->>C: manifestURL = {N}/stream/transcode/TOKEN/master.m3u8
loop manifest + segments
C->>N: GET /stream/transcode/TOKEN/...
N->>N: verifyToken(TOKEN)
N->>R: GET silo:streamauth:{sid} (lease guard)
alt deny
N-->>C: 403 (no bytes)
else allow / absent
N->>R: Track silo:sessions:{node}:{sid} (uid, mfid from claims)
N->>T: proxy /transcode/{sid}/...
T-->>N: bytes
N-->>C: bytes
end
end
```
The node copies `uid`/`mfid` from the verified token into the Redis tracker —
that is what enriches the live admin "active streams" view (who/what per node).
### 12.5 Restart → reconstruct (native transcode)
```mermaid
sequenceDiagram
participant C as Client
participant S as Central server (just restarted — session map empty)
Note over S: in-memory session GONE; NO transcode_recipes table to read
C->>S: GET /playback/transcode/{sid}/segment/seg_42.m4s?st=TOKEN
S->>S: card = streamCardFromQuery(?st) → Verify + decode recipe
S->>S: LoadOrReconstructSession → GetSession = NotFound, card != nil
S->>S: ReconstructSession (re-bind: refuse userID==0 / != card.uid; card.sid == {sid})
S->>S: ReconstructTranscode(seg=42) → single-flight, mark in-flight, respawn ffmpeg SEEKED to seg 42
S-->>C: bytes (resumes near where the client was)
```
- direct/remux restart is the same but Tier-1: reconstruct the `Session` from the
identity token, then `Range` / `?seek` re-serve — no ffmpeg rebuild.
- multi-node restart: the integrated front-end reconstructs only the `Session`
(to learn `tnode`) and re-proxies; the transcode node keeps serving. If the
transcode *node* itself reboots, the proxy forwards the stream token
(`X-Silo-Stream-Token`) and the node self-reconstructs ffmpeg seeked to the
requested segment (native path, P-3). A jellycompat session on a rebooted node
reconstructs too: the node fetches the recipe central wrote to the shared Redis
recipe store (`internal/noderecipe`) at transcode start, since the Jellyfin
client's token cannot carry it.
- cleanup race: a segment dir is reaped only if absent from the live map AND the
in-flight set AND older than 24h (max token TTL), so a token that can still
reconstruct never gets its dir wiped.
### 12.6 Auth deny — the session-deny marker
> **Status: deferred to a future PR.** The entire "Admin Stop/Terminate → write
> deny marker" and "Offload node → GET deny → 403" flow below is the planned
> revocation design and is **not** in the current implementation. As shipped,
> admin Stop/Terminate does the "realtime WS stop + producer teardown" half only
> (the right branch); it writes no marker, and the node performs no deny check, so
> a valid token reconstructs until its ≤24h TTL. New playback is still gated at
> `/playback/start` (the top subgraph is accurate).
```mermaid
flowchart TD
subgraph New["New playback (any topology)"]
P["/playback/start — RequireAuth + access check"] --> PB{allowed?}
PB -- no --> PR[refused — ban takes effect at next play]
PB -- yes --> PM[mint token]
end
subgraph Admin["Admin Stop/Terminate (event, not a timer)"]
K[denyStreamLease] --> WD[SET silo:streamauth:sid = deny<br/>TTL = 24h]
K --> WS[realtime WS stop + producer teardown]
end
subgraph Node["Offload node — every serve"]
G[verifyToken OK] --> L{GET silo:streamauth:sid}
L -- present deny --> F[403, no bytes]
L -- absent / redis-err --> SV[serve bytes fail-open]
end
```
Invariants:
- No steady-state writes: the marker is set only on an admin kill, never on a
timer. The normal serve path finds *no* key and serves.
- Fail-open: absent key or Redis error → serve. The only hard stop is a present
deny.
- The integrated box reads no marker — the producer-kill + session removal on
Terminate is the stop there; the marker only matters where nodes serve.
- Not covered here (by design): a *passive* ban / partial access change of an
in-flight non-cooperative node stream — enforced at next play, bounded by the
≤24h token (§7).
### 12.7 Jellycompat — what differs
```mermaid
sequenceDiagram
participant J as Jellyfin client
participant S as Central (jellycompat)
participant PG as Postgres compat store
J->>S: negotiate → PlaybackSession{UpstreamSessionID, MediaSources}
S->>PG: Put (jellycompat_playback_sessions)
Note over S: transcode start
S->>S: StartTranscode; recipe = NewRecipeCard(opts)
S->>PG: Update(ps): TranscodeStarted=true, Recipe=recipe (same write)
Note over J,S: restart → reconstruct from the COMPAT STORE, not a token
J->>S: GET HLS segment
S->>PG: Get(playSessionID) → ps.Recipe
S->>S: LoadOrReconstructSession(UpstreamSessionID, card = ps.Recipe) → ReconstructTranscode(*ps.Recipe)
S-->>J: bytes
```
Jellyfin clients cannot carry a native `?st` token, so the recipe is folded into
the durable compat row (`PlaybackSession.Recipe`, persisted in the `data` JSONB)
and written in the same `Update` that flips `TranscodeStarted`. The node hop
(multi-node jellycompat) still uses a path token exactly like native (the lease
guard is deferred along with the rest of the revocation path).
### 12.8 Hot-path cost per request
The "lease" / "SET deny" Redis costs below are **deferred** (the revocation path
is not shipped); as shipped the node does only the tracker write, and admin
Stop/Terminate writes no Redis key:
| request | Postgres | Redis | central CPU | reconstruct |
|---------|----------|-------|-------------|-------------|
| native segment, live, integrated | — | — | map hit | — |
| native segment, live, node | — | 1 track (+ deferred: GET lease) | — | — |
| native segment, after restart, integrated | — | — | verify JWT + ffmpeg respawn (once, single-flight) | yes |
| jellycompat segment, live | — | track (+ deferred: lease) | map hit | — |
| jellycompat segment, after restart | 1 Get (compat row) | track (+ deferred: lease) | respawn | yes |
| admin Stop/Terminate (event, not per-request) | — | deferred: 1 SET deny | one write | — |
---
## 13. Restart-resiliency behavior matrix (role × path)
The observable reconstruction behavior of every playback path under a restart of
each server role. **✅** = the path reconstructs and resumes on the client's next
request; **—** = the path never executes on that role (nothing to reconstruct);
**⚠️** = reconstructs only while a stated precondition holds.
| path | integrated restart | proxy-node restart | transcode-node restart |
|------|--------------------|--------------------|------------------------|
| native · direct | ✅ rebuild `Session` from `?st` token → `Range` re-serve | ✅ stateless re-serve from `claims.MediaPath` | — not served on a transcode node |
| native · remux | ✅ token + `?seek` → re-spawn pipe | ✅ stateless per-request pipe re-spawn | — |
| native · transcode | ✅ token (full recipe) → respawn ffmpeg seeked to segment | ✅ transparent re-route + token forward | ✅ self-contained — token is recipe-complete |
| jellycompat · direct | ✅ re-resolve from durable compat row → `Range` re-serve | ✅ stateless re-serve from token `MediaPath` | — |
| jellycompat · remux | ✅ compat row + `?seek` | ✅ stateless per-request pipe re-spawn | — |
| jellycompat · transcode | ✅ `ReconstructTranscode(*ps.Recipe)` from compat row | ✅ transparent re-route + token forward | ⚠️ reconstructs **iff** the shared Redis recipe (`internal/noderecipe`, ≤24h TTL) still exists; fails closed (404) otherwise |
**What carries the recovery state, per role**
| role | native carrier | jellycompat carrier |
|------|----------------|---------------------|
| integrated | `?st` stream token — identity for direct/remux, full recipe for transcode | durable compat row `PlaybackSession.Recipe` |
| proxy node | self-describing path token, re-derived per request — the proxy holds no per-session state | same path token (identity); the recipe is not needed at the proxy |
| transcode node | forwarded `X-Silo-Stream-Token`, recipe-complete | identity token + Redis `noderecipe` recipe (the Jellyfin token cannot carry it) |
**Reading the matrix**
- **Integrated and proxy-node restarts are unconditional** for every applicable
path. The proxy holds no authoritative per-session state — direct/remux are
re-served from the token's `MediaPath`, transcode is reverse-proxied — so a
proxy bounce, or a reroute to any in-group proxy (`pickProxy` is round-robin
with no session affinity), is transparent.
- **The four `—` cells are structural, not gaps.** Direct and remux never run on
a transcode node — the proxy (or the integrated box) serves them from source
media — so a transcode-node restart has nothing to reconstruct on those paths.
- **The one conditional (⚠️)** is jellycompat transcode after a *transcode-node*
restart. The node-hop token is identity-only — not because a Jellyfin client
cannot round-trip it (the token is server-minted and opaque to the client), but
because the recipe is mutated in place under a stable session id and a
third-party Jellyfin client cannot be driven to refresh a stale token, so a
token snapshot could reconstruct a stale rendition (see §10). The node therefore
rebuilds from the Redis `noderecipe` entry central overwrites on every switch
rather than from the token. Native transcode has no such dependency — every
native audio switch re-mints the token, so it is always recipe-complete and
current. This is the only native-vs-jellycompat asymmetry in reconstruction
(see §10).
**Out of scope of this matrix** (accepted behaviors, not restart-reconstruction
holes — see §7, §10):
- Passive ban / partial access change of an in-flight node stream — enforced at
next play, bounded by the ≤24h token, not by reconstruction.
- Integrated transcode behind a non-sticky load balancer can split-brain into
divergent segment dirs (P-1); safe single-front-end or with LB affinity. Remote
(node) sessions are immune — node affinity routes every front-end to the same
transcode node.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,711 @@
# Offline sync for mobile (downloads v2) — design
**Goal:** Let mobile clients (Android, Apple) download media for **fully offline** playback and keep
watch state consistent across reconnects. Three pillars: (1) a **device-scoped download registry** —
the server tracks what each device has downloaded; (2) an **offline playback manifest** — one bundle
giving the client everything it needs to play with no network (metadata, subtitle files, artwork,
chapter + intro/credits markers); (3) **offline progress reconciliation** — the client queues watch
progress while offline and the server merges it on reconnect (last-write-wins). Downloads are requested
through an admin-gated **quality** ladder (`original`, then bitrate presets), with server-side
**transcode-to-a-single-file** for bitrate requests and compatibility fallback for devices that cannot
play the source directly.
**Status:** Approved — implementing. This capability cleared the Silo v1 scope gate on
2026-06-18 (via the **v1 capability proposal** triage; template
`.github/ISSUE_TEMPLATE/v1-capability-proposal.yml`), and the offline-sync/downloads-v2 work is now
being implemented phase by phase against this design. Per the decision below, the
`/api/v1/downloads` contract is **reshaped, not just extended** — acceptable because the change lands
**before** scope lock (`docs/architecture/v1-scope.md`) and **before** any mobile client ships against
it. This document began as the design input to that proposal and now serves as the implementation
reference; it is kept in sync as the implementation lands.
Decisions locked with the requester:
- **Replace** the existing `internal/download` package with a new unified package (`internal/downloads`)
that absorbs today's download behavior and adds the three pillars.
- **The existing `/api/v1/downloads` contract may be reshaped, not just extended.** The only current
consumer is the first-party web app, which is updated in lockstep. So downloads v2 is a single
coherent surface: one device-aware `downloads` table and one `/downloads/*` endpoint family — *not* a
frozen legacy table plus a parallel registry. (This intentionally diverges from the v1 additive-only
API rule, which is acceptable here because the change lands **before** scope lock and **before** any
mobile client ships against it — exactly the right time to reshape. The proposal must call this out so
reviewers/clients know the download contract changed pre-lock.)
- **Quality is user-facing; delivery format is internal.** Clients request `quality` from the ordered
ladder `original > 20mbps > 10mbps > 5mbps > 2mbps > 1mbps`. `remux` is not a preset; it is a
server-selected `delivery_format` when `original` needs container/audio compatibility. Bitrate
presets are gated by a server setting **and** the per-user `download_transcode_allowed` flag.
- **Transcode-to-single-file is in v1.** (Today's transcode pipeline only emits ephemeral HLS segments.)
- **No expiry, no lease, no DRM.** Downloaded files persist on-device until the user deletes them; the
server can only revoke *future* downloads via `users.download_allowed`.
- **No cross-device visibility** in v1 (a device sees its own entries, not other devices').
**Implementation update (2026-06-20):** the current client-facing contract is documented in
`docs/downloads-api.md`. The public request field is `quality`, capabilities expose
`quality_presets`, rows expose `quality`, `effective_quality`, `delivery_format`,
`target_bitrate_kbps`, and `revision`, and batch manifests are implemented at
`GET /api/v1/downloads/batches/{batch_id}/manifests`.
> Commands and paths in this document are repository-relative; assume the repository root is the cwd.
## Why this shape
- **One device-aware table, not two.** The original plan kept today's `downloads` table
(`migrations/sql/042_downloads.sql`, keyed on `user_id`) frozen and added a separate device registry,
purely to avoid breaking the existing contract. With that constraint lifted, the cleaner model is a
single `downloads` table reshaped to carry `device_id` (and `profile_id`). `device_id` is **nullable**:
- `device_id IS NULL` → an **ephemeral / account-level** download (today's web-app browser download:
pick a version, pull the file; the row is a convenience record, not a managed library entry).
- `device_id` set → a **managed device-library entry** (a phone "has" this file; durable; unique per
`(user, profile, device, content, episode)`; the target of manifest + progress reconciliation).
One table, one endpoint family, and the format/transcode machinery is shared by web and mobile alike.
- **Device identity already exists in Postgres.** `migrations/sql/180_user_devices.sql` defines
`public.user_devices` keyed `(user_id, profile_id, device_id)` with FKs to `users` and
`user_profiles`. Device id arrives in the `X-Silo-Device-Id` header (see
`internal/api/handlers/settings.go:37`). Managed entries reference it; ephemeral rows leave it NULL.
- **The last-write-wins primitive already exists.** `userstore.UserStore.SetProgressIfNewer(... updatedAt) (bool, error)`
(`internal/userstore/store.go:26`, implemented in `internal/userdb/progress.go:131`) is exactly the
merge semantics offline reconciliation needs. Progress reconciliation is therefore **additive changes
to the existing progress endpoints**, not new machinery, and it deliberately stays in the watch-state
domain rather than being folded into the downloads package.
- **Manifests must never carry presigned URLs.** `catalog.ItemDetail` (`internal/catalog/detail.go`)
exposes `PosterURL`/`BackdropURL` as short-lived presigned S3 URLs — useless for an offline bundle
that may be opened days later. The manifest carries **stable** references (thumbhashes inline +
authenticated proxy endpoints the client fetches once at download time), so a stored manifest never
expires.
- **Remux and transcode both need a produced file, not a live pipe.** A clean, seekable, resumable
download requires a finalized file (`-movflags +faststart` relocates the moov atom in a finalization
pass — impossible over a pure pipe). `internal/playback`'s current `ServeRemux`/`StartTranscode`
stream ephemeral output. So remux and transcode share one **prepare-to-file** job pipeline; only
`original` is served straight from the source on disk.
## The replacement: `internal/downloads` (new package)
`internal/download` (singular) is replaced by `internal/downloads` (plural). The new package owns the
whole downloads domain; everything below lives in it unless noted.
```
internal/downloads/
model.go — Download (web + device, one type), Artifact, OfflineManifest,
quality/delivery constants, status constants, sentinel errors
repo.go — Postgres CRUD: downloads (reshaped), download_artifacts
service.go — orchestration: permission + quota checks, quality policy, create/list/serve
policy.go — DownloadQualityResolver (quality ladder + delivery decision + server/user gates)
bandwidth.go — BandwidthManager (ported from internal/download/bandwidth.go)
limiter.go — QuantityLimiter (ported from internal/download/limiter.go)
manifest.go — ManifestBuilder (assembles OfflineManifest; strips all presigned URLs)
artifacts.go — ArtifactManager (async prepare-to-file jobs: remux + transcode, dedup, cleanup)
serve.go — file serving for original source and prepared artifacts
```
**What it absorbs / reshapes** from `internal/download`: the `downloads` table and its endpoints, now
reshaped to be device-aware and quality/delivery-aware; `BandwidthManager`; `QuantityLimiter`; the `Service`
permission/quota/file-serving logic; and the `config.DownloadConfig` integration.
**What stays out of the package (by design):**
- **Progress reconciliation** is watch-state, not downloads. It is additive changes to
`internal/api/handlers/progress.go` + `internal/userstore`/`internal/userdb`, reusing
`SetProgressIfNewer`. Folding it into `internal/downloads` would couple two domains and duplicate the
LWW write.
- **The encode/remux primitive** stays in `internal/playback` (it owns ffmpeg). `internal/downloads`
calls a new `playback.PrepareFile(...)`; it does not shell out to ffmpeg itself.
**Cutover strategy:** because the web app is the only consumer and is updated in lockstep, Phase 0 is a
*reshape*, not a verbatim port: move the logic into `internal/downloads`, reshape the table + endpoints
(add device-awareness plus quality/delivery fields), update the web app's download hooks/components, delete
`internal/download`. The handler `internal/api/handlers/downloads.go` already depends only on a narrow
`DownloadService` interface, so the swap is mechanical; its request/response DTOs gain the new fields.
## Data model (Goose migrations)
Migrations are timestamped, created with `make migrate-create NAME=...` (never hand-numbered; never
`goose fix`).
### 1. Reshape `downloads` (device-aware, format-aware)
A single migration alters the existing table. Downloads ship default-off with little/no production data,
so widening it in place is safe.
```sql
-- +goose Up
-- +goose StatementBegin
ALTER TABLE public.downloads
ADD COLUMN profile_id text, -- NULL for ephemeral/web rows
ADD COLUMN device_id text, -- NULL = ephemeral; set = managed device entry
ADD COLUMN format text NOT NULL DEFAULT 'original',
ADD COLUMN artifact_id text; -- set for remux/transcode once prepared
ALTER TABLE public.downloads
ADD CONSTRAINT downloads_format_check CHECK (format IN ('original','remux','transcode'));
-- Widen the status enum to cover the managed-entry lifecycle alongside the existing serve states.
ALTER TABLE public.downloads DROP CONSTRAINT downloads_status_check;
ALTER TABLE public.downloads ADD CONSTRAINT downloads_status_check
CHECK (status IN ('queued','downloading','completed','failed','cancelled', -- existing
'registered','preparing','ready','revoked')); -- managed entries
-- One managed entry per (user, profile, device, content, episode). Ephemeral rows (NULL device_id)
-- are exempt via the partial index.
CREATE UNIQUE INDEX downloads_device_entry_uidx
ON public.downloads (user_id, profile_id, device_id, content_id, COALESCE(episode_id, ''))
WHERE device_id IS NOT NULL;
CREATE INDEX downloads_device_idx ON public.downloads (user_id, profile_id, device_id) WHERE device_id IS NOT NULL;
CREATE INDEX downloads_artifact_idx ON public.downloads (artifact_id) WHERE artifact_id IS NOT NULL;
-- Composite FK to user_devices. MATCH SIMPLE (the default) skips the check whenever any column is NULL,
-- so ephemeral rows (NULL device_id) are not constrained while managed entries are.
ALTER TABLE public.downloads
ADD CONSTRAINT downloads_device_fkey
FOREIGN KEY (user_id, profile_id, device_id)
REFERENCES public.user_devices(user_id, profile_id, device_id) ON DELETE CASCADE;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE public.downloads DROP CONSTRAINT IF EXISTS downloads_device_fkey;
DROP INDEX IF EXISTS downloads_artifact_idx;
DROP INDEX IF EXISTS downloads_device_idx;
DROP INDEX IF EXISTS downloads_device_entry_uidx;
ALTER TABLE public.downloads DROP CONSTRAINT IF EXISTS downloads_format_check;
ALTER TABLE public.downloads DROP CONSTRAINT IF EXISTS downloads_status_check;
ALTER TABLE public.downloads ADD CONSTRAINT downloads_status_check
CHECK (status IN ('queued','downloading','completed','failed','cancelled'));
ALTER TABLE public.downloads
DROP COLUMN artifact_id, DROP COLUMN format, DROP COLUMN device_id, DROP COLUMN profile_id;
-- +goose StatementEnd
```
Status lifecycle:
- **ephemeral / web** (`device_id IS NULL`): `queued → downloading → completed` / `failed` /
`cancelled` — exactly today's behavior, now also able to request `remux`/`transcode`.
- **managed entry** (`device_id` set):
- `original`: `registered → ready` immediately (client pulls the source), then client reports
`completed`.
- `remux` / `transcode`: `registered → preparing` (artifact job runs) `→ ready → completed`; or
`failed`.
- `revoked`: admin/permission revocation of *future* access. Cannot delete the on-device file (no DRM
by decision); it only stops the server serving new bytes.
The unique index uses `COALESCE(episode_id,'')` so movies (NULL `episode_id`) collapse to one managed
entry per `(device, content)` without NULL-comparison surprises.
### 2. `download_artifacts` — prepared (remux/transcode) files, deduplicated
```sql
-- +goose Up
-- +goose StatementBegin
CREATE TABLE public.download_artifacts (
id text NOT NULL,
media_file_id integer NOT NULL REFERENCES public.media_files(id) ON DELETE CASCADE,
format text NOT NULL, -- 'remux' | 'transcode'
params_hash text NOT NULL, -- sha256 of the encode parameters (see below)
container text NOT NULL DEFAULT 'mp4',
codec_video text NOT NULL DEFAULT '',
codec_audio text NOT NULL DEFAULT '',
resolution text NOT NULL DEFAULT '',
audio_track_index integer NOT NULL DEFAULT -1,
output_path text NOT NULL DEFAULT '', -- absolute path on the server's artifact volume
file_size bigint NOT NULL DEFAULT 0,
status text NOT NULL DEFAULT 'queued',
error_message text NOT NULL DEFAULT '',
-- Durable-queue / crash-recovery columns (see "Durable artifact queue" below).
attempts integer NOT NULL DEFAULT 0,
max_attempts integer NOT NULL DEFAULT 3,
lease_owner text, -- worker/node id holding the running lease
lease_expires_at timestamptz, -- NULL unless status='running'
next_retry_at timestamptz, -- backoff gate for re-enqueue after a failure
created_at timestamptz NOT NULL DEFAULT now(),
completed_at timestamptz,
last_used_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT download_artifacts_pkey PRIMARY KEY (id),
CONSTRAINT download_artifacts_unique UNIQUE (media_file_id, format, params_hash),
CONSTRAINT download_artifacts_status_check CHECK (status IN ('queued','running','ready','failed'))
);
CREATE INDEX download_artifacts_lru_idx ON public.download_artifacts (last_used_at) WHERE status = 'ready';
-- Claimable work: queued rows whose backoff has elapsed, plus running rows whose lease has expired.
CREATE INDEX download_artifacts_claimable_idx ON public.download_artifacts (status, next_retry_at);
CREATE INDEX download_artifacts_lease_idx ON public.download_artifacts (lease_expires_at) WHERE status = 'running';
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS public.download_artifacts;
-- +goose StatementEnd
```
`params_hash = sha256(format | container | codec_video | codec_audio | resolution | audio_track_index | subtitle_burn_in)`.
Two devices (or a web user and a phone) requesting the same 1080p H.264/AAC of the same source share one
artifact. `last_used_at` drives LRU cleanup; `output_path` is on a configured artifact volume. The
`attempts`/`lease_*`/`next_retry_at` columns make the table a **durable, recoverable job queue** (see the
"Durable artifact queue" subsection) so a crash mid-encode cannot strand a download in `preparing`.
### 3. Server settings (no DDL)
`server_settings` is key/value, so new keys need no migration — written when an admin saves them, parsed
in `internal/config/db_loader.go`:
- `download.transcode_enabled` (bool, default **false**) — server gate for transcode-to-file.
- `download.artifact_dir` (string) — artifact output volume; defaults under the existing transcode/temp root.
- `download.max_concurrent_prepares` (int, default 2) — encode/remux worker-pool size.
- `download.artifact_max_bytes` (int64, default 0 = unlimited) — LRU eviction budget for artifacts.
`config.DownloadConfig` (`internal/config/config.go:299`) gains the matching fields; they hot-reload via
the existing 30s-TTL cache (`internal/download/service.go:107` → ported), and the keys are registered as
non-restart in `internal/config/restart_keys.go`.
## Quality policy
Resolved in `internal/downloads/policy.go` at create time, for both web and device flows:
```go
const (
QualityOriginal = "original"
Quality20Mbps = "20mbps"
Quality10Mbps = "10mbps"
Quality5Mbps = "5mbps"
Quality2Mbps = "2mbps"
Quality1Mbps = "1mbps"
)
type QualityDecision struct {
RequestedQuality string
EffectiveQuality string
DeliveryFormat string // original | remux | transcode
TargetBitrateKbps int
RequiresArtifact bool
}
func (DownloadQualityResolver) Resolve(requested string, user *models.User,
cfg config.DownloadConfig, file *models.MediaFile, caps playback.ClientCapabilities,
artifactsAvailable bool) (QualityDecision, error)
```
Decision tree:
1. Empty quality defaults to `original`.
2. `original` direct-plays the source unless supplied device caps require a compatibility artifact.
Container/audio-only compatibility records `delivery_format=remux`; video transcode fallback records
`effective_quality=20mbps`, `delivery_format=transcode`.
3. Bitrate presets require `cfg.TranscodeEnabled` (server), `user.DownloadTranscodeAllowed`, and an
available artifact pipeline. Server gate off → `ErrTranscodeDisabled` (403). User flag off →
`ErrDownloadNotAllowed` (403). Missing artifact pipeline → `ErrQualityUnavailable` (501).
4. `remux` and `transcode` as request values are invalid public qualities.
Target codec/resolution for compatibility artifacts and bitrate transcodes is chosen by reusing `playback.Resolve` /
`playback.SelectVersion` (`internal/playback/resolver.go:60`) against the device's declared capabilities
and the admin transcode ceilings, so download encoding matches streaming decisions exactly (no
duplicated codec logic).
## Transcode-to-file pipeline (v1)
`internal/downloads/artifacts.go` — `ArtifactManager`. Remux and transcode share this; `original` never
enters it.
**On create of a remux/transcode download (web or device):**
1. Resolve target params (codec/res/container/audio track) → compute `params_hash`.
2. `INSERT ... ON CONFLICT (media_file_id, format, params_hash) DO NOTHING` into `download_artifacts`;
read the row. If `status = ready`, link `downloads.artifact_id`, set `status = ready`,
`file_size = artifact.file_size`, bump `last_used_at`. Done — no new encode.
3. Otherwise the download row is `preparing`; if the artifact is freshly `queued`, enqueue an encode job.
**Encode worker** (bounded pool of `download.max_concurrent_prepares`, hosted on the existing
`internal/taskmanager` worker machinery so jobs are tracked, observable in admin, and cancellable):
1. **Claim a job transactionally** (see "Durable artifact queue" below) — never just "enqueue freshly
queued rows," which loses work on restart.
2. Call a new `playback.PrepareFile(ctx, opts)` that builds **single-file** ffmpeg args and writes to
`output_path + ".part"`, then atomically renames to `output_path`:
- **remux:** `-map ... -c copy -movflags +faststart -f mp4` (fast, near disk-throughput).
- **transcode:** the existing encode args (reuse the `buildFFmpegArgs` logic / `TranscodeOpts` at
`internal/playback/transcode.go:30`, incl. HW-accel, audio-track selection, optional subtitle
burn-in) but targeting `-movflags +faststart -f mp4` to a file instead of `-f hls`.
`TranscodeOpts` already carries every needed field (`InputPath`, `TargetCodecVideo/Audio`,
`TargetResolution`, `HWAccel`, `AudioTrackIndex`, `SubtitleBurnIn`, `TotalDuration`).
3. On success: artifact → `ready` (`file_size`/`completed_at`, clear lease); flip every linked download
row → `ready`; publish a `user_state`/`downloads` event on the events hub (`internal/events`) so a
waiting client pulls without polling.
4. On failure: increment `attempts`, set `error_message`/`next_retry_at` (backoff). If
`attempts >= max_attempts` → artifact `failed` and linked rows `failed`; otherwise return it to
`queued` for retry. Either way, delete the stale `.part` file.
**Durable artifact queue (crash/restart recovery).** The artifact table is the queue of record, so a
process exit mid-encode cannot strand a download in `preparing` forever:
- **Transactional claim:** a worker claims a job with a single statement —
`UPDATE download_artifacts SET status='running', lease_owner=$me, lease_expires_at=now()+$lease, attempts=attempts+1
WHERE id = (SELECT id FROM download_artifacts
WHERE (status='queued' AND (next_retry_at IS NULL OR next_retry_at <= now()))
OR (status='running' AND lease_expires_at < now()) -- steal expired leases
ORDER BY created_at LIMIT 1 FOR UPDATE SKIP LOCKED) RETURNING *`.
`FOR UPDATE SKIP LOCKED` makes concurrent workers (and multiple nodes) safe without double-encoding.
- **Lease heartbeat:** a running worker periodically extends `lease_expires_at`; if it dies, the lease
expires and another worker (or the startup sweep) reclaims the job.
- **Startup sweep:** on boot, reset `running` rows with an expired lease back to `queued` (bounded by
`max_attempts`), and re-link any `preparing` download rows to their artifact's real state. A `ready`
artifact whose `output_path` is missing on disk is reset to `queued` (the file was lost).
- **Idempotency:** the encode writes to `output_path + ".part"` and atomically renames; a reclaimed job
overwrites its own `.part`. `output_path` is deterministic from `(media_file_id, format, params_hash)`.
- **No silent stranding:** a download stuck in `preparing` past a bound is surfaced (admin task + a
`failed` transition with reason), never left invisibly hung.
**Serving** (`internal/downloads/serve.go`, `GET /downloads/{id}/file`):
- `original`: open `media_files.file_path`, `http.ServeContent` (Range/resume) wrapped in the ported
`BandwidthManager.ThrottledReader`. Same path as today's `serveFileDownload`.
- `remux`/`transcode`: require `status = ready`; serve `artifact.output_path` the same way (Range/resume
+ throttle); bump `last_used_at`.
**Cleanup:** an admin/cron job (mirroring `internal/playback/transcode_cleanup.go`) evicts `ready`
artifacts whose `last_used_at` is older than a retention window or when the total exceeds
`download.artifact_max_bytes` (LRU), but never one still linked by an active managed download row,
including completed rows that represent a device's local library state.
Whatever is dropped is logged (no silent truncation).
## Offline playback manifest
`internal/downloads/manifest.go` builds `OfflineManifest` from `catalog`'s existing detail path (reusing
`catalog.ItemDetail` assembly — markers, chapters, subtitle list, versions) and then **strips every
presigned URL**. The manifest is stable: a client can store it and re-read it offline indefinitely.
Manifest + asset endpoints are only meaningful for managed device entries (`device_id` set).
```go
type OfflineManifest struct {
DownloadID string `json:"download_id"`
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
Type string `json:"type"` // movie | episode
Revision int `json:"revision"`
Quality string `json:"quality"`
EffectiveQuality string `json:"effective_quality"`
DeliveryFormat string `json:"delivery_format"`
TargetBitrateKbps int `json:"target_bitrate_kbps"`
MediaFileID int `json:"media_file_id"`
FileSize int64 `json:"file_size"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
Overview string `json:"overview,omitempty"`
Runtime int `json:"runtime,omitempty"`
ContentRating string `json:"content_rating,omitempty"`
Genres []string `json:"genres,omitempty"`
SeriesID string `json:"series_id,omitempty"`
SeriesTitle string `json:"series_title,omitempty"`
SeasonNumber *int `json:"season_number,omitempty"`
EpisodeNumber *int `json:"episode_number,omitempty"`
// Artwork: stable thumbhashes inline (immediate offline placeholder) + authenticated proxy URLs
// the client downloads ONCE at download time. Never presigned S3 URLs.
PosterThumbhash string `json:"poster_thumbhash,omitempty"`
BackdropThumbhash string `json:"backdrop_thumbhash,omitempty"`
ArtworkURLs struct {
Poster string `json:"poster,omitempty"`
Backdrop string `json:"backdrop,omitempty"`
Logo string `json:"logo,omitempty"`
} `json:"artwork_urls"`
// Playback metadata
Container string `json:"container"`
CodecVideo string `json:"codec_video"`
CodecAudio string `json:"codec_audio"`
Resolution string `json:"resolution"`
HDR bool `json:"hdr"`
Duration int `json:"duration_seconds"`
Chapters []OfflineChapter `json:"chapters,omitempty"`
Intro *Marker `json:"intro,omitempty"`
Credits *Marker `json:"credits,omitempty"`
Recap *Marker `json:"recap,omitempty"`
Preview *Marker `json:"preview,omitempty"`
Subtitles []OfflineSubtitle `json:"subtitles"`
// Stable identity so the client can re-resolve content_id after a server-side rescan
// (mirrors userstore.WatchIdentity at internal/userstore/types.go:130).
StableIdentity OfflineIdentity `json:"stable_identity"`
ManifestVersion int `json:"manifest_version"` // bump on DTO shape change
GeneratedAt string `json:"generated_at"` // RFC3339
}
type OfflineSubtitle struct {
Language string `json:"language"`
Format string `json:"format"` // srt | ass | vtt
Forced bool `json:"forced"`
HearingImpaired bool `json:"hearing_impaired"`
External bool `json:"external"`
FetchURL string `json:"fetch_url"` // authenticated proxy; client downloads at download time
FileSize int64 `json:"file_size,omitempty"`
}
```
**Artwork/subtitle proxies** (replace presigning):
- `GET /downloads/{id}/artwork/{kind}` (`kind` ∈ `poster|backdrop|logo`) reads the raw stored path and
**streams the bytes** through the existing image resolver instead of redirecting to a presigned URL.
The client caches these once.
- `GET /downloads/{id}/subtitles/{ref}` (`ref` encodes `external:{index}` or `downloaded:{id}`) streams
the subtitle file (external sub from disk; downloaded sub from S3 via the existing subtitle manager).
Both are authorized under the **full managed-entry rule** (see "Security & reliability invariants"):
the row must match `(user_id, profile_id, header device_id)`, and content/library access is **re-checked
for the requesting profile** before any bytes are served (a download id alone never authorizes access).
They are session-authenticated rather than time-limited tokens, so a manifest stored on-device stays valid.
## Progress reconciliation
Two **additive** changes. The naïve version (route the client's wall-clock straight into
`SetProgressIfNewer` and drive `?since=` off the same column) is **unsafe**: a skewed or malicious device
can submit a far-future timestamp, win LWW permanently over genuinely-newer server activity, and poison
the cursor (a future-dated row makes a reader advance its cursor past real writes). So the design
**separates client event time from server ordering**.
The watch-progress row gains two server-owned facets (LWW key vs. cursor ordering):
- **`event_at`** — the client-observed wall-clock for the progress event, used as the **LWW comparison
key**, but **bounded on ingest**: `event_at = min(client_updated_at, server_now + skew)` with a small
skew window (e.g. 2m); a value past the window is clamped to `server_now` (and logged). For an online
write `event_at = server_now`. Bounding means a client can at most claim "now" for its **own profile's**
progress — which it is already entitled to do — and can never lock in a far-future value.
- **`synced_seq`** — a **server-assigned monotonic** marker set on every write (a sequence, or
server-ingest timestamp with a tie-breaker). It is **never** client-influenced and is the sole basis
for the `?since=` cursor.
**Write — `POST /api/v1/sync/progress`** (`internal/api/handlers/progress.go:230`): add an optional
`updated_at` (RFC3339) per item = the client event time. The handler clamps it (above) and merges via an
extended `SetProgressIfNewer` whose comparison is on the **bounded `event_at`**, while the server stamps
`synced_seq` itself. Absent `updated_at` → behavior is exactly as today (server `now()`), so existing
callers (incl. watchsync) are unaffected. Completion is never inferred from the timestamp alone — the
existing watched-threshold logic still gates `completed`.
```go
type syncProgressItem struct {
MediaItemID string `json:"media_item_id"`
Position float64 `json:"position"`
Duration float64 `json:"duration"`
ForceOverwrite bool `json:"force_overwrite"`
UpdatedAt *string `json:"updated_at,omitempty"` // NEW: RFC3339; client EVENT time (server clamps it)
}
```
**Read — `GET /api/v1/progress?since=<cursor>`** (`internal/api/handlers/progress.go:86`): the `since`
cursor is an **opaque server token over `synced_seq`**, not a client timestamp. When present, only rows
with `synced_seq > cursor` are returned (use a sequence, or `>=` + id de-dup if a timestamp marker is
used, to avoid tie loss). The response carries the next cursor. This makes delta delivery immune to
client clock skew. Requires `UserStore.ListProgressSince(ctx, profileID, cursor)` in both backends
(`internal/userdb`, `internal/userstore/pgstore`) ordering/filtering on `synced_seq`. Absent `since` →
identical to today.
Adding a `UserStore` method (and the `event_at`/`synced_seq` columns to the per-user `watch_progress`
schema) touches every stub implementation; ship the implementations, the schema change, and the shared
store test suite update (`internal/userstore/storetest`) in the same PR.
**Net safety:** honest clients still get correct "most-recent actual watch wins" LWW; a dishonest clock
is bounded to claiming "now" for its own profile (no authority it didn't already have), and cursors —
the cross-device delivery mechanism — never depend on a client clock at all.
## API surface (one `/downloads/*` family; existing endpoints reshaped)
The web app is updated in lockstep, so existing endpoints are reshaped rather than frozen.
**Reshaped (existing):**
```
POST /downloads body {content_id, episode_id?, file_id?, quality?, series?, caps?};
X-Silo-Device-Id present → managed entry; absent → ephemeral/web row.
`quality` is original by default; bitrate presets are single-item only.
GET /downloads managed entries for the CALLING device (device from header); ephemeral rows otherwise
DELETE /downloads/{id} remove a row owned by (user, profile, header device)
GET /downloads/{id}/file serve original (source) or prepared artifact (Range/resume + throttle)
GET|HEAD /direct-download one-shot browser download, original-only (browser-friendly, token-in-URL)
```
The response DTO gains `device_id`, `quality`, `effective_quality`, `delivery_format`,
`target_bitrate_kbps`, `revision`, artifact-derived readiness, and the new statuses.
**`device_id` is authoritative only from the `X-Silo-Device-Id` header — never from the body or query**
(a body/query value could only ever be a display hint, and is not accepted as authority).
**Bitrate qualities are single-item only.** A series batch (`series:true`) and
`/direct-download` are **original-quality only**: `/direct-download` streams
synchronously and cannot wait on an async encode, and a series batch would otherwise
fan out N encode jobs. A client that wants prepared transcodes for a series requests
each episode individually. `DownloadQualityResolver` rejects bulk bitrate requests
with `ErrBulkQualityUnavailable` / HTTP 501.
**New — capability / feature detection** (pattern from `internal/api/handlers/notifications.go:324`):
```
GET /downloads/capability
→ { "enabled": bool, "download_allowed": bool,
"quality_presets": ["original","20mbps","10mbps","5mbps","2mbps","1mbps"],
"transcode_enabled": bool, "transcode_user_allowed": bool,
"season_download": bool, "series_monitoring": bool }
```
**New — managed-entry lifecycle + offline bundle:**
```
PATCH /downloads/{id} body {status:"downloading"|"completed"}
GET /downloads/{id}/manifest → OfflineManifest
GET /downloads/batches/{batch_id}/manifests → {manifests:[OfflineManifest]}
GET /downloads/{id}/artwork/{kind} → image bytes (poster|backdrop|logo)
GET /downloads/{id}/subtitles/{ref} → subtitle bytes
```
**Progress reconciliation:** additive fields on existing `POST /sync/progress` and `GET /progress`
(above). No new routes.
Routes mount in the existing authenticated group where the current download routes live
(`internal/api/router.go:2121`). **Every managed-entry endpoint — create, list, `PATCH`, `DELETE`,
`/file`, `/manifest`, `/artwork`, `/subtitles` — is profile-scoped (`RequireProfile`) and authorizes the
row on `(user_id, profile_id, header device_id)`** (not `user_id` alone — household profiles share a
`user_id`, so a user-only check would leak one profile's downloads to another). A managed call requires
`X-Silo-Device-Id` (missing header on a managed create → `400 device_id_required`); ephemeral web rows
omit it and stay account-scoped. Byte-serving and asset endpoints additionally **re-check per-profile
content/library access** (the existing access filter / `EnsureAccessible`) before serving, so a stale or
out-of-scope row — e.g. a child profile, or a profile that lost library access — cannot pull restricted
media by download id even if the row exists.
## Data flows
**Ephemeral web download (reshaped existing flow, quality-aware)**
```
POST /downloads {content_id, quality:"original", caps:{...}} (no device id → ephemeral)
→ Resolve quality/delivery · permission · quota
→ compatibility artifact: ArtifactManager.Ensure → preparing|ready ; direct original: ready immediately
client → GET /downloads/{id}/file → serveContent(source|artifact, Range, throttle)
```
**Register a managed bitrate download (mobile)**
```
POST /downloads {content_id, quality:"5mbps", caps:{...}} (+X-Silo-Device-Id, +X-Profile-Id)
→ Resolve: cfg.TranscodeEnabled && user.DownloadTranscodeAllowed → delivery_format=transcode
→ upsert downloads (device entry, status=preparing)
→ ArtifactManager.Ensure(media_file_id, params): ON CONFLICT dedup → enqueue encode if new
encode worker → playback.PrepareFile(... -movflags +faststart) → rename → artifact=ready
→ linked rows=ready → events hub notify
client (on notify) → GET /downloads/{id}/file → serveContent(artifact, Range, throttle)
→ GET /downloads/{id}/manifest → download ArtworkURLs.* and Subtitles[].FetchURL once
→ PATCH /downloads/{id} {status:"completed"}
```
**Offline progress, reconnect**
```
(offline) client queues {media_item_id, position, duration, updated_at} per stop
(online) POST /sync/progress {items:[... updated_at ...]} → SetProgressIfNewer per item
GET /progress?since=<cursor> → rows changed elsewhere since last sync
```
## Phased build sequence (each phase independently shippable)
- **Phase 0 — Reshape + scaffolding (coordinated web update).** Create `internal/downloads`; reshape the
`downloads` table (device plus quality/delivery columns, widened status) and the `/downloads` +
`/direct-download` endpoints to be quality-aware; port `BandwidthManager`/`QuantityLimiter`/serving;
extend `config.DownloadConfig` (+ new keys, default-off); add quality/delivery constants + sentinel
errors; add `GET /downloads/capability`. Update the web app's download hooks/components
(`web/src/hooks/queries/downloads.ts`, `web/src/components/DownloadVersionPicker.tsx`) to the reshaped
DTOs. Delete `internal/download`.
- **Phase 1 — Managed device entries.** Device-aware create/list/patch/delete; managed-entry serving for
`original`; `X-Silo-Device-Id` handling + the device-entry unique constraint. **Invariant 2** lands
here: profile+device authorization on every managed endpoint, header-only `device_id` authority.
*Acceptance tests:* a second profile on the same `user_id` is denied a managed row's `/file` and
`PATCH`/`DELETE`; a body/query `device_id` cannot override the header.
- **Phase 2 — Offline manifest.** `ManifestBuilder`; manifest + artwork + subtitle proxy endpoints, each
under the invariant-2 authorization + a per-profile content-access re-check before serving.
*Acceptance test:* a child/library-restricted profile is denied artwork/subtitle/manifest for an
out-of-scope download id.
- **Phase 3 — Prepare-to-file (remux + transcode) [v1-critical].** `download_artifacts` migration (incl.
lease/attempt columns); `ArtifactManager` as a **durable leased queue** (transactional claim, lease
heartbeat, startup sweep, `.part` cleanup) per **invariant 3**; `playback.PrepareFile`; encode worker on
`taskmanager`; ready-state serving; events notification; cleanup job. Turn `download.transcode_enabled`
on. *Acceptance test:* kill the process mid-encode → on restart the artifact re-enqueues and the linked
download reaches `ready` (no permanent `preparing`); concurrent workers never double-encode.
- **Phase 4 — Progress reconciliation.** `event_at` (clamped) + server-assigned `synced_seq` columns on
`watch_progress`; clamped `updated_at` on `POST /sync/progress`; server-token `?since=` on
`GET /progress`; `ListProgressSince` in both stores + store test suite, per **invariant 1**.
*Acceptance test:* a future-dated client `updated_at` is clamped (cannot win over a later real write)
and never advances another device's cursor.
Phase 0 precedes all and is the coordinated server+web reshape. Phases 1/2/4 are independent; Phase 3
depends on 1. Remux can ship in Phase 3 even if transcode stays admin-off.
## Client impact
- **Android (`silo-android`):** persist a stable `device_id` (UUID in app storage) and send
`X-Silo-Device-Id`; call `/downloads/capability` on login to gate UI; register downloads, poll/listen
for `ready`, pull the file, store the manifest + artwork + subtitle files locally; play the local file
with ExoPlayer; queue offline progress with timestamps in a Room table and flush via
`POST /sync/progress`, then pull deltas with `GET /progress?since=`.
- **Apple (`silo-apple`):** same flow; `device_id` from `identifierForVendor` (or stored UUID on macOS);
`URLSession` background download against `GET /downloads/{id}/file`; AVFoundation plays the local file;
offline progress queue drained on reconnect. The manifest's markers/chapters reuse the types already
used for online playback.
- **Web (`web/`):** updated in lockstep in **Phase 0** — the download hooks/components move to the
reshaped `/downloads` DTOs (quality/effective quality/delivery fields, new statuses). New admin additions: a
`download.transcode_enabled` toggle in admin settings, plus the existing per-user
`download_transcode_allowed` switch in `web/src/pages/AdminUserDetail.tsx`. No offline-library UI on web.
## Security & reliability invariants (must-hold)
These three are correctness requirements, not nice-to-haves; each ships with the acceptance test named in
the phase plan. They were hardened in response to an adversarial design review.
1. **Server-owned sync ordering.** Cross-device delta delivery (`?since=`) is driven only by the
server-assigned `synced_seq`; the client clock is bounded metadata (`event_at`, clamped to
`now + skew`) used only for LWW conflict resolution on the caller's own profile. A client-supplied
timestamp can never advance another reader's cursor or lock in a far-future "win."
2. **Full profile + device authorization.** Every managed-entry endpoint authorizes on
`(user_id, profile_id, header device_id)` and re-checks per-profile content/library access before
serving bytes/assets. A download id is never sufficient; `device_id` authority is the header only.
This holds the "no cross-device visibility" and "no cross-profile leakage" invariants given that
household profiles share a `user_id`.
3. **Durable artifact recovery.** Artifact preparation is a leased, attempt-counted, transactionally
claimed queue with a startup sweep, so no crash/restart can strand a download in `preparing` or
double-encode the same artifact.
## Risks & trade-offs
- **Coordinated server+web reshape (Phase 0).** Reshaping `/downloads` requires the web app to land in
the same milestone. Mitigated because both are first-party and the reshape is mechanical (added fields,
not removed behavior for the existing web path).
- **One table, two lifecycles.** A nullable `device_id` plus a wide status enum carries both ephemeral
and managed rows. Deliberate and managed (partial unique index + MATCH-SIMPLE FK keep the two modes'
invariants clean); the alternative two-table split was only justified by the now-lifted compat
constraint.
- **Artifact disk + CPU cost.** Mitigated by `(media_file_id, format, params_hash)` dedup (shared across
web/devices/users), a bounded encode pool, LRU/byte-budget cleanup, and transcode being **off by
default** (admin opt-in).
- **Remux is not free for a clean downloadable file.** `+faststart` needs a finalization pass, so even
remux is a prepare-to-file job. It is fast (`-c copy`) but not instant; `preparing → ready` + push
notification covers the UX.
- **`content_id` can change on rescan.** The manifest carries `StableIdentity` (provider IDs +
season/episode) mirroring `WatchIdentity` so the client re-resolves after a catalog rebind; the stale
row doesn't break the on-device file.
- **No expiry by decision.** A revoked `download_allowed` stops *future* serves (re-checked each serve,
as today at `internal/download/service.go:345`) but cannot remove an already-downloaded file. Call this
out in admin copy.
- **LWW vs. client clocks (resolved, see invariant 1).** Conflict resolution uses a clamped `event_at`
and delivery uses the server-owned `synced_seq`, so a skewed/malicious clock is bounded to "now" on its
own profile and cannot poison cursors. Residual: an honest-but-wrong client clock within the skew
window can still mis-order two of *its own* near-simultaneous events — acceptable.
- **Authorization breadth (resolved, see invariant 2).** The per-endpoint profile+device authorization
and pre-serve access re-check are load-bearing; the acceptance suite must include a cross-profile and a
cross-device denial test, since a regression here is a content-access leak, not just a bug.
- **`UserStore` gains a method + `watch_progress` gains columns.** `ListProgressSince` plus the
`event_at`/`synced_seq` columns touch every store implementor/stub and the per-user schema — all in the
Phase 4 PR.
- **Quota semantics.** v1: all download rows (web + device) count toward the **same** per-user
`QuantityLimiter`; revisit if too coarse.
## Out of scope (deferred)
Cross-device download visibility; download expiry/lease/DRM; cumulative per-user storage-byte quotas;
push-to-client "your transcode is ready" beyond the existing events hub; server-initiated deletion of
client-side files.
## Appendix — proposal-ready summary (maps to the v1 capability template)
- **Summary:** Mobile apps can download movies/episodes for fully offline playback — choosing original
or an admin-gated bitrate quality — and watch progress stays consistent when the device reconnects.
The web app's existing download feature is folded into the same reshaped contract. Remux remains an
internal delivery format, not a client-facing quality preset.
- **User value / why v1:** Offline playback is table-stakes for mobile media clients (commutes, flights,
spotty networks) and the client teams need the download + sync contract locked to build against.
Reshaping the existing download contract now — before lock and before any mobile client ships against
it — is the right time; the only current consumer is the first-party web app, updated in lockstep.
- **API surface:** the reshaped + new `/downloads/*` endpoints listed in "API surface" above (capability,
device-aware create/list/patch, file serving, manifest/artwork/subtitle proxies) plus
`updated_at`/`?since=` on the progress endpoints. **Note for reviewers:** this *changes* the existing
`/downloads` contract (additive-only is intentionally waived pre-lock for this first-party endpoint).
- **Client impact:** Android — full offline UI + sync; Apple — full offline UI + sync; Web — lockstep
Phase-0 DTO update + admin toggles.
- **Rough size (server-side):** **L** (new package replacing the old one, table reshape + artifacts
table, async encode pipeline, manifest assembly, additive progress sync).
@@ -0,0 +1,220 @@
# Download limits & restrictions (downloads v2) — design
**Goal:** Extend downloads v2 with the restriction knobs admins actually need: (1) a **quality
ceiling** — server-wide and per-user — so a user can be allowed to download, even transcode, but
only up to a capped bitrate; (2) a **batch size cap** protecting the server from a single series
request fanning out into hundreds of rows and transcode jobs; (3) **per-user overrides** for the
existing global quantity and bandwidth limits, following the established `users.max_streams` /
`users.max_transcodes` pattern, so households can tier users without changing the global policy.
**Status:** Draft — design input for a v1 capability proposal (Silo v1 scope gate,
`docs/architecture/v1-scope.md`). All API changes are additive within `/api/v1`.
Decisions locked with the requester:
- **No device/user storage caps.** Device disk is the client's job (per-subscription
`max_storage_bytes` already covers the monitoring case); server disk is already covered by
`download.artifact_max_bytes`; serving volume is already covered by bandwidth + per-period quotas.
- **No expiry / lease / revocation of already-downloaded files.** Unchanged from the downloads v2
design (`docs/superpowers/specs/2026-06-18-offline-sync-mobile-design.md`).
- **Transcode restriction already exists** (`download.transcode_enabled` +
`users.download_transcode_allowed`) and is kept as-is; the quality ceiling composes with it
("transcode allowed, but only up to 5mbps").
- **Admin observability (downloads dashboard, admin list/revoke endpoints) is a separate effort**,
not part of this spec.
> Commands and paths in this document are repository-relative; assume the repository root is the cwd.
## Current state (what this builds on)
Enforcement today lives in `internal/downloads`:
- **Policy** — `policy.go`: `DownloadQualityResolver.Resolve` validates a requested quality from the
ladder `original > 20mbps > 10mbps > 5mbps > 2mbps > 1mbps` and gates bitrate presets on
`cfg.TranscodeEnabled && user.DownloadTranscodeAllowed`. `PresetsFor` advertises the fulfillable
ladder through `GET /api/v1/downloads/capability` (`quality_presets`), so clients render only what
the server will honor.
- **Quantity** — `limiter.go`: `QuantityLimiter.Check(ctx, userID, batchSize)` enforces the global
`download.max_concurrent_per_user` and `download.max_per_period` / `download.period_duration`
settings (0 = unlimited) against `CountActiveByUser` / `CountByUserSince`.
- **Bandwidth** — `bandwidth.go`: `BandwidthManager` token-buckets file serving with one server-wide
rate and one per-user rate (`download.server_bandwidth_mbps`, `download.user_bandwidth_mbps`),
caching one `rate.Limiter` per user id.
- **Per-user flags** — `users.download_allowed`, `users.download_transcode_allowed`
(`internal/auth/repository.go`), exposed on the admin users API (`internal/api/handlers/admin.go`)
and web UI (`web/src/pages/AdminUserDetail.tsx`).
- **Bulk** — `service.go` `createSeriesScoped` fans a series/season request into managed entries with
no upper bound; `resolveBulkQuality` intentionally keeps batches **original-only**
(`ErrBulkQualityUnavailable`). Subscription sync auto-registers in-scope episodes without
consuming the quantity limiter (by design).
- **Serve-time re-check** — `Service.ServeFile` re-resolves policy on every file request, so
disabling downloads or revoking a user flag takes effect immediately for anything not yet pulled.
Settings are freeform keys with defaults and validation in `internal/config/db_loader.go`, hot-
reloaded by `Service.loadConfig` (which also reloads the limiter and bandwidth manager on change),
and edited in `web/src/pages/admin-settings/DownloadSettings.tsx`.
## 1. Quality ceiling
### Semantics
- New server setting **`download.max_quality`** (string; `""` = no cap; otherwise one of the quality
ladder values). New nullable per-user column **`users.download_max_quality`** (`NULL` = inherit the
server setting; otherwise one of the ladder values).
- The **effective ceiling** for a user is the *lower* of the server setting and the user column on
the ordered ladder (`original` is the top). A ceiling below `original` deliberately forbids
`original` — that is the point: "this user gets at most a 5mbps transcode, never the full file."
- Enforcement is **presets-first**: `PresetsFor` truncates the advertised ladder at the effective
ceiling, so compliant clients never offer a forbidden quality and **no client changes are
required** (clients already read `quality_presets` from the capability endpoint).
- `Resolve` independently rejects any request above the ceiling with a new sentinel
`ErrQualityNotPermitted` (handler maps it alongside the existing limit errors as HTTP 403,
distinct from `ErrInvalidQuality`), so a non-compliant client cannot bypass the cap.
- The **original-compatibility fallback** inside `Resolve` (original → remux/transcode when the
device can't play the source) must clamp its chosen `EffectiveQuality`/bitrate to the ceiling.
In practice a sub-original ceiling makes the fallback moot — `original` is already rejected — but
the clamp keeps the invariant local to `Resolve` rather than depending on call-site ordering.
- **Serve-time**: the existing `ServeFile` policy re-check additionally denies rows whose
`effective_quality` exceeds the user's *current* ceiling. Consistent with today's revocation
semantics: lowering a cap stops future serving of higher-quality registered entries; files already
on-device are untouched (no expiry).
### Interactions (documented behavior, not bugs)
- **Ceiling below `original` + `download_transcode_allowed = false`** → the fulfillable preset list
is empty and the capability endpoint reports no presets; the user effectively cannot download.
This is legal admin configuration. The admin UI shows a hint on the per-user field when this
combination is selected ("a sub-original cap requires the transcode permission").
- **Series/season/monitoring downloads are original-only today** (`resolveBulkQuality`). A user
capped below `original` therefore cannot create series/season batches or receive subscription
auto-registrations; those paths return `ErrQualityNotPermitted` (create) or skip registration with
a `SkippedDownload` reason `quality_not_permitted` (subscription sync). This is accepted for now
and called out here so it is not rediscovered as a bug. Lifting it means bulk transcode — a
deliberate follow-up, not smuggled into this change.
### Implementation
- `Resolve`/`PresetsFor` gain the ceiling input. Rather than growing the parameter lists further,
introduce a small `PolicyInput`/`userPolicy` struct (user + cfg + artifactsAvailable) if the
signatures get unwieldy — implementer's choice; behavior is what is specified above.
- Ladder comparison helper in `internal/downloads/model.go` (e.g. `QualityAtMost(q, ceiling) bool`
using the `QualityPresets` order) — no string parsing of bitrates.
- Validation: `db_loader.go` rejects a `download.max_quality` value not on the ladder; the users
API rejects invalid `download_max_quality` the same way (shared validation via
`downloads.ValidQuality`, with `""`/`NULL` allowed).
## 2. Batch size cap
- New server setting **`download.max_batch_size`** (int; default **100**; 0 = unlimited): the
maximum number of managed entries one explicit series/season create may register, and the maximum
auto-registered per subscription-sync pass.
- **Explicit series/season create** (`createSeriesScoped`): if the in-scope episode count (after
access filtering and skip computation) exceeds the cap, **reject** the whole request with a new
sentinel `ErrBatchTooLarge` (HTTP 400, error body includes the cap so clients can message "try a
season at a time or subscribe"). Rejection is chosen over truncation because a silently partial
one-shot batch never self-heals — there is no later pass that picks up the remainder.
- **Subscription sync** (`subscription_service.go`): **truncate** each pass at the cap (stable
episode order). Truncation is correct here because sync recomputes in-scope-but-unregistered
episodes every pass, so the remainder registers on subsequent syncs — natural pagination, and one
monster series cannot monopolize a sync pass or the artifact queue.
- The capability response additively gains **`max_batch_size`** (int, 0 = unlimited) so clients can
preflight ("this series has 340 episodes; the server accepts 100 per request") instead of
discovering the limit by error.
## 3. Per-user quantity & bandwidth overrides
- Three new nullable columns on `users`, mirroring the `max_streams` pattern
(`internal/models/user.go`, `internal/auth/repository.go`):
- `download_max_concurrent int` — overrides `download.max_concurrent_per_user`
- `download_max_per_period int` — overrides `download.max_per_period` (window stays the global
`download.period_duration`; a per-user window is deliberately out of scope)
- `download_bandwidth_mbps int` — overrides `download.user_bandwidth_mbps`
- Semantics: `NULL` = inherit the global setting; **0 = explicitly unlimited** (matches the global
settings' zero-means-unlimited convention, and is the reason `NULL` and `0` must stay distinct).
- **QuantityLimiter**: `Check` gains the caller-resolved user overrides — e.g.
`Check(ctx, userID, batchSize, overrides Overrides)` where `Overrides` carries the two optional
ints. Every existing call site already has the `*models.User` loaded (the create paths load it
for policy), so resolution stays in `service.go` and the limiter remains storage-free.
- **BandwidthManager**: the per-user limiter cache currently assumes one global rate. Store the
rate alongside each cached limiter and recreate the entry when the effective rate for that user
changes (`ThrottledReader` gains the resolved per-user bps from the caller; `ServeFile` already
loads the user for its policy re-check). `Reload` keeps its existing clear-all behavior.
- **Admin API**: the three fields (plus `download_max_quality` from §1) are added to the admin user
create/update/response DTOs in `internal/api/handlers/admin.go` and to
`UpdateUserInput`/`CreateUserInput` in `internal/models/user.go` +
`internal/auth/repository.go` — the exact `max_streams` additive pattern.
## Data model (Goose migration)
One timestamped migration (`make migrate-create NAME=user_download_limits`):
```sql
ALTER TABLE users
ADD COLUMN download_max_quality text,
ADD COLUMN download_max_concurrent integer,
ADD COLUMN download_max_per_period integer,
ADD COLUMN download_bandwidth_mbps integer;
```
All nullable, no defaults, no backfill — `NULL` means "inherit global" for every column. The two new
settings keys (`download.max_quality`, `download.max_batch_size`) are settings rows, not schema.
## API surface (all additive)
- `GET /api/v1/downloads/capability` — `quality_presets` now reflects the effective ceiling
(existing field, narrowed values); new field `max_batch_size`.
- `POST /api/v1/downloads` — new 403 error case (`quality_not_permitted`) and new 400 error case
(`batch_too_large`, body includes the configured cap).
- `POST /api/v1/downloads/subscriptions/sync` — skipped registrations may carry the new
`SkippedDownload` reason `quality_not_permitted`.
- Admin users endpoints — four new optional request fields / response fields (§1, §3).
- Settings endpoints — two new keys; both hot-reloadable (no restart-required entry in
`internal/config/restart_keys.go`).
## Web admin UI
- `web/src/pages/admin-settings/DownloadSettings.tsx`: "Max Download Quality" (select over the
ladder + "No limit") under Quantity Limits or a new Restrictions group; "Max Batch Size" numeric
field with the 0-=-unlimited hint.
- `web/src/pages/AdminUserDetail.tsx` (+ the user edit dialog): rows for the four per-user fields,
each showing the inherited global value when unset (as the streams/transcodes fields do), plus the
§1 hint when a sub-original cap is combined with transcode-not-allowed.
## Enforcement summary
| Restriction | Hard/soft | Enforced at |
|---|---|---|
| Quality ceiling | Hard | `Resolve` (create), `PresetsFor` (capability), `ServeFile` re-check |
| Batch size cap | Hard | `createSeriesScoped` (reject), subscription sync (truncate per pass) |
| Concurrent / per-period overrides | Hard | `QuantityLimiter.Check` on every create path |
| Bandwidth override | Hard | `ThrottledReader` on every byte served |
Nothing in this spec is client-enforced; the client-pull architecture is unchanged.
## Client follow-up (silo-android, silo-apple)
- **Quality ceiling / presets:** zero work if the client renders `quality_presets` from the
capability endpoint (the downloads v2 contract). Audit for hardcoded ladders.
- **Batch errors:** handle `batch_too_large` with a useful prompt (download per season / subscribe
instead of a raw error toast). Handle empty `quality_presets` as "downloads unavailable for this
account", which the capability contract already implies.
- No manifest or sync-protocol changes.
## Testing
- `policy_test.go`: ceiling truncation of `PresetsFor` (server-only, user-only, both, ties at
`original`); `Resolve` rejection above the ceiling; fallback clamp; the
cap-without-transcode-permission → empty presets case.
- `limiter_test.go` (new cases): override precedence NULL/0/positive vs. global for both limits.
- `service_*_test.go`: batch reject at cap boundary (cap, cap+1); subscription sync truncation and
next-pass continuation; serve-time denial after a cap is lowered.
- Handler tests for the new error mappings and admin-user field round-trips — run in a
libvips-capable container (bare-host `go test ./...` skips `internal/api/handlers`).
## Out of scope / follow-ups
- Admin downloads dashboard + admin list/revoke endpoints (observability; separate spec).
- Bulk (series/season) transcode quality — prerequisite for capped users to use series downloads
and monitoring; lifts the `resolveBulkQuality` original-only rule.
- Profile-level download permission (parental-controls integration; downloads are already keyed by
`profile_id`, so the data model is ready).
- Per-user managed-device count cap — revisit only if storage/quantity limits prove insufficient.
+6
View File
@@ -86,9 +86,15 @@ func (s stubStore) GetProgress(context.Context, string, string) (*userstore.Watc
func (s stubStore) ListProgress(context.Context, string, string, int, int) ([]userstore.WatchProgress, error) {
panic("unused")
}
func (s stubStore) ListProgressFiltered(context.Context, string, string, []string, *int, int, int) ([]userstore.WatchProgress, error) {
panic("unused")
}
func (s stubStore) ListProgressByMediaItems(context.Context, string, []string) (map[string]userstore.WatchProgress, error) {
panic("unused")
}
func (s stubStore) ListProgressSince(context.Context, string, string) ([]userstore.WatchProgress, string, error) {
panic("unused")
}
func (s stubStore) AddHistory(context.Context, userstore.WatchHistoryEntry) error { panic("unused") }
func (s stubStore) AddHistoryIfMissing(context.Context, userstore.WatchHistoryEntry) (bool, error) {
panic("unused")
+6
View File
@@ -89,6 +89,7 @@ func (l *PGLibraryRefreshItemLister) ListLibraryItems(ctx context.Context, libra
query := `
SELECT mi.content_id, COALESCE(mi.tmdb_id, ''), COALESCE(mi.tvdb_id, ''), COALESCE(mi.imdb_id, '')
FROM media_item_libraries mil
JOIN media_folders f ON f.id = mil.media_folder_id
JOIN media_items mi ON mi.content_id = mil.content_id
WHERE mil.media_folder_id = $1`
args := []any{libraryID}
@@ -123,6 +124,11 @@ func (l *PGLibraryRefreshItemLister) ListLibraryItems(ctx context.Context, libra
FROM stale_media_ids smi
WHERE smi.content_id = mi.content_id
)
OR (
COALESCE(mi.default_metadata_language, '') <> ''
AND LOWER(TRIM(mi.default_metadata_language))
<> LOWER(TRIM(COALESCE(NULLIF(f.metadata_language, ''), 'en')))
)
)`
}
@@ -0,0 +1,93 @@
package adminjob
import (
"context"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// TestQuickRefreshListsLanguageMismatchedItems covers the listing half of
// issue #211: quick-mode library refresh must include items whose stamped
// default_metadata_language differs from the library's configured metadata
// language, even when the item is otherwise complete (overview, poster,
// backdrop all present) — otherwise a library language change never revisits
// already-complete items.
func TestQuickRefreshListsLanguageMismatchedItems(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
suffix := time.Now().UnixNano()
mismatchID := fmt.Sprintf("lang-mismatch-%d", suffix)
matchedID := fmt.Sprintf("lang-matched-%d", suffix)
var folderID int
if err := pool.QueryRow(ctx, `
INSERT INTO media_folders (type, name, enabled, metadata_language)
VALUES ('movies', 'Lang Refresh Test', true, 'da')
RETURNING id
`).Scan(&folderID); err != nil {
t.Fatalf("seed folder: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = ANY($1)`, []string{mismatchID, matchedID})
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
for _, row := range []struct {
id, lang string
}{
{mismatchID, "zh"},
{matchedID, "da"},
} {
if _, err := pool.Exec(ctx, `
INSERT INTO media_items (
content_id, type, title, status, genres, tmdb_id,
default_metadata_language, overview, poster_path, backdrop_path,
last_refreshed, refresh_failures, episode_metadata_incomplete
) VALUES ($1, 'movie', 'Complete Item', 'matched', '{}'::text[], '42',
$2, 'An overview', '/p.jpg', '/b.jpg', NOW(), 0, FALSE)
`, row.id, row.lang); err != nil {
t.Fatalf("seed media item %s: %v", row.id, err)
}
if _, err := pool.Exec(ctx, `
INSERT INTO media_item_libraries (content_id, media_folder_id)
VALUES ($1, $2)
`, row.id, folderID); err != nil {
t.Fatalf("link media item %s: %v", row.id, err)
}
}
lister := NewPGLibraryRefreshItemLister(pool)
items, err := lister.ListLibraryItems(ctx, folderID, LibraryRefreshModeQuick)
if err != nil {
t.Fatalf("ListLibraryItems: %v", err)
}
var sawMismatch, sawMatched bool
for _, item := range items {
switch item.ContentID {
case mismatchID:
sawMismatch = true
case matchedID:
sawMatched = true
}
}
if !sawMismatch {
t.Errorf("quick refresh must include complete item with stamped language differing from the library language")
}
if sawMatched {
t.Errorf("quick refresh must not include complete item whose stamped language matches the library language")
}
}
+15
View File
@@ -2141,6 +2141,21 @@ func (h *AdminHandler) HandleUpdateSetting(w http.ResponseWriter, r *http.Reques
"subtitle_ai.transcribe_quota_period must be day, week, or month")
return
}
case notifications.SettingApplePushDeliveryEnabled:
enabled, err := strconv.ParseBool(strings.TrimSpace(req.Value))
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "notifications.apple_push_delivery_enabled must be true or false")
return
}
req.Value = strconv.FormatBool(enabled)
case notifications.SettingPushRelayURL, notifications.SettingPushRelayDeploymentID, notifications.SettingPushRelayAPIKey:
// The registration flow persists the relay URL, deployment id, and API
// key together; a direct write to any of them desyncs the stored URL
// from the credentials the relay minted for it (and feeds an arbitrary
// id into the next rotation request).
writeError(w, http.StatusBadRequest, "bad_request",
key+" is managed by the push relay registration flow; use POST /admin/notifications/push/relay/register")
return
case catalog.SearchSettingProvider:
switch strings.TrimSpace(strings.ToLower(req.Value)) {
case catalog.SearchProviderPostgres, catalog.SearchProviderMeilisearch:
+279
View File
@@ -0,0 +1,279 @@
package handlers
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/Silo-Server/silo-server/internal/notifications"
)
type AdminApplePushHandler struct {
system *notifications.System
settings ServerSettingsStore
client httpDoer
}
type httpDoer interface {
Do(*http.Request) (*http.Response, error)
}
func NewAdminApplePushHandler(system *notifications.System, settings ServerSettingsStore) *AdminApplePushHandler {
return &AdminApplePushHandler{
system: system,
settings: settings,
client: &http.Client{Timeout: 10 * time.Second},
}
}
type adminApplePushTestRequest struct {
ProfileID string `json:"profile_id"`
ServerDeviceID string `json:"server_device_id"`
}
type adminApplePushTestResponse struct {
AttemptID string `json:"attempt_id"`
PushDeviceID string `json:"push_device_id"`
ServerDeviceID string `json:"server_device_id"`
Outcome string `json:"outcome"`
RelayRequestID string `json:"relay_request_id,omitempty"`
UpstreamStatus *int `json:"upstream_status,omitempty"`
UpstreamReason string `json:"upstream_reason,omitempty"`
FailureMessage string `json:"failure_message,omitempty"`
}
type adminPushRelayRegisterRequest struct {
RelayURL string `json:"relay_url"`
}
type pushRelayRegisterRequest struct {
DeploymentID string `json:"deployment_id,omitempty"`
}
type pushRelayRegisterResponse struct {
RequestID string `json:"request_id"`
DeploymentID string `json:"deployment_id"`
APIKey string `json:"api_key"`
KeyPrefix string `json:"key_prefix"`
APNsTopics []string `json:"apns_topics"`
}
type pushRelayNestedError struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
RequestID string `json:"request_id"`
} `json:"error"`
}
type adminPushRelayRegisterResponse struct {
RelayURL string `json:"relay_url"`
DeploymentID string `json:"deployment_id"`
KeyPrefix string `json:"key_prefix"`
APIKeyConfigured bool `json:"api_key_configured"`
RelayRequestID string `json:"relay_request_id,omitempty"`
APNsTopics []string `json:"apns_topics,omitempty"`
}
// HandleTest handles POST /admin/notifications/push/apple/test.
func (h *AdminApplePushHandler) HandleTest(w http.ResponseWriter, r *http.Request) {
if h == nil || h.system == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Apple push delivery is not available")
return
}
var req adminApplePushTestRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
result, err := h.system.SendApplePushTest(r.Context(), req.ProfileID, req.ServerDeviceID)
if err != nil {
switch {
case errors.Is(err, notifications.ErrPushDeliveryInvalid):
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
case errors.Is(err, notifications.ErrPushDeliveryNotFound):
writeError(w, http.StatusNotFound, "not_found", "Apple push device not found")
case errors.Is(err, notifications.ErrPushDeliveryUnavailable):
writeError(w, http.StatusServiceUnavailable, "unavailable", "Apple push delivery is not available")
default:
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to send Apple push test")
}
return
}
writeJSON(w, http.StatusOK, adminApplePushTestResponse{
AttemptID: result.AttemptID,
PushDeviceID: result.PushDeviceID,
ServerDeviceID: result.ServerDeviceID,
Outcome: result.Outcome,
RelayRequestID: result.RelayRequestID,
UpstreamStatus: result.UpstreamStatus,
UpstreamReason: result.UpstreamReason,
FailureMessage: result.FailureMessage,
})
}
// HandleRegisterRelay handles POST /admin/notifications/push/relay/register.
func (h *AdminApplePushHandler) HandleRegisterRelay(w http.ResponseWriter, r *http.Request) {
if h == nil || h.settings == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Settings store is not available")
return
}
var req adminPushRelayRegisterRequest
dec := json.NewDecoder(r.Body)
dec.DisallowUnknownFields()
if err := dec.Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
relayURL, err := normalizePushRelayURL(req.RelayURL)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
existingDeploymentID, err := h.settings.Get(r.Context(), notifications.SettingPushRelayDeploymentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "settings_error", "Failed to load relay deployment id")
return
}
relayResp, err := h.registerWithRelay(r, relayURL, strings.TrimSpace(existingDeploymentID))
if err != nil {
status, code, message := mapRelayRegistrationError(err)
writeError(w, status, code, message)
return
}
if strings.TrimSpace(relayResp.DeploymentID) == "" || strings.TrimSpace(relayResp.APIKey) == "" {
writeError(w, http.StatusBadGateway, "relay_bad_response", "Push relay returned an incomplete registration response")
return
}
if err := h.settings.Set(r.Context(), notifications.SettingPushRelayURL, relayURL); err != nil {
writeError(w, http.StatusInternalServerError, "settings_error", "Failed to save push relay URL")
return
}
if err := h.settings.Set(r.Context(), notifications.SettingPushRelayDeploymentID, relayResp.DeploymentID); err != nil {
writeError(w, http.StatusInternalServerError, "settings_error", "Failed to save push relay deployment id")
return
}
if err := h.settings.Set(r.Context(), notifications.SettingPushRelayAPIKey, relayResp.APIKey); err != nil {
writeError(w, http.StatusInternalServerError, "settings_error", "Failed to save push relay API key")
return
}
if h.system != nil && h.system.Settings != nil {
h.system.Settings.Invalidate(
notifications.SettingPushRelayURL,
notifications.SettingPushRelayDeploymentID,
notifications.SettingPushRelayAPIKey,
)
}
writeJSON(w, http.StatusOK, adminPushRelayRegisterResponse{
RelayURL: relayURL,
DeploymentID: relayResp.DeploymentID,
KeyPrefix: relayResp.KeyPrefix,
APIKeyConfigured: true,
RelayRequestID: relayResp.RequestID,
APNsTopics: relayResp.APNsTopics,
})
}
func (h *AdminApplePushHandler) registerWithRelay(r *http.Request, relayURL, deploymentID string) (*pushRelayRegisterResponse, error) {
body, err := json.Marshal(pushRelayRegisterRequest{
DeploymentID: deploymentID,
})
if err != nil {
return nil, err
}
httpReq, err := http.NewRequestWithContext(r.Context(), http.MethodPost, relayURL+"/v1/deployments/register", bytes.NewReader(body))
if err != nil {
return nil, err
}
httpReq.Header.Set("Content-Type", "application/json")
httpReq.Header.Set("User-Agent", "Silo-Server/PushRelayRegistration")
resp, err := h.client.Do(httpReq)
if err != nil {
return nil, relayRegistrationError{status: http.StatusBadGateway, code: "relay_unreachable", message: "Push relay could not be reached"}
}
defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(io.LimitReader(resp.Body, 16<<10))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
var parsed pushRelayNestedError
_ = json.Unmarshal(data, &parsed)
code := parsed.Error.Code
if code == "" {
code = fmt.Sprintf("relay_http_%d", resp.StatusCode)
}
message := parsed.Error.Message
if message == "" {
message = http.StatusText(resp.StatusCode)
}
return nil, relayRegistrationError{status: resp.StatusCode, code: code, message: message}
}
var parsed pushRelayRegisterResponse
if err := json.Unmarshal(data, &parsed); err != nil {
return nil, relayRegistrationError{status: http.StatusBadGateway, code: "relay_bad_response", message: "Push relay returned invalid JSON"}
}
return &parsed, nil
}
type relayRegistrationError struct {
status int
code string
message string
}
func (e relayRegistrationError) Error() string {
return e.code
}
func mapRelayRegistrationError(err error) (int, string, string) {
var relayErr relayRegistrationError
if !errors.As(err, &relayErr) {
return http.StatusInternalServerError, "internal_error", "Failed to register push relay"
}
switch relayErr.status {
case http.StatusForbidden:
return http.StatusUnprocessableEntity, "relay_deployment_rejected", "Push relay rejected this deployment"
case http.StatusTooManyRequests:
return http.StatusTooManyRequests, "relay_rate_limited", relayErr.message
case http.StatusServiceUnavailable:
return http.StatusServiceUnavailable, "relay_unavailable", relayErr.message
default:
if relayErr.status >= 500 {
return http.StatusBadGateway, "relay_error", relayErr.message
}
return http.StatusBadGateway, relayErr.code, relayErr.message
}
}
// normalizePushRelayURLValue trims a relay URL and enforces https with a
// host. Empty input stays empty so callers choose their own default.
func normalizePushRelayURLValue(raw string) (string, error) {
value := strings.TrimRight(strings.TrimSpace(raw), "/")
if value == "" {
return "", nil
}
parsed, err := url.Parse(value)
if err != nil || parsed.Scheme != "https" || parsed.Host == "" {
return "", errors.New("relay_url must be an https URL")
}
return value, nil
}
func normalizePushRelayURL(raw string) (string, error) {
value, err := normalizePushRelayURLValue(raw)
if err != nil {
return "", err
}
if value == "" {
return notifications.DefaultPushRelayURL, nil
}
return value, nil
}
@@ -0,0 +1,123 @@
package handlers
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/notifications"
)
func TestAdminApplePushHandlerUnavailableWithoutSystem(t *testing.T) {
h := NewAdminApplePushHandler(nil, nil)
rec := httptest.NewRecorder()
h.HandleTest(rec, httptest.NewRequest(http.MethodPost, "/admin/notifications/push/apple/test", strings.NewReader(`{}`)))
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("HandleTest without system = %d, want 503", rec.Code)
}
}
func TestAdminApplePushHandlerRejectsInvalidJSON(t *testing.T) {
h := NewAdminApplePushHandler(&notifications.System{}, nil)
rec := httptest.NewRecorder()
h.HandleTest(rec, httptest.NewRequest(http.MethodPost, "/admin/notifications/push/apple/test", strings.NewReader(`{`)))
if rec.Code != http.StatusBadRequest {
t.Fatalf("HandleTest invalid JSON = %d, want 400", rec.Code)
}
}
func TestAdminApplePushHandlerRegistersRelayAndStoresKey(t *testing.T) {
settings := &fakeServerSettingsStore{values: map[string]string{
notifications.SettingPushRelayDeploymentID: "01EXISTING",
}}
var relayReq map[string]string
relay := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/deployments/register" {
t.Fatalf("relay path = %s", r.URL.Path)
}
if err := json.NewDecoder(r.Body).Decode(&relayReq); err != nil {
t.Fatalf("decode relay request: %v", err)
}
writeJSON(w, http.StatusOK, pushRelayRegisterResponse{
RequestID: "relay-request",
DeploymentID: "01RETURNED",
APIKey: "rk_live_raw-key",
KeyPrefix: "rk_live_raw",
APNsTopics: []string{"org.siloserver.silo"},
})
}))
t.Cleanup(relay.Close)
h := NewAdminApplePushHandler(&notifications.System{Settings: notifications.NewSettings(settings)}, settings)
h.client = relay.Client()
rec := httptest.NewRecorder()
h.HandleRegisterRelay(rec, httptest.NewRequest(http.MethodPost, "/admin/notifications/push/relay/register", strings.NewReader(`{
"relay_url":"`+relay.URL+`"
}`)))
if rec.Code != http.StatusBadRequest {
t.Fatalf("http relay URL status = %d, want 400 because HTTPS is required", rec.Code)
}
httpsRelay := httptest.NewTLSServer(relay.Config.Handler)
t.Cleanup(httpsRelay.Close)
h.client = httpsRelay.Client()
rec = httptest.NewRecorder()
h.HandleRegisterRelay(rec, httptest.NewRequest(http.MethodPost, "/admin/notifications/push/relay/register", strings.NewReader(`{
"relay_url":"`+httpsRelay.URL+`"
}`)))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d (%s), want 200", rec.Code, rec.Body.String())
}
if relayReq["deployment_id"] != "01EXISTING" || len(relayReq) != 1 {
t.Fatalf("relay request = %+v", relayReq)
}
if settings.values[notifications.SettingPushRelayURL] != httpsRelay.URL {
t.Fatalf("stored relay URL = %q", settings.values[notifications.SettingPushRelayURL])
}
if settings.values[notifications.SettingPushRelayDeploymentID] != "01RETURNED" {
t.Fatalf("stored deployment id = %q", settings.values[notifications.SettingPushRelayDeploymentID])
}
if settings.values[notifications.SettingPushRelayAPIKey] != "rk_live_raw-key" {
t.Fatalf("stored api key = %q", settings.values[notifications.SettingPushRelayAPIKey])
}
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response: %v", err)
}
if _, ok := body["api_key"]; ok {
t.Fatal("response leaked raw api_key")
}
if body["api_key_configured"] != true || body["deployment_id"] != "01RETURNED" {
t.Fatalf("response = %+v", body)
}
}
func TestAdminApplePushHandlerMapsRelayRateLimit(t *testing.T) {
settings := &fakeServerSettingsStore{values: map[string]string{}}
relay := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusTooManyRequests, map[string]any{
"error": map[string]string{"code": "rate_limited", "message": "too many deployment registrations from this network"},
})
}))
t.Cleanup(relay.Close)
h := NewAdminApplePushHandler(&notifications.System{}, settings)
h.client = relay.Client()
rec := httptest.NewRecorder()
h.HandleRegisterRelay(rec, httptest.NewRequest(http.MethodPost, "/admin/notifications/push/relay/register", strings.NewReader(`{
"relay_url":"`+relay.URL+`"
}`)))
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("status = %d (%s), want 429", rec.Code, rec.Body.String())
}
if settings.values[notifications.SettingPushRelayAPIKey] != "" {
t.Fatal("api key was stored after relay rejected registration")
}
}
+6 -2
View File
@@ -161,7 +161,11 @@ func (h *AdminMatchHandler) HandleApplyItemMatch(w http.ResponseWriter, r *http.
return
}
if len(req.ProviderIDs) == 0 {
// Normalize keys and values the same way the search endpoint does so a
// caller-supplied key like "TMDB" or " tmdb " cannot bypass downstream
// provider-id handling (e.g. stale-ID suppression in metadata.Process).
providerIDs := normalizeMatchProviderIDs(req.ProviderIDs)
if len(providerIDs) == 0 {
writeError(w, http.StatusBadRequest, "bad_request", "At least one provider ID is required")
return
}
@@ -191,7 +195,7 @@ func (h *AdminMatchHandler) HandleApplyItemMatch(w http.ResponseWriter, r *http.
result, err := h.metadata.Process(r.Context(), metadata.ProcessRequest{
ContentID: contentID,
ProviderIDs: req.ProviderIDs,
ProviderIDs: providerIDs,
FolderID: folderIDStr,
Mode: metadata.ModeIdentify,
})
+61
View File
@@ -326,6 +326,67 @@ func TestAdminMatchApply_RequiresProviderIDs(t *testing.T) {
}
}
func TestAdminMatchApply_NormalizesProviderIDKeys(t *testing.T) {
items := &fakeMatchItemLookup{
items: map[string]*models.MediaItem{
"item-5": {ContentID: "item-5", Title: "Test", Type: "movie"},
},
}
metaSvc := &fakeMatchMetadataService{
processResult: &metadata.ProcessResult{ContentID: "item-5", Updated: true},
}
h := NewAdminMatchHandler(items, nil, metaSvc)
router := buildMatchRouter(h)
body, _ := json.Marshal(matchApplyRequest{
ProviderIDs: map[string]string{
" TMDB ": " 157336 ",
"blank": " ",
},
})
req := httptest.NewRequest(http.MethodPost, "/admin/items/item-5/match/apply", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", rec.Code, rec.Body.String())
}
if got := metaSvc.lastProcess.ProviderIDs["tmdb"]; got != "157336" {
t.Errorf("Process provider_ids[tmdb] = %q, want 157336 (keys/values must be normalized)", got)
}
if _, ok := metaSvc.lastProcess.ProviderIDs[" TMDB "]; ok {
t.Errorf("Process received raw key %q, want normalized keys only: %v", " TMDB ", metaSvc.lastProcess.ProviderIDs)
}
if _, ok := metaSvc.lastProcess.ProviderIDs["blank"]; ok {
t.Errorf("blank provider IDs should be dropped, got %v", metaSvc.lastProcess.ProviderIDs)
}
}
func TestAdminMatchApply_RejectsAllBlankProviderIDs(t *testing.T) {
items := &fakeMatchItemLookup{
items: map[string]*models.MediaItem{
"item-6": {ContentID: "item-6", Title: "Test", Type: "movie"},
},
}
h := NewAdminMatchHandler(items, nil, &fakeMatchMetadataService{})
router := buildMatchRouter(h)
body, _ := json.Marshal(matchApplyRequest{
ProviderIDs: map[string]string{"tmdb": " ", " ": "123"},
})
req := httptest.NewRequest(http.MethodPost, "/admin/items/item-6/match/apply", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for all-blank provider_ids, got %d: %s", rec.Code, rec.Body.String())
}
}
func TestAdminMatchApply_ItemNotFound(t *testing.T) {
items := &fakeMatchItemLookup{items: map[string]*models.MediaItem{}}
h := NewAdminMatchHandler(items, nil, &fakeMatchMetadataService{})
@@ -179,7 +179,7 @@ func (h *AdminPlaybackControlHandler) handleSessionCommand(w http.ResponseWriter
fallback := func() {
h.playback.forgetRealtimeCommand(commandID)
_ = h.playback.stopPlaybackSessionByID(context.Background(), sessionID)
_ = h.playback.stopPlaybackSessionByID(context.Background(), sessionID, true)
}
h.playback.rememberRealtimeCommand(commandID, sessionID, name)
@@ -41,6 +41,8 @@ type serverChannelResponse struct {
Enabled bool `json:"enabled"`
NotifyNewMovies bool `json:"notify_new_movies"`
NotifyNewEpisodes bool `json:"notify_new_episodes"`
NotifyNewAudiobooks bool `json:"notify_new_audiobooks"`
NotifyNewEbooks bool `json:"notify_new_ebooks"`
NotifyRequestSubmitted bool `json:"notify_request_submitted"`
NotifyRequestApproved bool `json:"notify_request_approved"`
NotifyRequestDeclined bool `json:"notify_request_declined"`
@@ -65,6 +67,8 @@ func serverChannelToResponse(ch notifications.ServerChannel) serverChannelRespon
Enabled: ch.Enabled,
NotifyNewMovies: ch.NotifyNewMovies,
NotifyNewEpisodes: ch.NotifyNewEpisodes,
NotifyNewAudiobooks: ch.NotifyNewAudiobooks,
NotifyNewEbooks: ch.NotifyNewEbooks,
NotifyRequestSubmitted: ch.NotifyRequestSubmitted,
NotifyRequestApproved: ch.NotifyRequestApproved,
NotifyRequestDeclined: ch.NotifyRequestDeclined,
@@ -86,6 +90,8 @@ type serverChannelRequest struct {
Enabled *bool `json:"enabled"`
NotifyNewMovies *bool `json:"notify_new_movies"`
NotifyNewEpisodes *bool `json:"notify_new_episodes"`
NotifyNewAudiobooks *bool `json:"notify_new_audiobooks"`
NotifyNewEbooks *bool `json:"notify_new_ebooks"`
NotifyRequestSubmitted *bool `json:"notify_request_submitted"`
NotifyRequestApproved *bool `json:"notify_request_approved"`
NotifyRequestDeclined *bool `json:"notify_request_declined"`
@@ -100,6 +106,8 @@ func (r serverChannelRequest) toInput() notifications.ServerChannelInput {
Enabled: r.Enabled,
NotifyNewMovies: r.NotifyNewMovies,
NotifyNewEpisodes: r.NotifyNewEpisodes,
NotifyNewAudiobooks: r.NotifyNewAudiobooks,
NotifyNewEbooks: r.NotifyNewEbooks,
NotifyRequestSubmitted: r.NotifyRequestSubmitted,
NotifyRequestApproved: r.NotifyRequestApproved,
NotifyRequestDeclined: r.NotifyRequestDeclined,
@@ -0,0 +1,233 @@
package handlers
import (
"encoding/json"
"errors"
"log/slog"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/downloads"
)
// subscriptionRequest is the JSON body for POST /downloads/subscriptions.
type subscriptionRequest struct {
SeriesID string `json:"series_id"`
Mode string `json:"mode"` // all | future | latest_season | specific_seasons
SeasonNumbers []int `json:"season_numbers,omitempty"`
DeleteWatched bool `json:"delete_watched,omitempty"`
MaxStorageBytes int64 `json:"max_storage_bytes,omitempty"`
}
// patchSubscriptionRequest is the JSON body for PATCH /downloads/subscriptions/{id}.
// Pointer fields distinguish "absent" from a zero value so a partial update only
// touches the fields the client sent.
type patchSubscriptionRequest struct {
Mode *string `json:"mode,omitempty"`
SeasonNumbers *[]int `json:"season_numbers,omitempty"`
DeleteWatched *bool `json:"delete_watched,omitempty"`
MaxStorageBytes *int64 `json:"max_storage_bytes,omitempty"`
Active *bool `json:"active,omitempty"`
}
// subscriptionResponse is one subscription in API responses.
type subscriptionResponse struct {
ID string `json:"id"`
SeriesID string `json:"series_id"`
Mode string `json:"mode"`
SeasonNumbers []int `json:"season_numbers,omitempty"`
TargetSeason *int `json:"target_season,omitempty"`
DeleteWatched bool `json:"delete_watched"`
MaxStorageBytes int64 `json:"max_storage_bytes"`
Active bool `json:"active"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
}
type subscriptionsListResponse struct {
Subscriptions []subscriptionResponse `json:"subscriptions"`
}
// subscriptionResultResponse wraps a created/updated subscription plus the count
// of in-scope episodes registered as managed downloads.
type subscriptionResultResponse struct {
Subscription subscriptionResponse `json:"subscription"`
Registered int `json:"registered"`
}
// subscriptionSyncResponse reports how many newly in-scope episodes a sync
// registered across the calling device's monitors.
type subscriptionSyncResponse struct {
Registered int `json:"registered"`
}
func toSubscriptionResponse(s *downloads.Subscription) subscriptionResponse {
return subscriptionResponse{
ID: s.ID,
SeriesID: s.SeriesID,
Mode: s.Mode,
SeasonNumbers: s.SeasonNumbers,
TargetSeason: s.TargetSeason,
DeleteWatched: s.DeleteWatched,
MaxStorageBytes: s.MaxStorageBytes,
Active: s.Active,
CreatedAt: s.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
UpdatedAt: s.UpdatedAt.Format("2006-01-02T15:04:05Z07:00"),
}
}
func toSubscriptionResultResponse(result *downloads.SubscriptionResult) subscriptionResultResponse {
return subscriptionResultResponse{
Subscription: toSubscriptionResponse(result.Subscription),
Registered: result.Registered,
}
}
// HandleCreateSubscription handles POST /downloads/subscriptions (managed-only).
func (h *DownloadHandler) HandleCreateSubscription(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, deviceName, devicePlatform, ok := h.requireManaged(w, r)
if !ok {
return
}
var req subscriptionRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
if req.SeriesID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "series_id is required")
return
}
result, err := h.svc.CreateSubscription(r.Context(), userID, downloads.SubscriptionRequest{
SeriesID: req.SeriesID,
Mode: req.Mode,
SeasonNumbers: req.SeasonNumbers,
DeleteWatched: req.DeleteWatched,
MaxStorageBytes: req.MaxStorageBytes,
ProfileID: profileID,
DeviceID: deviceID,
DeviceName: deviceName,
DevicePlatform: devicePlatform,
}, requestAccessFilter(r))
if err != nil {
h.writeSubscriptionError(w, err)
return
}
writeJSON(w, http.StatusAccepted, toSubscriptionResultResponse(result))
}
// HandleListSubscriptions handles GET /downloads/subscriptions (managed-only).
func (h *DownloadHandler) HandleListSubscriptions(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
subs, err := h.svc.ListSubscriptions(r.Context(), userID, profileID, deviceID)
if err != nil {
h.writeSubscriptionError(w, err)
return
}
responses := make([]subscriptionResponse, 0, len(subs))
for _, s := range subs {
responses = append(responses, toSubscriptionResponse(s))
}
writeJSON(w, http.StatusOK, subscriptionsListResponse{Subscriptions: responses})
}
// HandleGetSubscription handles GET /downloads/subscriptions/{id} (managed-only).
func (h *DownloadHandler) HandleGetSubscription(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
sub, err := h.svc.GetSubscription(r.Context(), userID, profileID, deviceID, chi.URLParam(r, "id"))
if err != nil {
h.writeSubscriptionError(w, err)
return
}
writeJSON(w, http.StatusOK, toSubscriptionResponse(sub))
}
// HandlePatchSubscription handles PATCH /downloads/subscriptions/{id} (managed-only).
func (h *DownloadHandler) HandlePatchSubscription(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
var req patchSubscriptionRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
result, err := h.svc.UpdateSubscription(r.Context(), userID, profileID, deviceID, chi.URLParam(r, "id"), downloads.SubscriptionPatch{
Mode: req.Mode,
SeasonNumbers: req.SeasonNumbers,
DeleteWatched: req.DeleteWatched,
MaxStorageBytes: req.MaxStorageBytes,
Active: req.Active,
}, requestAccessFilter(r))
if err != nil {
h.writeSubscriptionError(w, err)
return
}
writeJSON(w, http.StatusOK, toSubscriptionResultResponse(result))
}
// HandleDeleteSubscription handles DELETE /downloads/subscriptions/{id} (managed-only).
func (h *DownloadHandler) HandleDeleteSubscription(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
if err := h.svc.DeleteSubscription(r.Context(), userID, profileID, deviceID, chi.URLParam(r, "id")); err != nil {
h.writeSubscriptionError(w, err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// HandleSyncSubscriptions handles POST /downloads/subscriptions/sync (managed-only):
// the client triggers it on open / background refresh to register newly available
// episodes for its monitors, then pulls the files on its own schedule.
func (h *DownloadHandler) HandleSyncSubscriptions(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
registered, err := h.svc.SyncSubscriptions(r.Context(), userID, profileID, deviceID, requestAccessFilter(r))
if err != nil {
h.writeSubscriptionError(w, err)
return
}
writeJSON(w, http.StatusOK, subscriptionSyncResponse{Registered: registered})
}
// writeSubscriptionError maps series-monitoring errors to HTTP responses.
func (h *DownloadHandler) writeSubscriptionError(w http.ResponseWriter, err error) {
switch {
case errors.Is(err, downloads.ErrSubscriptionNotFound):
writeError(w, http.StatusNotFound, "not_found", "Subscription not found")
case errors.Is(err, downloads.ErrSubscriptionsUnavailable):
writeError(w, http.StatusServiceUnavailable, "unavailable", "Series monitoring is not available")
case errors.Is(err, downloads.ErrInvalidSubscriptionMode):
writeError(w, http.StatusBadRequest, "invalid_mode", "Unknown monitor mode")
case errors.Is(err, downloads.ErrSeasonsRequired):
writeError(w, http.StatusBadRequest, "seasons_required", "season_numbers is required for specific_seasons mode")
case errors.Is(err, downloads.ErrInvalidSeasonNumbers):
writeError(w, http.StatusBadRequest, "invalid_season_numbers", "season_numbers values must be between 0 and 9999")
case errors.Is(err, downloads.ErrNotSeries):
writeError(w, http.StatusBadRequest, "not_series", "content_id is not a series")
case errors.Is(err, downloads.ErrProfileRequired):
writeError(w, http.StatusBadRequest, "profile_required", "A profile and device are required for series monitoring")
case errors.Is(err, downloads.ErrFeatureDisabled), errors.Is(err, downloads.ErrDownloadNotAllowed):
writeError(w, http.StatusForbidden, "forbidden", "You are not allowed to download")
case errors.Is(err, catalog.ErrItemNotFound):
writeError(w, http.StatusNotFound, "not_found", "Series not found")
default:
slog.Error("download subscription operation failed", "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to process subscription")
}
}
+405 -75
View File
@@ -12,17 +12,33 @@ import (
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/download"
"github.com/Silo-Server/silo-server/internal/downloads"
"github.com/Silo-Server/silo-server/internal/playback"
)
// DownloadService is the interface that the download handler depends on.
// DownloadService is the interface that the download handler depends on. A
// non-empty deviceID (from the X-Silo-Device-Id header) selects the managed
// device-library lifecycle; empty is the ephemeral/account-level path.
type DownloadService interface {
CreateQueued(ctx context.Context, userID int, req download.CreateRequest, filter catalog.AccessFilter) (*download.Download, error)
CreateQueuedBatch(ctx context.Context, userID int, seriesContentID string, filter catalog.AccessFilter) ([]*download.Download, string, error)
ServeDirect(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, fileID int, filter catalog.AccessFilter) error
ServeFile(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, downloadID string) error
List(ctx context.Context, userID int) ([]*download.Download, error)
Cancel(ctx context.Context, userID int, downloadID string) error
Capability(ctx context.Context, userID int) (downloads.Capability, error)
Create(ctx context.Context, userID int, req downloads.CreateRequest, filter catalog.AccessFilter) (*downloads.Download, error)
CreateSeries(ctx context.Context, userID int, req downloads.CreateRequest, filter catalog.AccessFilter) ([]*downloads.Download, string, []downloads.SkippedDownload, error)
CreateSeason(ctx context.Context, userID int, req downloads.CreateRequest, seasonNumber int, filter catalog.AccessFilter) ([]*downloads.Download, string, []downloads.SkippedDownload, error)
CreateSubscription(ctx context.Context, userID int, req downloads.SubscriptionRequest, filter catalog.AccessFilter) (*downloads.SubscriptionResult, error)
ListSubscriptions(ctx context.Context, userID int, profileID, deviceID string) ([]*downloads.Subscription, error)
GetSubscription(ctx context.Context, userID int, profileID, deviceID, id string) (*downloads.Subscription, error)
UpdateSubscription(ctx context.Context, userID int, profileID, deviceID, id string, patch downloads.SubscriptionPatch, filter catalog.AccessFilter) (*downloads.SubscriptionResult, error)
DeleteSubscription(ctx context.Context, userID int, profileID, deviceID, id string) error
SyncSubscriptions(ctx context.Context, userID int, profileID, deviceID string, filter catalog.AccessFilter) (int, error)
ServeDirect(ctx context.Context, w http.ResponseWriter, r *http.Request, userID, fileID int, format string, filter catalog.AccessFilter) error
ServeFile(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, profileID, deviceID, downloadID string, filter catalog.AccessFilter) error
List(ctx context.Context, userID int, profileID, deviceID string) ([]*downloads.Download, error)
Delete(ctx context.Context, userID int, profileID, deviceID, downloadID string) error
PatchStatus(ctx context.Context, userID int, profileID, deviceID, downloadID, status string) error
BuildManifest(ctx context.Context, userID int, profileID, deviceID, downloadID string, filter catalog.AccessFilter) (*downloads.OfflineManifest, error)
BuildBatchManifests(ctx context.Context, userID int, profileID, deviceID, batchID string, filter catalog.AccessFilter) ([]*downloads.OfflineManifest, []downloads.SkippedManifest, error)
ServeArtwork(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, profileID, deviceID, downloadID, kind string, filter catalog.AccessFilter) error
ServeSubtitle(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, profileID, deviceID, downloadID, ref string, filter catalog.AccessFilter) error
}
// DownloadHandler handles download endpoints.
@@ -37,44 +53,96 @@ func NewDownloadHandler(svc DownloadService) *DownloadHandler {
// downloadRequest represents the JSON body for POST /downloads.
type downloadRequest struct {
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
FileID int `json:"file_id,omitempty"`
Series bool `json:"series,omitempty"` // if true, downloads all episodes
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
FileID int `json:"file_id,omitempty"`
Quality string `json:"quality,omitempty"` // original (default) | 20mbps | 10mbps | 5mbps | 2mbps | 1mbps
Series bool `json:"series,omitempty"` // if true, downloads all episodes
Season *int `json:"season_number,omitempty"` // with series=true, downloads only this season (0 = Specials)
Caps *downloadCaps `json:"caps,omitempty"` // device decode capability (original fallback / transcode target)
}
// downloadCaps mirrors playback.ClientCapabilities for the request body.
type downloadCaps struct {
CodecsVideo []string `json:"codecs_video,omitempty"`
CodecsAudio []string `json:"codecs_audio,omitempty"`
AudioPassthroughCodecs []string `json:"audio_passthrough_codecs,omitempty"`
Containers []string `json:"containers,omitempty"`
MaxResolution string `json:"max_resolution,omitempty"`
HDR bool `json:"hdr,omitempty"`
}
// patchDownloadRequest is the JSON body for PATCH /downloads/{id}.
type patchDownloadRequest struct {
Status string `json:"status"`
}
// downloadResponse represents a download entry in API responses.
type downloadResponse struct {
ID string `json:"id"`
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
BatchID string `json:"batch_id,omitempty"`
MediaFileID int `json:"media_file_id"`
FileSize int64 `json:"file_size"`
BytesSent int64 `json:"bytes_sent"`
Kind string `json:"kind"`
Status string `json:"status"`
CreatedAt string `json:"created_at"`
CompletedAt *string `json:"completed_at,omitempty"`
ID string `json:"id"`
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
BatchID string `json:"batch_id,omitempty"`
DeviceID string `json:"device_id,omitempty"`
MediaFileID int `json:"media_file_id"`
FileSize int64 `json:"file_size"`
BytesSent int64 `json:"bytes_sent"`
Kind string `json:"kind"`
Status string `json:"status"`
Quality string `json:"quality"`
EffectiveQuality string `json:"effective_quality"`
DeliveryFormat string `json:"delivery_format"`
TargetBitrateKbps int `json:"target_bitrate_kbps"`
Revision int `json:"revision"`
CreatedAt string `json:"created_at"`
CompletedAt *string `json:"completed_at,omitempty"`
}
// downloadsListResponse wraps the downloads list for JSON serialization.
type downloadsListResponse struct {
Downloads []downloadResponse `json:"downloads"`
Downloads []downloadResponse `json:"downloads"`
Skipped []downloads.SkippedDownload `json:"skipped,omitempty"`
}
func toDownloadResponse(d *download.Download) downloadResponse {
type batchManifestsResponse struct {
Manifests []*downloads.OfflineManifest `json:"manifests"`
// Skipped lists batch entries whose manifest could not be built (revoked,
// deleted from the catalog, access-filtered); the rest of the batch is
// still delivered.
Skipped []downloads.SkippedManifest `json:"skipped,omitempty"`
}
// downloadCapabilityResponse is the GET /downloads/capability payload clients
// use for feature detection instead of introspecting admin settings.
type downloadCapabilityResponse struct {
Enabled bool `json:"enabled"`
DownloadAllowed bool `json:"download_allowed"`
QualityPresets []string `json:"quality_presets"`
TranscodeEnabled bool `json:"transcode_enabled"`
TranscodeUserAllowed bool `json:"transcode_user_allowed"`
SeasonDownload bool `json:"season_download"`
SeriesMonitoring bool `json:"series_monitoring"`
MonitoringModes []string `json:"monitoring_modes,omitempty"`
}
func toDownloadResponse(d *downloads.Download) downloadResponse {
resp := downloadResponse{
ID: d.ID,
ContentID: d.ContentID,
EpisodeID: d.EpisodeID,
BatchID: d.BatchID,
MediaFileID: d.MediaFileID,
FileSize: d.FileSize,
BytesSent: d.BytesSent,
Kind: d.Kind,
Status: d.Status,
CreatedAt: d.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
ID: d.ID,
ContentID: d.ContentID,
EpisodeID: d.EpisodeID,
BatchID: d.BatchID,
DeviceID: d.DeviceID,
MediaFileID: d.MediaFileID,
FileSize: d.FileSize,
BytesSent: d.BytesSent,
Kind: d.Kind,
Status: d.Status,
Quality: d.Quality,
EffectiveQuality: d.EffectiveQuality,
DeliveryFormat: d.Format,
TargetBitrateKbps: d.TargetBitrateKbps,
Revision: d.Revision,
CreatedAt: d.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
}
if d.CompletedAt != nil {
s := d.CompletedAt.Format("2006-01-02T15:04:05Z07:00")
@@ -83,14 +151,53 @@ func toDownloadResponse(d *download.Download) downloadResponse {
return resp
}
// HandleCreateDownload handles POST /downloads.
// managedIdentity returns the (profileID, deviceID) the request is acting as.
// deviceID comes ONLY from the X-Silo-Device-Id header (never the body/query);
// profileID is resolved by the viewer-access middleware from X-Profile-Id.
func managedIdentity(r *http.Request) (profileID, deviceID, deviceName, devicePlatform string) {
device := deviceMetadataFromRequest(r)
return apimw.GetProfileID(r.Context()), device.DeviceID, device.DeviceName, device.DevicePlatform
}
// HandleCapability handles GET /downloads/capability.
func (h *DownloadHandler) HandleCapability(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
}
capInfo, err := h.svc.Capability(r.Context(), userID)
if err != nil {
slog.Error("failed to load download capability", "user_id", userID, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load download capability")
return
}
writeJSON(w, http.StatusOK, downloadCapabilityResponse{
Enabled: capInfo.Enabled,
DownloadAllowed: capInfo.DownloadAllowed,
QualityPresets: capInfo.QualityPresets,
TranscodeEnabled: capInfo.TranscodeEnabled,
TranscodeUserAllowed: capInfo.TranscodeUserAllowed,
SeasonDownload: capInfo.SeasonDownload,
SeriesMonitoring: capInfo.SeriesMonitoring,
MonitoringModes: capInfo.MonitoringModes,
})
}
// HandleCreateDownload handles POST /downloads. The X-Silo-Device-Id header
// (if present) makes this a managed device entry; otherwise it is ephemeral.
func (h *DownloadHandler) HandleCreateDownload(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
@@ -101,65 +208,102 @@ func (h *DownloadHandler) HandleCreateDownload(w http.ResponseWriter, r *http.Re
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
if req.ContentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return
}
profileID, deviceID, deviceName, devicePlatform := managedIdentity(r)
filter := requestAccessFilter(r)
createReq := downloads.CreateRequest{
ContentID: req.ContentID,
EpisodeID: req.EpisodeID,
FileID: req.FileID,
Quality: req.Quality,
ProfileID: profileID,
DeviceID: deviceID,
DeviceName: deviceName,
DevicePlatform: devicePlatform,
}
if req.Caps != nil {
createReq.Caps = playback.ClientCapabilities{
CodecsVideo: req.Caps.CodecsVideo,
CodecsAudio: req.Caps.CodecsAudio,
AudioPassthroughCodecs: req.Caps.AudioPassthroughCodecs,
Containers: req.Caps.Containers,
MaxResolution: req.Caps.MaxResolution,
HDR: req.Caps.HDR,
}
}
if req.Series {
downloads, batchID, err := h.svc.CreateQueuedBatch(r.Context(), userID, req.ContentID, filter)
var (
list []*downloads.Download
batchID string
skipped []downloads.SkippedDownload
err error
)
// Dispatch on presence, not value: season 0 is the Specials season.
if req.Season != nil {
if *req.Season < 0 {
writeError(w, http.StatusBadRequest, "bad_request", "season_number must be >= 0")
return
}
list, batchID, skipped, err = h.svc.CreateSeason(r.Context(), userID, createReq, *req.Season, filter)
} else {
list, batchID, skipped, err = h.svc.CreateSeries(r.Context(), userID, createReq, filter)
}
if err != nil {
h.writeDownloadError(w, err)
return
}
responses := make([]downloadResponse, 0, len(downloads))
for _, d := range downloads {
responses := make([]downloadResponse, 0, len(list))
for _, d := range list {
resp := toDownloadResponse(d)
resp.BatchID = batchID
if resp.BatchID == "" {
resp.BatchID = batchID
}
responses = append(responses, resp)
}
writeJSON(w, http.StatusAccepted, downloadsListResponse{Downloads: responses})
writeJSON(w, http.StatusAccepted, downloadsListResponse{Downloads: responses, Skipped: skipped})
return
}
dl, err := h.svc.CreateQueued(r.Context(), userID, download.CreateRequest{
ContentID: req.ContentID,
EpisodeID: req.EpisodeID,
FileID: req.FileID,
}, filter)
dl, err := h.svc.Create(r.Context(), userID, createReq, filter)
if err != nil {
h.writeDownloadError(w, err)
return
}
writeJSON(w, http.StatusAccepted, toDownloadResponse(dl))
}
// HandleListDownloads handles GET /downloads.
// HandleListDownloads handles GET /downloads. With a device header it returns
// the calling device's managed entries; otherwise the user's ephemeral rows.
func (h *DownloadHandler) HandleListDownloads(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
}
downloads, err := h.svc.List(r.Context(), userID)
profileID, deviceID, _, _ := managedIdentity(r)
list, err := h.svc.List(r.Context(), userID, profileID, deviceID)
if err != nil {
if errors.Is(err, downloads.ErrProfileRequired) {
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
return
}
slog.Error("failed to list downloads", "user_id", userID, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list downloads")
return
}
responses := make([]downloadResponse, 0, len(downloads))
for _, d := range downloads {
responses := make([]downloadResponse, 0, len(list))
for _, d := range list {
responses = append(responses, toDownloadResponse(d))
}
writeJSON(w, http.StatusOK, downloadsListResponse{Downloads: responses})
@@ -172,28 +316,66 @@ func (h *DownloadHandler) HandleDeleteDownload(w http.ResponseWriter, r *http.Re
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
id := chi.URLParam(r, "id")
if id == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Download ID is required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
}
profileID, deviceID, _, _ := managedIdentity(r)
if err := h.svc.Delete(r.Context(), userID, profileID, deviceID, id); err != nil {
if errors.Is(err, downloads.ErrNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Download not found")
return
}
if errors.Is(err, downloads.ErrProfileRequired) {
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
return
}
slog.Error("failed to delete download", "download_id", id, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to delete download")
return
}
w.WriteHeader(http.StatusNoContent)
}
// HandlePatchDownload handles PATCH /downloads/{id} — a managed-only endpoint
// where a client confirms local state (downloading|completed).
func (h *DownloadHandler) HandlePatchDownload(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
id := chi.URLParam(r, "id")
if id == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Download ID is required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
var req patchDownloadRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
if err := h.svc.Cancel(r.Context(), userID, id); err != nil {
if errors.Is(err, download.ErrNotFound) {
if err := h.svc.PatchStatus(r.Context(), userID, profileID, deviceID, id, req.Status); err != nil {
switch {
case errors.Is(err, downloads.ErrNotFound):
writeError(w, http.StatusNotFound, "not_found", "Download not found")
return
case errors.Is(err, downloads.ErrInvalidStatus):
writeError(w, http.StatusBadRequest, "invalid_status", "status must be downloading or completed")
case errors.Is(err, downloads.ErrProfileRequired):
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
default:
slog.Error("failed to patch download", "download_id", id, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to update download")
}
slog.Error("failed to cancel download", "download_id", id, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to cancel download")
return
}
w.WriteHeader(http.StatusNoContent)
}
@@ -204,41 +386,49 @@ func (h *DownloadHandler) HandleDownloadFile(w http.ResponseWriter, r *http.Requ
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
id := chi.URLParam(r, "id")
if id == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Download ID is required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
}
if err := h.svc.ServeFile(r.Context(), w, r, userID, id); err != nil {
if errors.Is(err, download.ErrNotFound) {
profileID, deviceID, _, _ := managedIdentity(r)
filter := requestAccessFilter(r)
if err := h.svc.ServeFile(r.Context(), w, r, userID, profileID, deviceID, id, filter); err != nil {
if errors.Is(err, downloads.ErrNotFound) || errors.Is(err, catalog.ErrItemNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Download not found")
return
}
if errors.Is(err, download.ErrDownloadNotActive) {
if errors.Is(err, downloads.ErrDownloadNotActive) {
writeError(w, http.StatusConflict, "download_inactive", "This download is no longer active")
return
}
if errors.Is(err, downloads.ErrProfileRequired) {
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
return
}
if errors.Is(err, downloads.ErrFeatureDisabled) || errors.Is(err, downloads.ErrDownloadNotAllowed) {
writeError(w, http.StatusForbidden, "forbidden", "You are not allowed to download")
return
}
slog.Error("failed to serve download file", "download_id", id, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to serve download")
return
}
}
// HandleDirectDownload handles GET /direct-download?file_id=N.
// HandleDirectDownload handles GET /direct-download?file_id=N. A legacy
// format=original query is accepted, but direct downloads remain original-only.
func (h *DownloadHandler) HandleDirectDownload(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
@@ -252,23 +442,163 @@ func (h *DownloadHandler) HandleDirectDownload(w http.ResponseWriter, r *http.Re
}
filter := requestAccessFilter(r)
if err := h.svc.ServeDirect(r.Context(), w, r, userID, fileID, filter); err != nil {
if err := h.svc.ServeDirect(r.Context(), w, r, userID, fileID, r.URL.Query().Get("format"), filter); err != nil {
h.writeDownloadError(w, err)
return
}
}
// requireManaged validates a managed (device-scoped) request: authentication, a
// configured service, and the device + profile identity (device_id from the
// X-Silo-Device-Id header only, never the body). On failure it writes the error
// response and returns ok=false. Shared by every managed-only endpoint — the
// offline assets and the series-monitoring subscriptions.
func (h *DownloadHandler) requireManaged(w http.ResponseWriter, r *http.Request) (userID int, profileID, deviceID, deviceName, devicePlatform string, ok bool) {
userID = apimw.GetUserID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h.svc == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Downloads not configured")
return
}
profileID, deviceID, deviceName, devicePlatform = managedIdentity(r)
if deviceID == "" {
writeError(w, http.StatusBadRequest, "device_id_required", "X-Silo-Device-Id header is required")
return
}
if profileID == "" {
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
return
}
return userID, profileID, deviceID, deviceName, devicePlatform, true
}
// managedAssetIdentity is requireManaged plus the URL {id} param, for the offline
// asset endpoints (manifest/artwork/subtitle).
func (h *DownloadHandler) managedAssetIdentity(w http.ResponseWriter, r *http.Request) (userID int, profileID, deviceID, id string, ok bool) {
userID, profileID, deviceID, _, _, ok = h.requireManaged(w, r)
if !ok {
return userID, profileID, deviceID, "", false
}
id = chi.URLParam(r, "id")
if id == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Download ID is required")
return userID, profileID, deviceID, "", false
}
return userID, profileID, deviceID, id, true
}
// HandleManifest handles GET /downloads/{id}/manifest (managed-only).
func (h *DownloadHandler) HandleManifest(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, id, ok := h.managedAssetIdentity(w, r)
if !ok {
return
}
manifest, err := h.svc.BuildManifest(r.Context(), userID, profileID, deviceID, id, requestAccessFilter(r))
if err != nil {
h.writeAssetError(w, "manifest", id, err)
return
}
writeJSON(w, http.StatusOK, manifest)
}
// HandleBatchManifests handles GET /downloads/batches/{batch_id}/manifests
// (managed-only).
func (h *DownloadHandler) HandleBatchManifests(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, _, _, ok := h.requireManaged(w, r)
if !ok {
return
}
batchID := chi.URLParam(r, "batch_id")
if batchID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Batch ID is required")
return
}
manifests, skipped, err := h.svc.BuildBatchManifests(r.Context(), userID, profileID, deviceID, batchID, requestAccessFilter(r))
if err != nil {
h.writeAssetError(w, "batch_manifests", batchID, err)
return
}
writeJSON(w, http.StatusOK, batchManifestsResponse{Manifests: manifests, Skipped: skipped})
}
// HandleArtwork handles GET /downloads/{id}/artwork/{kind} (managed-only).
func (h *DownloadHandler) HandleArtwork(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, id, ok := h.managedAssetIdentity(w, r)
if !ok {
return
}
kind := chi.URLParam(r, "kind")
if err := h.svc.ServeArtwork(r.Context(), w, r, userID, profileID, deviceID, id, kind, requestAccessFilter(r)); err != nil {
h.writeAssetError(w, "artwork", id, err)
return
}
}
// HandleSubtitle handles GET /downloads/{id}/subtitles/{ref} (managed-only).
func (h *DownloadHandler) HandleSubtitle(w http.ResponseWriter, r *http.Request) {
userID, profileID, deviceID, id, ok := h.managedAssetIdentity(w, r)
if !ok {
return
}
ref := chi.URLParam(r, "ref")
if err := h.svc.ServeSubtitle(r.Context(), w, r, userID, profileID, deviceID, id, ref, requestAccessFilter(r)); err != nil {
h.writeAssetError(w, "subtitle", id, err)
return
}
}
// writeAssetError maps offline asset (manifest/artwork/subtitle) errors. Access
// denials and missing assets collapse to 404 so a download id never reveals the
// existence of out-of-scope content.
func (h *DownloadHandler) writeAssetError(w http.ResponseWriter, asset, id string, err error) {
switch {
case errors.Is(err, downloads.ErrNotFound),
errors.Is(err, downloads.ErrAssetNotFound),
errors.Is(err, catalog.ErrItemNotFound):
writeError(w, http.StatusNotFound, "not_found", "Not found")
case errors.Is(err, downloads.ErrInvalidSubtitleRef):
writeError(w, http.StatusBadRequest, "invalid_subtitle_ref", "Invalid subtitle reference")
case errors.Is(err, downloads.ErrDownloadNotActive):
writeError(w, http.StatusConflict, "download_inactive", "This download is no longer active")
case errors.Is(err, downloads.ErrProfileRequired):
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
case errors.Is(err, downloads.ErrManifestUnavailable):
writeError(w, http.StatusServiceUnavailable, "unavailable", "Offline assets are not configured")
case errors.Is(err, downloads.ErrFeatureDisabled), errors.Is(err, downloads.ErrDownloadNotAllowed):
writeError(w, http.StatusForbidden, "forbidden", "You are not allowed to download")
default:
slog.Error("failed to serve download asset", "asset", asset, "download_id", id, "error", err)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to serve download asset")
}
}
func (h *DownloadHandler) writeDownloadError(w http.ResponseWriter, err error) {
switch {
case errors.Is(err, download.ErrFeatureDisabled):
case errors.Is(err, downloads.ErrFeatureDisabled):
writeError(w, http.StatusForbidden, "feature_disabled", "Downloads are disabled")
case errors.Is(err, download.ErrDownloadNotAllowed):
case errors.Is(err, downloads.ErrDownloadNotAllowed):
writeError(w, http.StatusForbidden, "forbidden", "You are not allowed to download")
case errors.Is(err, download.ErrConcurrentLimitReached):
case errors.Is(err, downloads.ErrTranscodeDisabled):
writeError(w, http.StatusForbidden, "transcode_disabled", "Download transcoding is disabled")
case errors.Is(err, downloads.ErrInvalidQuality):
writeError(w, http.StatusBadRequest, "invalid_quality", "Unknown download quality")
case errors.Is(err, downloads.ErrProfileRequired):
writeError(w, http.StatusBadRequest, "profile_required", "A profile is required for managed downloads")
case errors.Is(err, downloads.ErrBulkQualityUnavailable):
writeError(w, http.StatusNotImplemented, "bulk_quality_unavailable", "Bitrate quality is not available for bulk downloads yet")
case errors.Is(err, downloads.ErrQualityUnavailable):
writeError(w, http.StatusNotImplemented, "quality_unavailable", "This download quality is not available")
case errors.Is(err, downloads.ErrFormatUnavailable):
writeError(w, http.StatusNotImplemented, "format_unavailable", "This download format is not available yet")
case errors.Is(err, downloads.ErrConcurrentLimitReached):
writeError(w, http.StatusTooManyRequests, "download_limit_exceeded", "Maximum concurrent downloads reached")
case errors.Is(err, download.ErrPeriodLimitReached):
case errors.Is(err, downloads.ErrPeriodLimitReached):
writeError(w, http.StatusTooManyRequests, "download_quota_exceeded", "Download quota exceeded for this period")
case errors.Is(err, downloads.ErrNoDownloadableEpisodes):
writeError(w, http.StatusNotFound, "no_downloadable_episodes", "No downloadable episodes found")
case errors.Is(err, catalog.ErrItemNotFound):
writeError(w, http.StatusNotFound, "not_found", "Media item not found")
default:
+826
View File
@@ -0,0 +1,826 @@
package handlers
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/downloads"
)
// fakeDownloadService is a programmable DownloadService for handler tests. It
// records the identity (user/profile/device) it was called with so tests can
// assert the handler threads header-only device authority correctly.
type fakeDownloadService struct {
capability downloads.Capability
created *downloads.Download
createErr error
series []*downloads.Download
seriesID string
seriesErr error
list []*downloads.Download
listErr error
deleteErr error
patchErr error
serveErr error
directErr error
manifest *downloads.OfflineManifest
batchManifests []*downloads.OfflineManifest
batchSkipped []downloads.SkippedManifest
manifestErr error
artworkErr error
subtitleErr error
gotCreateReq downloads.CreateRequest
gotSeriesReq downloads.CreateRequest
gotList identityCall
gotServe identityCall
gotDelete identityCall
gotPatch identityCall
gotPatchStatus string
gotManifest identityCall
gotBatchManifest identityCall
gotArtwork identityCall
gotArtworkKind string
gotSubtitle identityCall
gotSubtitleRef string
gotDirectFormat string
gotDirectFileID int
// Season download + series-monitoring (subscription) fakes.
season []*downloads.Download
seasonID string
seasonErr error
gotSeasonReq downloads.CreateRequest
gotSeasonNum int
seasonCalled bool
subResult *downloads.SubscriptionResult
subList []*downloads.Subscription
sub *downloads.Subscription
subErr error
subDeleteErr error
gotSubReq downloads.SubscriptionRequest
gotSubPatch downloads.SubscriptionPatch
gotSubIdent identityCall
syncRegistered int
}
type identityCall struct {
userID int
profileID string
deviceID string
downloadID string
}
func (f *fakeDownloadService) Capability(context.Context, int) (downloads.Capability, error) {
return f.capability, nil
}
func (f *fakeDownloadService) Create(_ context.Context, _ int, req downloads.CreateRequest, _ catalog.AccessFilter) (*downloads.Download, error) {
f.gotCreateReq = req
if f.createErr != nil {
return nil, f.createErr
}
return f.created, nil
}
func (f *fakeDownloadService) CreateSeries(_ context.Context, _ int, req downloads.CreateRequest, _ catalog.AccessFilter) ([]*downloads.Download, string, []downloads.SkippedDownload, error) {
f.gotSeriesReq = req
if f.seriesErr != nil {
return nil, "", nil, f.seriesErr
}
return f.series, f.seriesID, nil, nil
}
func (f *fakeDownloadService) CreateSeason(_ context.Context, _ int, req downloads.CreateRequest, seasonNumber int, _ catalog.AccessFilter) ([]*downloads.Download, string, []downloads.SkippedDownload, error) {
f.gotSeasonReq = req
f.gotSeasonNum = seasonNumber
f.seasonCalled = true
if f.seasonErr != nil {
return nil, "", nil, f.seasonErr
}
return f.season, f.seasonID, nil, nil
}
func (f *fakeDownloadService) CreateSubscription(_ context.Context, _ int, req downloads.SubscriptionRequest, _ catalog.AccessFilter) (*downloads.SubscriptionResult, error) {
f.gotSubReq = req
if f.subErr != nil {
return nil, f.subErr
}
return f.subResult, nil
}
func (f *fakeDownloadService) ListSubscriptions(_ context.Context, userID int, profileID, deviceID string) ([]*downloads.Subscription, error) {
f.gotSubIdent = identityCall{userID: userID, profileID: profileID, deviceID: deviceID}
return f.subList, f.subErr
}
func (f *fakeDownloadService) GetSubscription(_ context.Context, userID int, profileID, deviceID, id string) (*downloads.Subscription, error) {
f.gotSubIdent = identityCall{userID, profileID, deviceID, id}
if f.subErr != nil {
return nil, f.subErr
}
return f.sub, nil
}
func (f *fakeDownloadService) UpdateSubscription(_ context.Context, userID int, profileID, deviceID, id string, patch downloads.SubscriptionPatch, _ catalog.AccessFilter) (*downloads.SubscriptionResult, error) {
f.gotSubIdent = identityCall{userID, profileID, deviceID, id}
f.gotSubPatch = patch
if f.subErr != nil {
return nil, f.subErr
}
return f.subResult, nil
}
func (f *fakeDownloadService) DeleteSubscription(_ context.Context, userID int, profileID, deviceID, id string) error {
f.gotSubIdent = identityCall{userID, profileID, deviceID, id}
return f.subDeleteErr
}
func (f *fakeDownloadService) SyncSubscriptions(_ context.Context, userID int, profileID, deviceID string, _ catalog.AccessFilter) (int, error) {
f.gotSubIdent = identityCall{userID: userID, profileID: profileID, deviceID: deviceID}
if f.subErr != nil {
return 0, f.subErr
}
return f.syncRegistered, nil
}
func (f *fakeDownloadService) ServeDirect(_ context.Context, w http.ResponseWriter, _ *http.Request, _, fileID int, format string, _ catalog.AccessFilter) error {
f.gotDirectFileID = fileID
f.gotDirectFormat = format
if f.directErr != nil {
return f.directErr
}
_, _ = w.Write([]byte("served"))
return nil
}
func (f *fakeDownloadService) ServeFile(_ context.Context, w http.ResponseWriter, _ *http.Request, userID int, profileID, deviceID, downloadID string, _ catalog.AccessFilter) error {
f.gotServe = identityCall{userID, profileID, deviceID, downloadID}
if f.serveErr != nil {
return f.serveErr
}
_, _ = w.Write([]byte("served"))
return nil
}
func (f *fakeDownloadService) List(_ context.Context, userID int, profileID, deviceID string) ([]*downloads.Download, error) {
f.gotList = identityCall{userID: userID, profileID: profileID, deviceID: deviceID}
return f.list, f.listErr
}
func (f *fakeDownloadService) Delete(_ context.Context, userID int, profileID, deviceID, downloadID string) error {
f.gotDelete = identityCall{userID, profileID, deviceID, downloadID}
return f.deleteErr
}
func (f *fakeDownloadService) PatchStatus(_ context.Context, userID int, profileID, deviceID, downloadID, status string) error {
f.gotPatch = identityCall{userID, profileID, deviceID, downloadID}
f.gotPatchStatus = status
return f.patchErr
}
func (f *fakeDownloadService) BuildManifest(_ context.Context, userID int, profileID, deviceID, downloadID string, _ catalog.AccessFilter) (*downloads.OfflineManifest, error) {
f.gotManifest = identityCall{userID, profileID, deviceID, downloadID}
if f.manifestErr != nil {
return nil, f.manifestErr
}
return f.manifest, nil
}
func (f *fakeDownloadService) BuildBatchManifests(_ context.Context, userID int, profileID, deviceID, batchID string, _ catalog.AccessFilter) ([]*downloads.OfflineManifest, []downloads.SkippedManifest, error) {
f.gotBatchManifest = identityCall{userID, profileID, deviceID, batchID}
if f.manifestErr != nil {
return nil, nil, f.manifestErr
}
return f.batchManifests, f.batchSkipped, nil
}
func (f *fakeDownloadService) ServeArtwork(_ context.Context, w http.ResponseWriter, _ *http.Request, userID int, profileID, deviceID, downloadID, kind string, _ catalog.AccessFilter) error {
f.gotArtwork = identityCall{userID, profileID, deviceID, downloadID}
f.gotArtworkKind = kind
if f.artworkErr != nil {
return f.artworkErr
}
_, _ = w.Write([]byte("img"))
return nil
}
func (f *fakeDownloadService) ServeSubtitle(_ context.Context, w http.ResponseWriter, _ *http.Request, userID int, profileID, deviceID, downloadID, ref string, _ catalog.AccessFilter) error {
f.gotSubtitle = identityCall{userID, profileID, deviceID, downloadID}
f.gotSubtitleRef = ref
if f.subtitleErr != nil {
return f.subtitleErr
}
_, _ = w.Write([]byte("WEBVTT"))
return nil
}
// downloadTestRequest builds a request with auth claims and optional profile +
// device identity (as the viewer-access middleware / client headers would set).
func downloadTestRequest(method, target string, body []byte, userID int, profileID, deviceID string) *http.Request {
var r *http.Request
if body != nil {
r = httptest.NewRequest(method, target, bytes.NewReader(body))
} else {
r = httptest.NewRequest(method, target, nil)
}
if userID != 0 {
ctx := apimw.SetClaims(r.Context(), &auth.Claims{
UserID: userID,
Role: "user",
TokenType: auth.TokenTypeAccess,
})
if profileID != "" {
ctx = apimw.SetProfileID(ctx, profileID)
}
r = r.WithContext(ctx)
}
if deviceID != "" {
r.Header.Set("X-Silo-Device-Id", deviceID)
}
return r
}
func withChiID(r *http.Request, id string) *http.Request {
rctx := chi.NewRouteContext()
rctx.URLParams.Add("id", id)
return r.WithContext(context.WithValue(r.Context(), chi.RouteCtxKey, rctx))
}
func TestHandleCapability(t *testing.T) {
svc := &fakeDownloadService{capability: downloads.Capability{
Enabled: true,
DownloadAllowed: true,
QualityPresets: []string{downloads.QualityOriginal},
TranscodeEnabled: false,
TranscodeUserAllowed: true,
}}
h := NewDownloadHandler(svc)
rec := httptest.NewRecorder()
h.HandleCapability(rec, downloadTestRequest(http.MethodGet, "/downloads/capability", nil, 7, "", ""))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
var resp downloadCapabilityResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if !resp.Enabled || !resp.DownloadAllowed || resp.TranscodeEnabled || !resp.TranscodeUserAllowed {
t.Fatalf("unexpected capability flags: %+v", resp)
}
if len(resp.QualityPresets) != 1 || resp.QualityPresets[0] != downloads.QualityOriginal {
t.Fatalf("quality presets = %v, want [original]", resp.QualityPresets)
}
}
func TestHandleCapabilityUnauthorized(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
rec := httptest.NewRecorder()
h.HandleCapability(rec, httptest.NewRequest(http.MethodGet, "/downloads/capability", nil))
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rec.Code)
}
}
func TestHandleCapabilityNilService(t *testing.T) {
h := NewDownloadHandler(nil)
rec := httptest.NewRecorder()
h.HandleCapability(rec, downloadTestRequest(http.MethodGet, "/downloads/capability", nil, 7, "", ""))
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status = %d, want 503", rec.Code)
}
}
func TestHandleCreateDownloadThreadsQuality(t *testing.T) {
svc := &fakeDownloadService{created: &downloads.Download{
ID: "dl1", ContentID: "c1", Status: downloads.StatusQueued, Format: downloads.FormatTranscode,
Quality: downloads.Quality5Mbps, EffectiveQuality: downloads.Quality5Mbps, TargetBitrateKbps: 5000, Revision: 2,
}}
h := NewDownloadHandler(svc)
body, _ := json.Marshal(downloadRequest{ContentID: "c1", Quality: downloads.Quality5Mbps})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "", ""))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotCreateReq.Quality != downloads.Quality5Mbps {
t.Fatalf("service received quality %q, want 5mbps", svc.gotCreateReq.Quality)
}
var resp downloadResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if resp.Quality != downloads.Quality5Mbps || resp.DeliveryFormat != downloads.FormatTranscode ||
resp.TargetBitrateKbps != 5000 || resp.Revision != 2 {
t.Fatalf("response = %+v, want quality/delivery/bitrate/revision", resp)
}
}
func TestHandleCreateDownloadSeriesThreadsQuality(t *testing.T) {
svc := &fakeDownloadService{
series: []*downloads.Download{{ID: "dl1", ContentID: "s1", Format: downloads.FormatOriginal, Quality: downloads.QualityOriginal, EffectiveQuality: downloads.QualityOriginal, Revision: 1}},
seriesID: "batch1",
}
h := NewDownloadHandler(svc)
body, _ := json.Marshal(downloadRequest{ContentID: "s1", Series: true, Quality: downloads.QualityOriginal})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "", ""))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotSeriesReq.Quality != downloads.QualityOriginal {
t.Fatalf("series service received quality %q, want original", svc.gotSeriesReq.Quality)
}
}
func TestHandleCreateDownloadMissingContentID(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
body, _ := json.Marshal(downloadRequest{Quality: downloads.QualityOriginal})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "", ""))
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", rec.Code)
}
}
func TestHandleCreateDownloadErrorMapping(t *testing.T) {
cases := []struct {
name string
err error
wantCode int
}{
{"feature disabled", downloads.ErrFeatureDisabled, http.StatusForbidden},
{"not allowed", downloads.ErrDownloadNotAllowed, http.StatusForbidden},
{"transcode disabled", downloads.ErrTranscodeDisabled, http.StatusForbidden},
{"invalid quality", downloads.ErrInvalidQuality, http.StatusBadRequest},
{"quality unavailable", downloads.ErrQualityUnavailable, http.StatusNotImplemented},
{"bulk quality unavailable", downloads.ErrBulkQualityUnavailable, http.StatusNotImplemented},
{"format unavailable", downloads.ErrFormatUnavailable, http.StatusNotImplemented},
{"profile required", downloads.ErrProfileRequired, http.StatusBadRequest},
{"concurrent limit", downloads.ErrConcurrentLimitReached, http.StatusTooManyRequests},
{"period limit", downloads.ErrPeriodLimitReached, http.StatusTooManyRequests},
{"item not found", catalog.ErrItemNotFound, http.StatusNotFound},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{createErr: tc.err})
body, _ := json.Marshal(downloadRequest{ContentID: "c1"})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "", ""))
if rec.Code != tc.wantCode {
t.Fatalf("status = %d, want %d (body: %s)", rec.Code, tc.wantCode, rec.Body.String())
}
})
}
}
// TestManagedCreateUsesHeaderDeviceNotBody verifies invariant 2's "device_id
// authority is the header only": a device_id placed in the JSON body is ignored,
// and the service receives the X-Silo-Device-Id header value + the profile.
func TestManagedCreateUsesHeaderDeviceNotBody(t *testing.T) {
svc := &fakeDownloadService{created: &downloads.Download{ID: "dl1", ContentID: "c1", DeviceID: "devA"}}
h := NewDownloadHandler(svc)
// Body smuggles a device_id; it is not a field of downloadRequest, so it is
// structurally dropped and must never reach the service.
body := []byte(`{"content_id":"c1","device_id":"EVIL","profile_id":"EVIL"}`)
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "pA", "devA"))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotCreateReq.DeviceID != "devA" {
t.Fatalf("service device id = %q, want header value devA", svc.gotCreateReq.DeviceID)
}
if svc.gotCreateReq.ProfileID != "pA" {
t.Fatalf("service profile id = %q, want context value pA", svc.gotCreateReq.ProfileID)
}
}
func TestManagedFileThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
req := withChiID(downloadTestRequest(http.MethodGet, "/downloads/dl1/file", nil, 7, "pA", "devA"), "dl1")
rec := httptest.NewRecorder()
h.HandleDownloadFile(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotServe != (identityCall{7, "pA", "devA", "dl1"}) {
t.Fatalf("serve identity = %+v, want {7 pA devA dl1}", svc.gotServe)
}
}
func TestManagedListThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
rec := httptest.NewRecorder()
h.HandleListDownloads(rec, downloadTestRequest(http.MethodGet, "/downloads", nil, 7, "pA", "devA"))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
if svc.gotList.deviceID != "devA" || svc.gotList.profileID != "pA" {
t.Fatalf("list identity = %+v, want profile pA device devA", svc.gotList)
}
}
func TestManagedDeleteThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
req := withChiID(downloadTestRequest(http.MethodDelete, "/downloads/dl1", nil, 7, "pA", "devA"), "dl1")
rec := httptest.NewRecorder()
h.HandleDeleteDownload(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want 204 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotDelete != (identityCall{7, "pA", "devA", "dl1"}) {
t.Fatalf("delete identity = %+v, want {7 pA devA dl1}", svc.gotDelete)
}
}
func TestHandleDeleteDownloadNotFound(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{deleteErr: downloads.ErrNotFound})
rec := httptest.NewRecorder()
req := withChiID(downloadTestRequest(http.MethodDelete, "/downloads/dl1", nil, 7, "", ""), "dl1")
h.HandleDeleteDownload(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404", rec.Code)
}
}
func TestManagedPatchRequiresDeviceHeader(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
// No device header → 400 device_id_required, even with a profile present.
req := withChiID(downloadTestRequest(http.MethodPatch, "/downloads/dl1", []byte(`{"status":"completed"}`), 7, "pA", ""), "dl1")
rec := httptest.NewRecorder()
h.HandlePatchDownload(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 device_id_required (body: %s)", rec.Code, rec.Body.String())
}
}
func TestManagedPatchThreadsIdentityAndStatus(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
req := withChiID(downloadTestRequest(http.MethodPatch, "/downloads/dl1", []byte(`{"status":"completed"}`), 7, "pA", "devA"), "dl1")
rec := httptest.NewRecorder()
h.HandlePatchDownload(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want 204 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotPatch != (identityCall{7, "pA", "devA", "dl1"}) {
t.Fatalf("patch identity = %+v, want {7 pA devA dl1}", svc.gotPatch)
}
if svc.gotPatchStatus != downloads.StatusCompleted {
t.Fatalf("patch status = %q, want completed", svc.gotPatchStatus)
}
}
func TestHandleDirectDownloadThreadsOriginalFormat(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
rec := httptest.NewRecorder()
h.HandleDirectDownload(rec, downloadTestRequest(http.MethodGet, "/direct-download?file_id=42&format=original", nil, 7, "", ""))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotDirectFileID != 42 {
t.Fatalf("file id = %d, want 42", svc.gotDirectFileID)
}
if svc.gotDirectFormat != downloads.FormatOriginal {
t.Fatalf("direct format = %q, want original", svc.gotDirectFormat)
}
}
func TestHandleDirectDownloadMissingFileID(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
rec := httptest.NewRecorder()
h.HandleDirectDownload(rec, downloadTestRequest(http.MethodGet, "/direct-download", nil, 7, "", ""))
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", rec.Code)
}
}
func withChiParams(r *http.Request, params map[string]string) *http.Request {
rctx := chi.NewRouteContext()
for k, v := range params {
rctx.URLParams.Add(k, v)
}
return r.WithContext(context.WithValue(r.Context(), chi.RouteCtxKey, rctx))
}
func TestManagedManifestRequiresDeviceHeader(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
// Profile present but no device header → 400 device_id_required.
req := withChiID(downloadTestRequest(http.MethodGet, "/downloads/dl1/manifest", nil, 7, "pA", ""), "dl1")
rec := httptest.NewRecorder()
h.HandleManifest(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 (body: %s)", rec.Code, rec.Body.String())
}
}
func TestManagedManifestThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{manifest: &downloads.OfflineManifest{DownloadID: "dl1", Title: "Movie"}}
h := NewDownloadHandler(svc)
req := withChiID(downloadTestRequest(http.MethodGet, "/downloads/dl1/manifest", nil, 7, "pA", "devA"), "dl1")
rec := httptest.NewRecorder()
h.HandleManifest(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotManifest != (identityCall{7, "pA", "devA", "dl1"}) {
t.Fatalf("manifest identity = %+v, want {7 pA devA dl1}", svc.gotManifest)
}
}
func TestManagedBatchManifestsThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{batchManifests: []*downloads.OfflineManifest{
{DownloadID: "dl1", Title: "Episode 1"},
}}
h := NewDownloadHandler(svc)
req := withChiParams(downloadTestRequest(http.MethodGet, "/downloads/batches/b1/manifests", nil, 7, "pA", "devA"),
map[string]string{"batch_id": "b1"})
rec := httptest.NewRecorder()
h.HandleBatchManifests(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotBatchManifest != (identityCall{7, "pA", "devA", "b1"}) {
t.Fatalf("batch manifest identity = %+v, want {7 pA devA b1}", svc.gotBatchManifest)
}
var resp batchManifestsResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if len(resp.Manifests) != 1 || resp.Manifests[0].DownloadID != "dl1" {
t.Fatalf("manifests = %+v, want one dl1 manifest", resp.Manifests)
}
}
// TestManagedAssetsDenyRestrictedProfile is the Phase 2 acceptance test: a
// profile denied content access (the service returns ErrItemNotFound) gets a
// 404 from manifest, artwork, and subtitle — a download id never reveals
// out-of-scope content.
func TestManagedAssetsDenyRestrictedProfile(t *testing.T) {
t.Run("manifest", func(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{manifestErr: catalog.ErrItemNotFound})
req := withChiID(downloadTestRequest(http.MethodGet, "/downloads/dl1/manifest", nil, 7, "child", "devC"), "dl1")
rec := httptest.NewRecorder()
h.HandleManifest(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("manifest status = %d, want 404", rec.Code)
}
})
t.Run("artwork", func(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{artworkErr: catalog.ErrItemNotFound})
req := withChiParams(downloadTestRequest(http.MethodGet, "/downloads/dl1/artwork/poster", nil, 7, "child", "devC"),
map[string]string{"id": "dl1", "kind": "poster"})
rec := httptest.NewRecorder()
h.HandleArtwork(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("artwork status = %d, want 404", rec.Code)
}
})
t.Run("subtitle", func(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{subtitleErr: catalog.ErrItemNotFound})
req := withChiParams(downloadTestRequest(http.MethodGet, "/downloads/dl1/subtitles/external:0", nil, 7, "child", "devC"),
map[string]string{"id": "dl1", "ref": "external:0"})
rec := httptest.NewRecorder()
h.HandleSubtitle(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("subtitle status = %d, want 404", rec.Code)
}
})
}
func TestManagedArtworkThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{}
h := NewDownloadHandler(svc)
req := withChiParams(downloadTestRequest(http.MethodGet, "/downloads/dl1/artwork/backdrop", nil, 7, "pA", "devA"),
map[string]string{"id": "dl1", "kind": "backdrop"})
rec := httptest.NewRecorder()
h.HandleArtwork(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotArtwork != (identityCall{7, "pA", "devA", "dl1"}) || svc.gotArtworkKind != "backdrop" {
t.Fatalf("artwork identity = %+v kind = %q", svc.gotArtwork, svc.gotArtworkKind)
}
}
func TestManagedSubtitleInvalidRef(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{subtitleErr: downloads.ErrInvalidSubtitleRef})
req := withChiParams(downloadTestRequest(http.MethodGet, "/downloads/dl1/subtitles/bogus", nil, 7, "pA", "devA"),
map[string]string{"id": "dl1", "ref": "bogus"})
rec := httptest.NewRecorder()
h.HandleSubtitle(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 (body: %s)", rec.Code, rec.Body.String())
}
}
func intPtr(i int) *int { return &i }
// TestHandleCreateDownloadSeasonRoutes verifies that series=true + a season
// number routes to CreateSeason (not CreateSeries) with the season threaded.
func TestHandleCreateDownloadSeasonRoutes(t *testing.T) {
svc := &fakeDownloadService{
season: []*downloads.Download{{ID: "dl1", ContentID: "s1", EpisodeID: "e1", Format: downloads.FormatOriginal}},
seasonID: "batch1",
}
h := NewDownloadHandler(svc)
body, _ := json.Marshal(downloadRequest{ContentID: "s1", Series: true, Season: intPtr(2)})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "pA", "devA"))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotSeasonNum != 2 {
t.Fatalf("season number = %d, want 2", svc.gotSeasonNum)
}
if svc.gotSeasonReq.ContentID != "s1" {
t.Fatalf("season content id = %q, want s1", svc.gotSeasonReq.ContentID)
}
}
// TestHandleCreateDownloadSpecialsSeason pins the Specials boundary: season 0
// must route to CreateSeason(0), not silently broaden to a full-series
// download, and negative seasons are rejected.
func TestHandleCreateDownloadSpecialsSeason(t *testing.T) {
svc := &fakeDownloadService{
season: []*downloads.Download{{ID: "dl1", ContentID: "s1", EpisodeID: "e1", Format: downloads.FormatOriginal}},
seasonID: "batch1",
}
h := NewDownloadHandler(svc)
body, _ := json.Marshal(downloadRequest{ContentID: "s1", Series: true, Season: intPtr(0)})
rec := httptest.NewRecorder()
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "pA", "devA"))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if !svc.seasonCalled || svc.gotSeasonNum != 0 {
t.Fatalf("seasonCalled=%v num=%d, want CreateSeason(0)", svc.seasonCalled, svc.gotSeasonNum)
}
if svc.gotSeriesReq.ContentID != "" {
t.Fatal("season 0 must not fall through to CreateSeries")
}
rec = httptest.NewRecorder()
body, _ = json.Marshal(downloadRequest{ContentID: "s1", Series: true, Season: intPtr(-1)})
h.HandleCreateDownload(rec, downloadTestRequest(http.MethodPost, "/downloads", body, 7, "pA", "devA"))
if rec.Code != http.StatusBadRequest {
t.Fatalf("negative season status = %d, want 400", rec.Code)
}
}
func TestHandleCreateSubscriptionThreadsRequest(t *testing.T) {
svc := &fakeDownloadService{subResult: &downloads.SubscriptionResult{
Subscription: &downloads.Subscription{ID: "sub1", SeriesID: "s1", Mode: downloads.SubModeLatestSeason, Active: true},
Registered: 3,
}}
h := NewDownloadHandler(svc)
body, _ := json.Marshal(subscriptionRequest{SeriesID: "s1", Mode: downloads.SubModeLatestSeason, DeleteWatched: true, MaxStorageBytes: 1024})
rec := httptest.NewRecorder()
h.HandleCreateSubscription(rec, downloadTestRequest(http.MethodPost, "/downloads/subscriptions", body, 7, "pA", "devA"))
if rec.Code != http.StatusAccepted {
t.Fatalf("status = %d, want 202 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotSubReq.SeriesID != "s1" || svc.gotSubReq.Mode != downloads.SubModeLatestSeason {
t.Fatalf("subscription request = %+v", svc.gotSubReq)
}
if svc.gotSubReq.DeviceID != "devA" || svc.gotSubReq.ProfileID != "pA" {
t.Fatalf("subscription identity = profile %q device %q, want pA/devA", svc.gotSubReq.ProfileID, svc.gotSubReq.DeviceID)
}
if !svc.gotSubReq.DeleteWatched || svc.gotSubReq.MaxStorageBytes != 1024 {
t.Fatalf("subscription options not threaded: %+v", svc.gotSubReq)
}
var resp subscriptionResultResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if resp.Registered != 3 || resp.Subscription.ID != "sub1" {
t.Fatalf("unexpected response: %+v", resp)
}
}
// TestHandleCreateSubscriptionRequiresDevice enforces that monitoring is
// device-scoped: a missing X-Silo-Device-Id header is a 400 even with a profile.
func TestHandleCreateSubscriptionRequiresDevice(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
body, _ := json.Marshal(subscriptionRequest{SeriesID: "s1", Mode: downloads.SubModeAll})
rec := httptest.NewRecorder()
h.HandleCreateSubscription(rec, downloadTestRequest(http.MethodPost, "/downloads/subscriptions", body, 7, "pA", ""))
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 (body: %s)", rec.Code, rec.Body.String())
}
}
func TestHandleListSubscriptionsThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{subList: []*downloads.Subscription{{ID: "sub1", SeriesID: "s1", Mode: downloads.SubModeAll, Active: true}}}
h := NewDownloadHandler(svc)
rec := httptest.NewRecorder()
h.HandleListSubscriptions(rec, downloadTestRequest(http.MethodGet, "/downloads/subscriptions", nil, 7, "pA", "devA"))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotSubIdent.profileID != "pA" || svc.gotSubIdent.deviceID != "devA" {
t.Fatalf("list identity = %+v, want profile pA device devA", svc.gotSubIdent)
}
var resp subscriptionsListResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if len(resp.Subscriptions) != 1 || resp.Subscriptions[0].ID != "sub1" {
t.Fatalf("unexpected subscriptions: %+v", resp.Subscriptions)
}
}
func TestHandleSubscriptionErrorMapping(t *testing.T) {
cases := []struct {
name string
err error
wantCode int
}{
{"not found", downloads.ErrSubscriptionNotFound, http.StatusNotFound},
{"unavailable", downloads.ErrSubscriptionsUnavailable, http.StatusServiceUnavailable},
{"invalid mode", downloads.ErrInvalidSubscriptionMode, http.StatusBadRequest},
{"seasons required", downloads.ErrSeasonsRequired, http.StatusBadRequest},
{"not series", downloads.ErrNotSeries, http.StatusBadRequest},
{"not allowed", downloads.ErrDownloadNotAllowed, http.StatusForbidden},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{subErr: tc.err})
body, _ := json.Marshal(subscriptionRequest{SeriesID: "s1", Mode: downloads.SubModeAll})
rec := httptest.NewRecorder()
h.HandleCreateSubscription(rec, downloadTestRequest(http.MethodPost, "/downloads/subscriptions", body, 7, "pA", "devA"))
if rec.Code != tc.wantCode {
t.Fatalf("status = %d, want %d (body: %s)", rec.Code, tc.wantCode, rec.Body.String())
}
})
}
}
func TestHandleSyncSubscriptionsThreadsIdentity(t *testing.T) {
svc := &fakeDownloadService{syncRegistered: 5}
h := NewDownloadHandler(svc)
rec := httptest.NewRecorder()
h.HandleSyncSubscriptions(rec, downloadTestRequest(http.MethodPost, "/downloads/subscriptions/sync", nil, 7, "pA", "devA"))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
if svc.gotSubIdent.profileID != "pA" || svc.gotSubIdent.deviceID != "devA" {
t.Fatalf("sync identity = %+v, want profile pA device devA", svc.gotSubIdent)
}
var resp subscriptionSyncResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if resp.Registered != 5 {
t.Fatalf("registered = %d, want 5", resp.Registered)
}
}
func TestHandleSyncSubscriptionsRequiresDevice(t *testing.T) {
h := NewDownloadHandler(&fakeDownloadService{})
rec := httptest.NewRecorder()
// Profile present, no device header → 400 device_id_required.
h.HandleSyncSubscriptions(rec, downloadTestRequest(http.MethodPost, "/downloads/subscriptions/sync", nil, 7, "pA", ""))
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 (body: %s)", rec.Code, rec.Body.String())
}
}
+16 -3
View File
@@ -56,14 +56,27 @@ func (h *FilesystemHandler) HandleBrowse(w http.ResponseWriter, r *http.Request)
result := make([]filesystemBrowseEntry, 0, len(entries))
for _, entry := range entries {
name := entry.Name()
full := filepath.Join(cleaned, name)
// os.ReadDir reports lstat semantics, so a symlink to a directory has
// IsDir() == false. Follow symlink entries with os.Stat and keep them only
// when the target is a directory; skip links whose target cannot be stat'd
// (broken or looping). Non-symlink, non-directory entries (regular files)
// are skipped without the extra stat so large media folders stay fast.
if !entry.IsDir() {
continue
if entry.Type()&os.ModeSymlink == 0 {
continue
}
target, err := os.Stat(full)
if err != nil || !target.IsDir() {
continue
}
}
name := entry.Name()
result = append(result, filesystemBrowseEntry{
Name: name,
Path: filepath.Join(cleaned, name),
Path: full,
})
}
+97
View File
@@ -0,0 +1,97 @@
package handlers
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"testing"
)
// TestHandleBrowseFollowsDirectorySymlinks verifies that the folder browser
// lists symlinks that resolve to directories (issue #208). os.ReadDir returns
// lstat-based entries, so a directory symlink reports IsDir() == false and was
// silently dropped; the handler must os.Stat each entry to follow the link.
func TestHandleBrowseFollowsDirectorySymlinks(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlink creation is unreliable on Windows CI")
}
t.Parallel()
root := t.TempDir()
// A real directory that should always be listed.
realDir := filepath.Join(root, "real-dir")
if err := os.Mkdir(realDir, 0o755); err != nil {
t.Fatalf("mkdir real-dir: %v", err)
}
// The symlink-to-directory target lives outside root so the browser cannot
// reach it except by following the link — mirrors the reported setup where
// entries under a bind mount point into a separate FUSE mount.
linkTarget := filepath.Join(t.TempDir(), "link-target")
if err := os.Mkdir(linkTarget, 0o755); err != nil {
t.Fatalf("mkdir link-target: %v", err)
}
dirSymlink := filepath.Join(root, "symlinked-dir")
if err := os.Symlink(linkTarget, dirSymlink); err != nil {
t.Fatalf("symlink dir: %v", err)
}
// A regular file: must never be listed by the folder browser.
regularFile := filepath.Join(root, "note.txt")
if err := os.WriteFile(regularFile, []byte("x"), 0o644); err != nil {
t.Fatalf("write file: %v", err)
}
// A symlink to a file: resolves to a non-directory, must not be listed.
fileSymlink := filepath.Join(root, "note-link")
if err := os.Symlink(regularFile, fileSymlink); err != nil {
t.Fatalf("symlink file: %v", err)
}
// A broken symlink: os.Stat fails; it must be skipped, not error the request.
brokenSymlink := filepath.Join(root, "broken-link")
if err := os.Symlink(filepath.Join(root, "does-not-exist"), brokenSymlink); err != nil {
t.Fatalf("symlink broken: %v", err)
}
handler := NewFilesystemHandler()
req := httptest.NewRequest(http.MethodGet, "/filesystem/browse?path="+root, nil)
rec := httptest.NewRecorder()
handler.HandleBrowse(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp filesystemBrowseResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
got := make(map[string]string, len(resp.Entries))
for _, e := range resp.Entries {
got[e.Name] = e.Path
}
if _, ok := got["real-dir"]; !ok {
t.Errorf("real-dir missing from entries: %+v", resp.Entries)
}
if path, ok := got["symlinked-dir"]; !ok {
t.Errorf("symlinked-dir (directory symlink) missing from entries: %+v", resp.Entries)
} else if want := filepath.Join(root, "symlinked-dir"); path != want {
t.Errorf("symlinked-dir path = %q, want %q", path, want)
}
if _, ok := got["note.txt"]; ok {
t.Errorf("regular file note.txt should not be listed: %+v", resp.Entries)
}
if _, ok := got["note-link"]; ok {
t.Errorf("file symlink note-link should not be listed: %+v", resp.Entries)
}
if _, ok := got["broken-link"]; ok {
t.Errorf("broken symlink broken-link should not be listed: %+v", resp.Entries)
}
}
+28 -3
View File
@@ -687,6 +687,26 @@ func (h *LibraryHandler) HandleUpdateLibrary(w http.ResponseWriter, r *http.Requ
}
}
// Re-fetch metadata when the library's metadata language changed, so
// existing items adopt the new language instead of keeping the one
// stamped at first match. Quick mode suffices: the refresh item lister
// includes complete-but-language-mismatched items.
if h.JobRepo != nil && !strings.EqualFold(strings.TrimSpace(oldFolder.MetadataLanguage), strings.TrimSpace(folder.MetadataLanguage)) {
job, jobErr := h.JobRepo.CreateLibraryRefresh(r.Context(), currentAdminUserID(r), adminjob.LibraryRefreshRequest{
LibraryID: folder.ID,
LibraryName: folder.Name,
Mode: adminjob.LibraryRefreshModeQuick,
}, "Queued metadata refresh after library language change")
if jobErr != nil {
var conflict *adminjob.ActiveJobConflictError
if !errors.As(jobErr, &conflict) {
slog.Warn("queue language-change metadata refresh failed", "library_id", folder.ID, "error", jobErr)
}
} else {
publishEventJob(r.Context(), h.EventsHub, "job.created", job)
}
}
// Rescan when paths have changed (folders added or removed).
if req.Paths != nil && !slices.Equal(oldFolder.Paths, *req.Paths) {
if h.ScanQueue != nil {
@@ -2495,10 +2515,15 @@ func (h *LibraryHandler) HandleListUnmatchedItems(w http.ResponseWriter, r *http
)`
}
// Manga chapters carry their series' match state; the chapter rows
// themselves stay 'pending' and are resolved through the manga series,
// so they must not surface as actionable unmatched items here.
mangaChapterGuard := ` AND ` + catalog.MangaChapterExclusionWhere("mi")
countSQL := `
SELECT COUNT(*)
FROM media_items mi
WHERE mi.status IN ('unmatched', 'pending', 'ambiguous')`
WHERE mi.status IN ('unmatched', 'pending', 'ambiguous')` + mangaChapterGuard
countSQL += filter
var total int
@@ -2521,10 +2546,10 @@ func (h *LibraryHandler) HandleListUnmatchedItems(w http.ResponseWriter, r *http
WHERE mil.content_id = mi.content_id
LIMIT 1
) lib ON true
WHERE mi.status IN ('unmatched', 'pending', 'ambiguous')%s
WHERE mi.status IN ('unmatched', 'pending', 'ambiguous')%s%s
ORDER BY mi.title ASC, mi.content_id ASC
LIMIT $%d OFFSET $%d
`, filter, len(filterArgs)+1, len(filterArgs)+2)
`, mangaChapterGuard, filter, len(filterArgs)+1, len(filterArgs)+2)
rows, err := h.pool.Query(r.Context(), listSQL, listArgs...)
if err != nil {
@@ -0,0 +1,112 @@
package handlers
import (
"context"
"net/http"
"net/http/httptest"
"os"
"strconv"
"strings"
"sync"
"testing"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/adminjob"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
type fakeAdminJobCreator struct {
mu sync.Mutex
refreshs []adminjob.LibraryRefreshRequest
}
func (f *fakeAdminJobCreator) Create(context.Context, adminjob.CreateJobInput) (*models.AdminJob, error) {
return &models.AdminJob{}, nil
}
func (f *fakeAdminJobCreator) CreateLibraryRefresh(_ context.Context, _ int, req adminjob.LibraryRefreshRequest, _ string) (*models.AdminJob, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.refreshs = append(f.refreshs, req)
return &models.AdminJob{}, nil
}
func (f *fakeAdminJobCreator) libraryRefreshes() []adminjob.LibraryRefreshRequest {
f.mu.Lock()
defer f.mu.Unlock()
return append([]adminjob.LibraryRefreshRequest(nil), f.refreshs...)
}
// TestHandleUpdateLibraryLanguageChangeQueuesRefresh covers the trigger half
// of issue #211: changing a library's metadata language must enqueue a
// library metadata refresh so existing items re-fetch in the new language,
// while an update that does not change the language must not enqueue one.
func TestHandleUpdateLibraryLanguageChangeQueuesRefresh(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect db: %v", err)
}
t.Cleanup(pool.Close)
var folderID int
if err := pool.QueryRow(ctx, `
INSERT INTO media_folders (type, name, enabled, metadata_language)
VALUES ('movies', 'Lang Change Test', true, 'zh')
RETURNING id
`).Scan(&folderID); err != nil {
t.Fatalf("seed folder: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
jobs := &fakeAdminJobCreator{}
h := NewLibraryHandler(catalog.NewFolderRepository(pool), nil, nil, pool, nil)
h.JobRepo = jobs
doUpdate := func(body string) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPut, "/libraries/"+strconv.Itoa(folderID), strings.NewReader(body))
chiCtx := chi.NewRouteContext()
chiCtx.URLParams.Add("id", strconv.Itoa(folderID))
req = req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, chiCtx))
rec := httptest.NewRecorder()
h.HandleUpdateLibrary(rec, req)
return rec
}
// An update that does not touch the language must not enqueue a refresh.
if rec := doUpdate(`{"name":"Lang Change Test Renamed"}`); rec.Code != http.StatusOK {
t.Fatalf("rename update status = %d (body: %s)", rec.Code, rec.Body.String())
}
if got := jobs.libraryRefreshes(); len(got) != 0 {
t.Fatalf("refresh jobs after rename = %d, want 0", len(got))
}
// Changing the metadata language must enqueue a refresh for this library.
if rec := doUpdate(`{"metadata_language":"da"}`); rec.Code != http.StatusOK {
t.Fatalf("language update status = %d (body: %s)", rec.Code, rec.Body.String())
}
refreshes := jobs.libraryRefreshes()
if len(refreshes) != 1 {
t.Fatalf("refresh jobs after language change = %d, want 1", len(refreshes))
}
if refreshes[0].LibraryID != folderID {
t.Errorf("refresh library id = %d, want %d", refreshes[0].LibraryID, folderID)
}
// Re-submitting the same language is a no-op update and must not enqueue.
if rec := doUpdate(`{"metadata_language":"da"}`); rec.Code != http.StatusOK {
t.Fatalf("same-language update status = %d (body: %s)", rec.Code, rec.Body.String())
}
if got := jobs.libraryRefreshes(); len(got) != 1 {
t.Fatalf("refresh jobs after same-language update = %d, want still 1", len(got))
}
}
@@ -0,0 +1,104 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// TestHandleListUnmatchedItemsExcludesMangaChapters verifies that manga
// chapter rows (type='ebook' items linked into a series via manga_chapters)
// do not appear in the admin Unmatched queue, while genuinely pending items
// still do. Regression test for issue #204.
func TestHandleListUnmatchedItemsExcludesMangaChapters(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("set SILO_TEST_DATABASE_URL to run DB-backed unmatched items handler test")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect db: %v", err)
}
t.Cleanup(pool.Close)
suffix := time.Now().UnixNano()
// Embed a unique per-run tag in every seeded title and scope the search
// query to it, so concurrent or leftover rows in a shared test database
// cannot skew the exact-total assertion below.
tag := fmt.Sprintf("issue204-%d", suffix)
seriesID := fmt.Sprintf("manga-series-%d", suffix)
chapterID := fmt.Sprintf("manga-chapter-%d", suffix)
pendingID := fmt.Sprintf("pending-ebook-%d", suffix)
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = ANY($1)`,
[]string{chapterID, seriesID, pendingID})
})
for _, row := range []struct {
id, typ, title, status string
}{
{seriesID, "manga", tag + " Series", "matched"},
{chapterID, "ebook", tag + " Series c001", "pending"},
{pendingID, "ebook", tag + " Plain Ebook", "pending"},
} {
if _, err := pool.Exec(ctx, `
INSERT INTO media_items (content_id, type, title, status, genres)
VALUES ($1, $2, $3, $4, '{}'::text[])
`, row.id, row.typ, row.title, row.status); err != nil {
t.Fatalf("seed media item %s: %v", row.id, err)
}
}
if _, err := pool.Exec(ctx, `
INSERT INTO manga_chapters (chapter_content_id, series_content_id, chapter_index)
VALUES ($1, $2, 1)
`, chapterID, seriesID); err != nil {
t.Fatalf("seed manga chapter link: %v", err)
}
h := NewLibraryHandler(nil, nil, nil, pool, nil)
req := httptest.NewRequest(http.MethodGet, "/libraries/unmatched-items?q="+url.QueryEscape(tag), nil)
rec := httptest.NewRecorder()
h.HandleListUnmatchedItems(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
var resp struct {
Items []struct {
ContentID string `json:"content_id"`
} `json:"items"`
Total int `json:"total"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
var sawChapter, sawPending bool
for _, item := range resp.Items {
switch item.ContentID {
case chapterID:
sawChapter = true
case pendingID:
sawPending = true
}
}
if !sawPending {
t.Errorf("pending ebook %s missing from unmatched list", pendingID)
}
if sawChapter {
t.Errorf("manga chapter %s should be excluded from unmatched list", chapterID)
}
if resp.Total != 1 {
t.Errorf("total = %d, want 1 (chapter must be excluded from the count too)", resp.Total)
}
}
+93 -6
View File
@@ -1,6 +1,7 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"log/slog"
@@ -13,8 +14,27 @@ import (
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/metadata/translation"
"github.com/Silo-Server/silo-server/internal/models"
)
type metadataAIItemAccess interface {
GetByID(ctx context.Context, contentID string) (*models.MediaItem, error)
EnsureAccessible(ctx context.Context, contentID string, filter catalog.AccessFilter) error
}
type metadataAISeasonLookup interface {
GetByID(ctx context.Context, contentID string) (*models.Season, error)
}
type metadataAIEpisodeLookup interface {
GetByID(ctx context.Context, contentID string) (*models.Episode, error)
}
type metadataAITarget struct {
kind translation.TargetKind
accessContentID string
}
// MetadataAIHandler exposes AI translation of catalog descriptions into the
// localization tables. The admin routes are mounted under the per-item
// metadata curation guard; the on-view route is viewer-facing and enforces
@@ -22,7 +42,11 @@ import (
type MetadataAIHandler struct {
service *translation.Service
// ItemAccess authorizes the viewer-facing on-view route; nil disables it.
ItemAccess *catalog.ItemRepository
ItemAccess metadataAIItemAccess
// SeasonLookup and EpisodeLookup let the viewer route resolve non-item
// detail pages to the parent series for authorization.
SeasonLookup metadataAISeasonLookup
EpisodeLookup metadataAIEpisodeLookup
}
// NewMetadataAIHandler creates a handler backed by the given service.
@@ -82,7 +106,16 @@ func (h *MetadataAIHandler) HandleTranslateOnView(w http.ResponseWriter, r *http
UserID: scope.UserID,
ProfileID: scope.ProfileID,
}
if err := h.ItemAccess.EnsureAccessible(r.Context(), contentID, filter); err != nil {
target, err := h.resolveTranslationTarget(r.Context(), contentID)
if err != nil {
if errors.Is(err, catalog.ErrItemNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to authorize item")
return
}
if err := h.ItemAccess.EnsureAccessible(r.Context(), target.accessContentID, filter); err != nil {
if errors.Is(err, catalog.ErrItemNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
@@ -96,7 +129,7 @@ func (h *MetadataAIHandler) HandleTranslateOnView(w http.ResponseWriter, r *http
requestedBy = &userID
}
job, err := h.service.RequestOnView(r.Context(), contentID, req.TargetLanguage, requestedBy)
job, err := h.service.RequestOnView(r.Context(), target.kind, contentID, req.TargetLanguage, requestedBy)
if err != nil {
switch {
case errors.Is(err, translation.ErrNotConfigured):
@@ -115,6 +148,51 @@ func (h *MetadataAIHandler) HandleTranslateOnView(w http.ResponseWriter, r *http
writeJSON(w, http.StatusAccepted, map[string]any{"job": job})
}
func (h *MetadataAIHandler) resolveTranslationTarget(ctx context.Context, contentID string) (metadataAITarget, error) {
if h.ItemAccess == nil {
return metadataAITarget{}, catalog.ErrItemNotFound
}
item, err := h.ItemAccess.GetByID(ctx, contentID)
switch {
case err == nil:
if item == nil {
return metadataAITarget{}, catalog.ErrItemNotFound
}
return metadataAITarget{kind: translation.TargetItem, accessContentID: contentID}, nil
case !errors.Is(err, catalog.ErrItemNotFound):
return metadataAITarget{}, err
}
if h.SeasonLookup != nil {
season, err := h.SeasonLookup.GetByID(ctx, contentID)
switch {
case err == nil:
if season == nil {
return metadataAITarget{}, catalog.ErrItemNotFound
}
return metadataAITarget{kind: translation.TargetSeason, accessContentID: season.SeriesID}, nil
case !errors.Is(err, catalog.ErrSeasonNotFound):
return metadataAITarget{}, err
}
}
if h.EpisodeLookup != nil {
episode, err := h.EpisodeLookup.GetByID(ctx, contentID)
switch {
case err == nil:
if episode == nil {
return metadataAITarget{}, catalog.ErrItemNotFound
}
return metadataAITarget{kind: translation.TargetEpisode, accessContentID: episode.SeriesID}, nil
case !errors.Is(err, catalog.ErrEpisodeNotFound):
return metadataAITarget{}, err
}
}
return metadataAITarget{}, catalog.ErrItemNotFound
}
type translateMetadataRequest struct {
TargetLanguage string `json:"target_language"`
IncludeChildren *bool `json:"include_children"` // default true
@@ -134,9 +212,18 @@ func (h *MetadataAIHandler) HandleTranslate(w http.ResponseWriter, r *http.Reque
writeError(w, http.StatusBadRequest, "bad_request", "target_language is required")
return
}
includeChildren := true
target, err := h.resolveTranslationTarget(r.Context(), contentID)
if err != nil {
if errors.Is(err, catalog.ErrItemNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to resolve item")
return
}
includeChildren := target.kind == translation.TargetItem
if req.IncludeChildren != nil {
includeChildren = *req.IncludeChildren
includeChildren = *req.IncludeChildren && target.kind == translation.TargetItem
}
var requestedBy *int
@@ -145,7 +232,7 @@ func (h *MetadataAIHandler) HandleTranslate(w http.ResponseWriter, r *http.Reque
}
job, err := h.service.Enqueue(r.Context(), translation.JobRequest{
TargetKind: translation.TargetItem,
TargetKind: target.kind,
ContentID: contentID,
TargetLanguage: req.TargetLanguage,
IncludeChildren: includeChildren,
+124
View File
@@ -0,0 +1,124 @@
package handlers
import (
"context"
"errors"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/metadata/translation"
"github.com/Silo-Server/silo-server/internal/models"
)
type fakeMetadataAIItemAccess struct {
items map[string]*models.MediaItem
ensureErr map[string]error
checked []string
}
func (f *fakeMetadataAIItemAccess) GetByID(_ context.Context, contentID string) (*models.MediaItem, error) {
if item := f.items[contentID]; item != nil {
return item, nil
}
return nil, catalog.ErrItemNotFound
}
func (f *fakeMetadataAIItemAccess) EnsureAccessible(_ context.Context, contentID string, _ catalog.AccessFilter) error {
f.checked = append(f.checked, contentID)
return f.ensureErr[contentID]
}
type fakeMetadataAISeasonLookup map[string]*models.Season
func (f fakeMetadataAISeasonLookup) GetByID(_ context.Context, contentID string) (*models.Season, error) {
if season := f[contentID]; season != nil {
return season, nil
}
return nil, catalog.ErrSeasonNotFound
}
type fakeMetadataAIEpisodeLookup map[string]*models.Episode
func (f fakeMetadataAIEpisodeLookup) GetByID(_ context.Context, contentID string) (*models.Episode, error) {
if episode := f[contentID]; episode != nil {
return episode, nil
}
return nil, catalog.ErrEpisodeNotFound
}
func TestMetadataAIResolveOnViewTarget(t *testing.T) {
itemAccess := &fakeMetadataAIItemAccess{
items: map[string]*models.MediaItem{
"movie-1": {ContentID: "movie-1", Type: "movie"},
},
ensureErr: map[string]error{},
}
handler := &MetadataAIHandler{
ItemAccess: itemAccess,
SeasonLookup: fakeMetadataAISeasonLookup{
"season-1": {ContentID: "season-1", SeriesID: "series-1"},
},
EpisodeLookup: fakeMetadataAIEpisodeLookup{
"episode-1": {ContentID: "episode-1", SeriesID: "series-1"},
},
}
tests := []struct {
name string
contentID string
wantKind translation.TargetKind
wantAccess string
}{
{
name: "item",
contentID: "movie-1",
wantKind: translation.TargetItem,
wantAccess: "movie-1",
},
{
name: "season",
contentID: "season-1",
wantKind: translation.TargetSeason,
wantAccess: "series-1",
},
{
name: "episode",
contentID: "episode-1",
wantKind: translation.TargetEpisode,
wantAccess: "series-1",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
itemAccess.checked = nil
got, err := handler.resolveTranslationTarget(context.Background(), tc.contentID)
if err != nil {
t.Fatalf("resolveTranslationTarget: %v", err)
}
if got.kind != tc.wantKind {
t.Fatalf("kind = %s, want %s", got.kind, tc.wantKind)
}
if got.accessContentID != tc.wantAccess {
t.Fatalf("accessContentID = %q, want %q", got.accessContentID, tc.wantAccess)
}
})
}
}
func TestMetadataAIResolveTranslationTargetReportsMissingContent(t *testing.T) {
itemAccess := &fakeMetadataAIItemAccess{
items: map[string]*models.MediaItem{},
ensureErr: map[string]error{},
}
handler := &MetadataAIHandler{
ItemAccess: itemAccess,
SeasonLookup: fakeMetadataAISeasonLookup{},
EpisodeLookup: fakeMetadataAIEpisodeLookup{},
}
_, err := handler.resolveTranslationTarget(context.Background(), "missing")
if !errors.Is(err, catalog.ErrItemNotFound) {
t.Fatalf("err = %v, want %v", err, catalog.ErrItemNotFound)
}
}
+39 -4
View File
@@ -43,6 +43,8 @@ type notificationSyncResponse struct {
UnreadCount int `json:"unread_count"`
}
type notificationApplePushDisplayResponse = notifications.NotificationDisplay
type unreadCountResponse struct {
Count int `json:"count"`
}
@@ -150,6 +152,29 @@ func (h *NotificationsHandler) HandleGet(w http.ResponseWriter, r *http.Request)
writeJSON(w, http.StatusOK, h.system.PayloadForRow(r.Context(), *row))
}
// HandleApplePushDisplay handles GET /notifications/push/apple/display/{delivery_id}.
// It returns only compact display metadata for notification-service-extension
// enrichment, scoped to the active profile.
func (h *NotificationsHandler) HandleApplePushDisplay(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "no-store")
profileID := apimw.GetProfileID(r.Context())
id := chi.URLParam(r, "delivery_id")
if id == "" {
writeError(w, http.StatusNotFound, "not_found", "Notification not found")
return
}
row, err := h.system.Deliveries.GetByID(r.Context(), profileID, id)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load notification")
return
}
if row == nil {
writeError(w, http.StatusNotFound, "not_found", "Notification not found")
return
}
writeJSON(w, http.StatusOK, notificationApplePushDisplayResponse(notifications.BuildNotificationDisplay(*row)))
}
// HandleUnreadCount handles GET /notifications/unread-count.
func (h *NotificationsHandler) HandleUnreadCount(w http.ResponseWriter, r *http.Request) {
profileID := apimw.GetProfileID(r.Context())
@@ -318,9 +343,7 @@ type capabilityWebhooks struct {
}
// HandleCapability handles GET /notifications/capability. Clients render
// setup UI from this response instead of introspecting admin settings. Push
// channels report unavailable until they ship
// (docs/superpowers/plans/notifications/02-03).
// setup UI from this response instead of introspecting admin settings.
func (h *NotificationsHandler) HandleCapability(w http.ResponseWriter, r *http.Request) {
webhooks := capabilityWebhooks{Available: false, MaxPerProfile: 0, SupportedTypes: []string{}}
if h.system.Webhooks != nil && h.system.Settings.WebhooksEnabled(r.Context()) {
@@ -364,9 +387,21 @@ func (h *NotificationsHandler) HandleCapability(w http.ResponseWriter, r *http.R
DigestHour: h.system.Settings.DiscordDigestHour(r.Context()),
}
}
applePush := capabilityPush{Available: false, Provider: "off", SupportedModes: []string{"in_app_only"}}
// Like web push above, availability requires both the wiring (cipher/store)
// and the admin delivery toggle: Available must mean "setup will actually
// deliver", not "the server could store a token".
if h.system.PushDevices != nil && h.system.PushDevices.Available() &&
h.system.Settings.ApplePushDeliveryEnabled(r.Context()) {
applePush = capabilityPush{
Available: true,
Provider: notifications.PushProviderSiloRelay,
SupportedModes: []string{notifications.PushModePrivatePush, notifications.PushModeInAppOnly},
}
}
writeJSON(w, http.StatusOK, capabilityResponse{
InApp: capabilityInApp{Enabled: h.system.Settings.UIEnabled(r.Context())},
ApplePush: capabilityPush{Available: false, Provider: "off", SupportedModes: []string{"in_app_only"}},
ApplePush: applePush,
AndroidPush: capabilityPush{Available: false, Provider: "off", SupportedModes: []string{"in_app_only"}},
WebPush: webPush,
Webhooks: webhooks,
@@ -0,0 +1,134 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/notifications"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestHandleApplePushDisplayDB(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("set SILO_TEST_DATABASE_URL to run DB-backed notification display handler test")
}
ctx := context.Background()
config, err := pgxpool.ParseConfig(dsn)
if err != nil {
t.Fatalf("parse db config: %v", err)
}
config.MaxConns = 1
pool, err := pgxpool.NewWithConfig(ctx, config)
if err != nil {
t.Fatalf("connect db: %v", err)
}
t.Cleanup(pool.Close)
if _, err := pool.Exec(ctx, `
CREATE TEMP TABLE notification_deliveries (
id text PRIMARY KEY,
release_event_id text,
user_id integer NOT NULL,
profile_id text NOT NULL,
library_id integer,
series_id text,
episode_id text,
type text NOT NULL,
reason_flags jsonb NOT NULL,
status text NOT NULL DEFAULT 'delivered',
read_at timestamptz,
delivered_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TEMP TABLE episodes (
content_id text PRIMARY KEY,
title text,
season_number integer,
episode_number integer,
overview text
);
CREATE TEMP TABLE media_items (
content_id text PRIMARY KEY,
title text,
poster_path text,
poster_thumbhash text,
poster_source_path text,
type text,
year integer,
overview text,
genres text[],
content_rating text,
rating_imdb double precision,
rating_tmdb double precision,
imdb_id text,
tmdb_id text,
tvdb_id text
);
INSERT INTO media_items (content_id, title, type, genres)
VALUES ('series-1', 'Severance', 'series', ARRAY[]::text[]);
INSERT INTO episodes (content_id, title, season_number, episode_number)
VALUES ('episode-1', 'Hello, Ms. Cobel', 2, 1);
INSERT INTO notification_deliveries (
id, release_event_id, user_id, profile_id, library_id, series_id, episode_id,
type, reason_flags, status
) VALUES (
'delivery-1', 'event-1', 42, 'profile-1', 7, 'series-1', 'episode-1',
'episode.available', '{"favorite":true}', 'delivered'
);
`); err != nil {
t.Fatalf("seed temp tables: %v", err)
}
handler := NewNotificationsHandler(&notifications.System{
Deliveries: notifications.NewDeliveryRepository(pool),
}, nil)
router := chi.NewRouter()
router.Get("/notifications/push/apple/display/{delivery_id}", handler.HandleApplePushDisplay)
req := httptest.NewRequest(http.MethodGet, "/notifications/push/apple/display/delivery-1", nil)
req = req.WithContext(apimw.SetProfileID(req.Context(), "profile-1"))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
if got := rr.Header().Get("Cache-Control"); got != "no-store" {
t.Fatalf("Cache-Control = %q", got)
}
var response notifications.NotificationDisplay
if err := json.NewDecoder(rr.Body).Decode(&response); err != nil {
t.Fatalf("decode response: %v", err)
}
if response.DeliveryID != "delivery-1" ||
response.Title != "The latest episode of Severance S02E01 just dropped!" ||
response.Body != "Hello, Ms. Cobel" ||
response.ThreadID != "series:series-1" ||
response.Category != "episode_available" ||
response.URL != "/item/episode-1" {
t.Fatalf("response = %+v", response)
}
req = httptest.NewRequest(http.MethodGet, "/notifications/push/apple/display/delivery-1", nil)
req = req.WithContext(apimw.SetProfileID(req.Context(), "other-profile"))
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusNotFound {
t.Fatalf("cross-profile status = %d, body = %s", rr.Code, rr.Body.String())
}
if got := rr.Header().Get("Cache-Control"); got != "no-store" {
t.Fatalf("cross-profile Cache-Control = %q", got)
}
if strings.Contains(rr.Body.String(), "Severance") {
t.Fatalf("cross-profile response leaked display metadata: %s", rr.Body.String())
}
}
@@ -0,0 +1,75 @@
package handlers
import (
"encoding/json"
"errors"
"net/http"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/notifications"
)
type applePushRegisterRequest struct {
DeviceID string `json:"device_id"`
APNsToken string `json:"apns_token"`
APNsEnvironment string `json:"apns_environment"`
APNsTopic string `json:"apns_topic"`
PushMode string `json:"push_mode"`
}
type applePushRegisterResponse struct {
ID string `json:"id"`
ServerDeviceID string `json:"server_device_id"`
Enabled bool `json:"enabled"`
PushMode string `json:"push_mode"`
}
func (h *NotificationsHandler) pushDevices() *notifications.PushDeviceService {
if h == nil || h.system == nil {
return nil
}
return h.system.PushDevices
}
// HandleRegisterApplePushDevice handles POST /devices/push/apple.
func (h *NotificationsHandler) HandleRegisterApplePushDevice(w http.ResponseWriter, r *http.Request) {
service := h.pushDevices()
if service == nil || !service.Available() {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Apple push registration is not available")
return
}
var req applePushRegisterRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
device, err := service.RegisterApple(r.Context(), apimw.GetUserID(r.Context()), apimw.GetProfileID(r.Context()), notifications.ApplePushRegistrationInput{
DeviceID: req.DeviceID,
APNsToken: req.APNsToken,
APNsEnvironment: req.APNsEnvironment,
APNsTopic: req.APNsTopic,
PushMode: req.PushMode,
})
if err != nil {
switch {
case errors.Is(err, notifications.ErrPushDeviceInvalid):
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
case errors.Is(err, notifications.ErrPushDeviceUnsupported):
writeError(w, http.StatusUnprocessableEntity, "unsupported_push_device", err.Error())
case errors.Is(err, notifications.ErrPushDeviceUnavailable):
writeError(w, http.StatusServiceUnavailable, "unavailable", "Apple push registration is not available")
default:
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to register Apple push device")
}
return
}
writeJSON(w, http.StatusOK, applePushRegisterResponse{
ID: device.ID,
ServerDeviceID: device.ServerDeviceID,
Enabled: device.Enabled,
PushMode: device.PushMode,
})
}
@@ -0,0 +1,152 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/notifications"
"github.com/Silo-Server/silo-server/internal/secret"
)
type handlerPushStore struct {
got notifications.ApplePushDeviceRegistration
calls int
device *notifications.PushDevice
err error
}
func (f *handlerPushStore) UpsertApple(ctx context.Context, registration notifications.ApplePushDeviceRegistration, cipher *secret.Cipher) (*notifications.PushDevice, error) {
f.calls++
f.got = registration
if f.err != nil {
return nil, f.err
}
if f.device != nil {
return f.device, nil
}
return &notifications.PushDevice{
ID: "push-row",
ServerDeviceID: "server-device",
Enabled: true,
PushMode: registration.PushMode,
}, nil
}
func handlerPushCipher(t *testing.T) *secret.Cipher {
t.Helper()
cipher, err := secret.New([]byte("01234567890123456789012345678901"))
if err != nil {
t.Fatalf("new cipher: %v", err)
}
return cipher
}
func newApplePushRequest(body string) *http.Request {
req := httptest.NewRequest(http.MethodPost, "/api/v1/devices/push/apple", strings.NewReader(body))
ctx := apimw.SetClaims(req.Context(), &auth.Claims{UserID: 42, Role: "user", TokenType: auth.TokenTypeAccess})
ctx = apimw.SetProfileID(ctx, "profile-1")
return req.WithContext(ctx)
}
func TestHandleRegisterApplePushDevice(t *testing.T) {
store := &handlerPushStore{}
handler := NewNotificationsHandler(&notifications.System{
PushDevices: notifications.NewPushDeviceService(store, handlerPushCipher(t)),
}, nil)
body := `{
"device_id":"local-device",
"apns_token":"` + strings.Repeat("a", 64) + `",
"apns_environment":"production",
"apns_topic":"org.siloserver.silo",
"push_mode":"private_push"
}`
rr := httptest.NewRecorder()
handler.HandleRegisterApplePushDevice(rr, newApplePushRequest(body))
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
var response applePushRegisterResponse
if err := json.NewDecoder(rr.Body).Decode(&response); err != nil {
t.Fatalf("decode response: %v", err)
}
if response.ID != "push-row" || response.ServerDeviceID != "server-device" || !response.Enabled || response.PushMode != notifications.PushModePrivatePush {
t.Fatalf("unexpected response: %+v", response)
}
if store.calls != 1 {
t.Fatalf("store calls = %d, want 1", store.calls)
}
if store.got.UserID != 42 || store.got.ProfileID != "profile-1" || store.got.DeviceID != "local-device" {
t.Fatalf("unexpected stored registration: %+v", store.got)
}
}
func TestHandleRegisterApplePushDeviceErrors(t *testing.T) {
tests := []struct {
name string
system *notifications.System
body string
wantStatus int
}{
{
name: "service unavailable",
system: &notifications.System{},
body: `{}`,
wantStatus: http.StatusServiceUnavailable,
},
{
name: "invalid json",
system: &notifications.System{
PushDevices: notifications.NewPushDeviceService(&handlerPushStore{}, handlerPushCipher(t)),
},
body: `{`,
wantStatus: http.StatusBadRequest,
},
{
name: "invalid field",
system: &notifications.System{
PushDevices: notifications.NewPushDeviceService(&handlerPushStore{}, handlerPushCipher(t)),
},
body: `{
"device_id":"local-device",
"apns_token":"abcd",
"apns_environment":"production",
"apns_topic":"org.siloserver.silo",
"push_mode":"private_push"
}`,
wantStatus: http.StatusBadRequest,
},
{
name: "unsupported topic",
system: &notifications.System{
PushDevices: notifications.NewPushDeviceService(&handlerPushStore{}, handlerPushCipher(t)),
},
body: `{
"device_id":"local-device",
"apns_token":"` + strings.Repeat("a", 64) + `",
"apns_environment":"production",
"apns_topic":"com.example.app",
"push_mode":"private_push"
}`,
wantStatus: http.StatusUnprocessableEntity,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
handler := NewNotificationsHandler(tt.system, nil)
rr := httptest.NewRecorder()
handler.HandleRegisterApplePushDevice(rr, newApplePushRequest(tt.body))
if rr.Code != tt.wantStatus {
t.Fatalf("status = %d, want %d, body = %s", rr.Code, tt.wantStatus, rr.Body.String())
}
})
}
}
File diff suppressed because it is too large Load Diff
+8 -4
View File
@@ -13,7 +13,11 @@ type playbackCommandRecord struct {
Name playback.CommandName
}
func (h *PlaybackHandler) stopPlaybackSession(ctx context.Context, session *playback.Session) error {
// stopPlaybackSession stops a session. userInitiated must be true only for a
// genuine user/admin stop (the recipe card is then deleted); false for system
// teardown such as an ffmpeg-exit cleanup, where the card is kept so the client
// can reconstruct.
func (h *PlaybackHandler) stopPlaybackSession(ctx context.Context, session *playback.Session, userInitiated bool) error {
if h == nil || session == nil || session.ID == "" {
return playback.ErrSessionNotFound
}
@@ -21,11 +25,11 @@ func (h *PlaybackHandler) stopPlaybackSession(ctx context.Context, session *play
if err := h.sessionMgr.StopSession(session.ID); err != nil {
return err
}
h.finalizeSessionStop(ctx, session, true, "stop")
h.finalizeSessionStop(ctx, session, true, "stop", userInitiated)
return nil
}
func (h *PlaybackHandler) stopPlaybackSessionByID(ctx context.Context, sessionID string) error {
func (h *PlaybackHandler) stopPlaybackSessionByID(ctx context.Context, sessionID string, userInitiated bool) error {
if h == nil || sessionID == "" {
return playback.ErrSessionNotFound
}
@@ -33,7 +37,7 @@ func (h *PlaybackHandler) stopPlaybackSessionByID(ctx context.Context, sessionID
if err != nil {
return err
}
return h.stopPlaybackSession(ctx, session)
return h.stopPlaybackSession(ctx, session, userInitiated)
}
func (h *PlaybackHandler) abortPlaybackSession(ctx context.Context, session *playback.Session) error {
+241 -2
View File
@@ -22,6 +22,7 @@ import (
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/streamtoken"
"github.com/Silo-Server/silo-server/internal/transcodenode"
"github.com/Silo-Server/silo-server/internal/userdb"
"github.com/Silo-Server/silo-server/internal/userstore"
@@ -1162,6 +1163,244 @@ func TestHandleStartTranscode_PreservesRecomputedBaseMethodAfterFallback(t *test
}
}
// newRemoteAudioSwitchSession builds a session that is already an offloaded
// (remote) transcode with a populated target recipe, mirroring the post-start
// state HandleChangeAudioTrack sees for an offloaded transcode.
func newRemoteAudioSwitchSession(t *testing.T, sessionMgr *playback.SessionManager, file *models.MediaFile, nodeURL string) string {
t.Helper()
session, err := sessionMgr.StartSession(1, "profile-1", file.ID, playback.PlayTranscode, true)
if err != nil {
t.Fatalf("StartSession: %v", err)
}
if err := sessionMgr.UpdateStreamState(session.ID, playback.SessionStreamState{
PlayMethod: playback.PlayTranscode,
BasePlayMethod: playback.PlayTranscode,
AudioTrackIndex: 0,
TranscodeAudio: true,
TargetResolution: "720p",
TargetVideoCodec: "h264",
TargetAudioCodec: "aac",
TargetBitrateKbps: 2000,
TranscodeHWAccel: "none",
}); err != nil {
t.Fatalf("UpdateStreamState: %v", err)
}
if err := sessionMgr.SetTranscodeNodeURL(session.ID, nodeURL); err != nil {
t.Fatalf("SetTranscodeNodeURL: %v", err)
}
return session.ID
}
// TestHandleChangeAudioTrack_RemoteTranscodeRestartsNodeAndMintsFullRecipe
// verifies BUG A + BUG B: an audio switch on an offloaded transcode POSTs a
// fresh /transcode/start to the node carrying the NEW AudioTrackIndex (so the
// node's ffmpeg actually switches tracks), and the returned proxy URL carries a
// recipe-COMPLETE stream token (not identity-only) so a later node restart can
// reconstruct with the switched audio.
func TestHandleChangeAudioTrack_RemoteTranscodeRestartsNodeAndMintsFullRecipe(t *testing.T) {
sessionMgr := playback.NewSessionManager(0, 0)
file := &models.MediaFile{
ID: 42,
ContentID: "movie-1",
FilePath: writePlaybackTestMediaFile(t, "movie.mkv"),
Resolution: "1080p",
CodecVideo: "h264",
CodecAudio: "aac",
Container: "mkv",
Bitrate: 8000,
Duration: 3600,
AudioTracks: []models.AudioTrack{
{Codec: "aac", Default: true},
{Codec: "ac3"},
},
}
var remoteStartRequests int
var remoteStartReq transcodenode.TranscodeStartRequest
remote := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || r.URL.Path != "/transcode/start" {
t.Errorf("unexpected remote request %s %s", r.Method, r.URL.Path)
http.Error(w, "unexpected", http.StatusBadRequest)
return
}
remoteStartRequests++
if err := json.NewDecoder(r.Body).Decode(&remoteStartReq); err != nil {
t.Errorf("decode remote start request: %v", err)
}
w.WriteHeader(http.StatusAccepted)
_ = json.NewEncoder(w).Encode(transcodenode.TranscodeStartResponse{
SessionID: remoteStartReq.SessionID,
Status: "accepted",
HWAccel: "none",
})
}))
defer remote.Close()
handler := NewPlaybackHandler(sessionMgr, testPlaybackFileResolver{file: file})
handler.ItemAccess = allowAllPlaybackItemAccess{}
handler.JWTSecret = "test-secret"
group := "rack-a"
proxies := nodepool.NewProxyPool()
proxies.SetNodes([]*nodepool.Node{{
ID: 1, Name: "proxy-1", Type: nodepool.NodeTypeProxy, URL: "http://proxy-1",
Enabled: true, Healthy: true, Group: &group,
}})
transcodes := nodepool.NewTranscodePool()
transcodes.SetNodes([]*nodepool.Node{{
ID: 2, Name: "transcode-1", Type: nodepool.NodeTypeTranscode, URL: remote.URL,
Enabled: true, Healthy: true, Group: &group,
}})
handler.NodePlanner = nodepool.NewPlanner(proxies, transcodes)
sessionID := newRemoteAudioSwitchSession(t, sessionMgr, file, remote.URL)
changeReq := httptest.NewRequest(
http.MethodPatch,
"/api/v1/playback/"+sessionID+"/audio",
strings.NewReader(`{"audio_track_index":1,"position":120}`),
)
changeReq = changeReq.WithContext(newAuthorizedPlaybackContext())
changeReq = withPlaybackRouteParam(changeReq, "session_id", sessionID)
rr := httptest.NewRecorder()
handler.HandleChangeAudioTrack(rr, changeReq)
if rr.Code != http.StatusOK {
t.Fatalf("change status = %d, body = %s", rr.Code, rr.Body.String())
}
// BUG A: the node must have been told to restart on the new track.
if remoteStartRequests != 1 {
t.Fatalf("remote /transcode/start calls = %d, want 1", remoteStartRequests)
}
if remoteStartReq.AudioTrackIndex != 1 {
t.Fatalf("remote AudioTrackIndex = %d, want 1", remoteStartReq.AudioTrackIndex)
}
if remoteStartReq.SessionID != sessionID {
t.Fatalf("remote SessionID = %q, want %q", remoteStartReq.SessionID, sessionID)
}
// Seek 120s with the node-default 2s segments => start segment 60.
if remoteStartReq.StartSegmentNumber != 60 {
t.Fatalf("remote StartSegmentNumber = %d, want 60", remoteStartReq.StartSegmentNumber)
}
if remoteStartReq.TargetResolution != "720p" || remoteStartReq.TargetCodecVideo != "h264" {
t.Fatalf("remote target recipe = %q/%q, want 720p/h264", remoteStartReq.TargetResolution, remoteStartReq.TargetCodecVideo)
}
var resp changeAudioResponse
if err := json.NewDecoder(rr.Body).Decode(&resp); err != nil {
t.Fatalf("decode change response: %v", err)
}
// BUG B: the returned proxy URL token must be recipe-COMPLETE.
prefix := "http://proxy-1/stream/transcode/"
suffix := "/master.m3u8"
if !strings.HasPrefix(resp.StreamURL, prefix) || !strings.HasSuffix(resp.StreamURL, suffix) {
t.Fatalf("stream URL = %q, want proxy transcode manifest URL", resp.StreamURL)
}
token := strings.TrimSuffix(strings.TrimPrefix(resp.StreamURL, prefix), suffix)
claims, err := streamtoken.Verify(token, handler.JWTSecret)
if err != nil {
t.Fatalf("verify stream token: %v", err)
}
if claims.AudioTrackIndex != 1 {
t.Fatalf("token AudioTrackIndex = %d, want 1", claims.AudioTrackIndex)
}
// Identity-only claims would leave these zero; a full recipe carries them.
if claims.TargetCodec != "h264" {
t.Fatalf("token TargetCodec (video) = %q, want h264 (recipe-complete)", claims.TargetCodec)
}
if claims.TargetCodecAudio != "aac" {
t.Fatalf("token TargetCodecAudio = %q, want aac (recipe-complete)", claims.TargetCodecAudio)
}
if claims.TargetRes != "720p" {
t.Fatalf("token TargetRes = %q, want 720p (recipe-complete)", claims.TargetRes)
}
if claims.TargetBitrateKbps != 2000 {
t.Fatalf("token TargetBitrateKbps = %d, want 2000 (recipe-complete)", claims.TargetBitrateKbps)
}
if claims.SeekSeconds != 120 {
t.Fatalf("token SeekSeconds = %v, want 120", claims.SeekSeconds)
}
if claims.StartSegmentNumber != 60 {
t.Fatalf("token StartSegmentNumber = %d, want 60", claims.StartSegmentNumber)
}
// SegmentDuration must be > 0: the node treats a zero/omitted value as an
// incomplete token and falls back to a recipe store the native path never
// populates, which would 404 on a node restart. This guards that regression.
if claims.SegmentDuration != playback.DefaultSegmentDuration {
t.Fatalf("token SegmentDuration = %d, want %d (recipe-complete)", claims.SegmentDuration, playback.DefaultSegmentDuration)
}
if claims.MediaPath != file.FilePath {
t.Fatalf("token MediaPath = %q, want %q", claims.MediaPath, file.FilePath)
}
if claims.TranscodeNode != remote.URL {
t.Fatalf("token TranscodeNode = %q, want %q", claims.TranscodeNode, remote.URL)
}
}
// TestHandleChangeAudioTrack_RemoteTranscodeNodeUnreachableSurfacesError
// verifies the chosen error posture: when the transcode node cannot be reached
// for the restart, the handler surfaces a 502 rather than returning a 200 that
// clients (silo-android / silo-apple) would trust as "audio switched".
func TestHandleChangeAudioTrack_RemoteTranscodeNodeUnreachableSurfacesError(t *testing.T) {
sessionMgr := playback.NewSessionManager(0, 0)
file := &models.MediaFile{
ID: 42,
ContentID: "movie-1",
FilePath: writePlaybackTestMediaFile(t, "movie.mkv"),
Resolution: "1080p",
CodecVideo: "h264",
CodecAudio: "aac",
Container: "mkv",
Bitrate: 8000,
Duration: 3600,
AudioTracks: []models.AudioTrack{
{Codec: "aac", Default: true},
{Codec: "ac3"},
},
}
// A node URL that nothing is listening on => connection refused.
deadNodeURL := "http://127.0.0.1:1"
handler := NewPlaybackHandler(sessionMgr, testPlaybackFileResolver{file: file})
handler.ItemAccess = allowAllPlaybackItemAccess{}
handler.JWTSecret = "test-secret"
group := "rack-a"
proxies := nodepool.NewProxyPool()
proxies.SetNodes([]*nodepool.Node{{
ID: 1, Name: "proxy-1", Type: nodepool.NodeTypeProxy, URL: "http://proxy-1",
Enabled: true, Healthy: true, Group: &group,
}})
transcodes := nodepool.NewTranscodePool()
transcodes.SetNodes([]*nodepool.Node{{
ID: 2, Name: "transcode-1", Type: nodepool.NodeTypeTranscode, URL: deadNodeURL,
Enabled: true, Healthy: true, Group: &group,
}})
handler.NodePlanner = nodepool.NewPlanner(proxies, transcodes)
sessionID := newRemoteAudioSwitchSession(t, sessionMgr, file, deadNodeURL)
changeReq := httptest.NewRequest(
http.MethodPatch,
"/api/v1/playback/"+sessionID+"/audio",
strings.NewReader(`{"audio_track_index":1,"position":0}`),
)
changeReq = changeReq.WithContext(newAuthorizedPlaybackContext())
changeReq = withPlaybackRouteParam(changeReq, "session_id", sessionID)
rr := httptest.NewRecorder()
handler.HandleChangeAudioTrack(rr, changeReq)
// Chosen posture: surface the failure (mirror HandleStartTranscode) so a 200
// never lies to the client about an audio switch that didn't happen.
if rr.Code != http.StatusBadGateway {
t.Fatalf("change status = %d, want %d, body = %s", rr.Code, http.StatusBadGateway, rr.Body.String())
}
}
func TestHandleStartTranscode_LocalPathPropagatesSelectedAudioTrack(t *testing.T) {
sessionMgr := playback.NewSessionManager(0, 0)
filePath := writePlaybackTestMediaFile(t, "movie.mkv")
@@ -1213,7 +1452,7 @@ func TestHandleStartTranscode_LocalPathPropagatesSelectedAudioTrack(t *testing.T
t.Fatalf("transcode status = %d, body = %s", transcodeRR.Code, transcodeRR.Body.String())
}
transcodeSession := handler.getTranscodeSession(startResp.SessionID)
transcodeSession := handler.tm.GetTranscodeSession(startResp.SessionID)
if transcodeSession == nil {
t.Fatal("expected local transcode session")
}
@@ -1264,7 +1503,7 @@ func TestHandleStartTranscode_MPEG2SeekedCopyRemainsCopyVideo(t *testing.T) {
t.Fatalf("transcode status = %d, body = %s", transcodeRR.Code, transcodeRR.Body.String())
}
transcodeSession := handler.getTranscodeSession(session.ID)
transcodeSession := handler.tm.GetTranscodeSession(session.ID)
if transcodeSession == nil {
t.Fatal("expected local transcode session")
}
+13
View File
@@ -28,6 +28,12 @@ type ProfileHandler struct {
ProfileTokens *access.ProfileTokenService
AvatarStore profileAvatarStore
AvatarTTL time.Duration
// DeviceLibraryPurger removes a deleted profile's device rows (and, via
// cascade, its managed downloads and subscriptions). Profiles may live
// outside Postgres, so no FK cascade covers these shared tables.
DeviceLibraryPurger interface {
PurgeProfileDevices(ctx context.Context, userID int, profileID string) error
}
}
// NewProfileHandler creates a new ProfileHandler.
@@ -589,6 +595,13 @@ func (h *ProfileHandler) HandleDeleteProfile(w http.ResponseWriter, r *http.Requ
slog.Warn("profile avatar cleanup failed after delete", "user_id", userID, "profile_id", profileID, "error", cleanupErr)
}
}
if h.DeviceLibraryPurger != nil {
purgeCtx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), 30*time.Second)
defer cancel()
if purgeErr := h.DeviceLibraryPurger.PurgeProfileDevices(purgeCtx, userID, profileID); purgeErr != nil {
slog.Warn("profile device-library purge failed after delete", "user_id", userID, "profile_id", profileID, "error", purgeErr)
}
}
w.WriteHeader(http.StatusNoContent)
}
+73 -4
View File
@@ -3,8 +3,10 @@ package handlers
import (
"context"
"encoding/json"
"log/slog"
"net/http"
"strconv"
"time"
"github.com/Silo-Server/silo-server/internal/access"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
@@ -12,6 +14,34 @@ import (
"github.com/Silo-Server/silo-server/internal/userstore"
)
// progressClockSkew bounds how far ahead of server time a client-supplied
// progress event time may sit before it is clamped to "now".
const progressClockSkew = 2 * time.Minute
// parseClientEventTime parses an RFC3339 client event time. Malformed values
// are an error the caller must reject: treating them as "now" would let a
// stale offline event win LWW as a fresh server-time write.
func parseClientEventTime(s string) (time.Time, error) {
t, err := time.Parse(time.RFC3339, s)
if err != nil {
return time.Time{}, err
}
return t.UTC(), nil
}
// clampEventAt bounds a client event time to at most now+skew: a value past the
// window is clamped to now, so a skewed or malicious clock can at most claim
// "now" for its own profile and never lock in a far-future LWW win (invariant 1).
func clampEventAt(client, now time.Time) time.Time {
if client.IsZero() {
return now
}
if client.After(now.Add(progressClockSkew)) {
return now
}
return client
}
// ProgressLibraryLookup resolves which progress items belong to a library.
type ProgressLibraryLookup interface {
GetItemsInFolder(ctx context.Context, contentIDs []string, folderID int) (map[string]bool, error)
@@ -57,6 +87,8 @@ type progressEntryResponse struct {
type progressListResponse struct {
Progress []progressEntryResponse `json:"progress"`
// NextCursor is the opaque server token to resume a ?since= delta from.
NextCursor string `json:"next_cursor,omitempty"`
}
type syncProgressItem struct {
@@ -64,6 +96,9 @@ type syncProgressItem struct {
Position float64 `json:"position"`
Duration float64 `json:"duration"`
ForceOverwrite bool `json:"force_overwrite"`
// UpdatedAt is the client EVENT time (RFC3339) for an offline-queued item.
// The server clamps it to now+skew and uses it only as the LWW key.
UpdatedAt *string `json:"updated_at,omitempty"`
}
type syncProgressRequest struct {
@@ -94,6 +129,7 @@ func (h *ProgressHandler) HandleListProgress(w http.ResponseWriter, r *http.Requ
}
status := r.URL.Query().Get("status")
since := r.URL.Query().Get("since")
limit, offset := parsePagination(r)
libraryID, err := parseLibraryIDParam(r)
if err != nil {
@@ -101,7 +137,16 @@ func (h *ProgressHandler) HandleListProgress(w http.ResponseWriter, r *http.Requ
return
}
entries, err := store.ListProgress(r.Context(), profileID, status, limit, offset)
// A ?since= cursor switches to server-ordered delta delivery (rows changed
// elsewhere since the cursor), immune to client clock skew. Absent since →
// today's status/pagination listing.
var entries []userstore.WatchProgress
var nextCursor string
if since != "" {
entries, nextCursor, err = store.ListProgressSince(r.Context(), profileID, since)
} else {
entries, err = store.ListProgress(r.Context(), profileID, status, limit, offset)
}
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list progress")
return
@@ -138,7 +183,8 @@ func (h *ProgressHandler) HandleListProgress(w http.ResponseWriter, r *http.Requ
}
resp := progressListResponse{
Progress: make([]progressEntryResponse, 0, len(entries)),
Progress: make([]progressEntryResponse, 0, len(entries)),
NextCursor: nextCursor,
}
for _, e := range entries {
resp.Progress = append(resp.Progress, progressEntryResponse{
@@ -280,9 +326,32 @@ func (h *ProgressHandler) HandleSyncProgress(w http.ResponseWriter, r *http.Requ
}
var updateErr error
if item.ForceOverwrite {
switch {
case item.UpdatedAt != nil:
// Offline-queued event: clamp the client event time and merge
// last-write-wins on the bounded event_at. synced_seq (the cursor) is
// stamped server-side; completion still comes from the threshold logic,
// never the timestamp alone.
client, parseErr := parseClientEventTime(*item.UpdatedAt)
if parseErr != nil {
result.Status = "error"
result.Error = "updated_at must be RFC3339"
results = append(results, result)
continue
}
now := time.Now()
eventAt := clampEventAt(client, now)
if !client.IsZero() && client.After(now.Add(progressClockSkew)) {
slog.Warn("clamped future-dated progress event time",
"profile_id", profileID, "media_item_id", item.MediaItemID)
}
pos, completed, skip := userstore.ResolveProgressState(item.Position, item.Duration, thresholds)
if !skip {
_, updateErr = store.SetProgressIfNewer(r.Context(), profileID, item.MediaItemID, pos, item.Duration, completed, eventAt)
}
case item.ForceOverwrite:
updateErr = store.SetProgress(r.Context(), profileID, item.MediaItemID, item.Position, item.Duration, thresholds)
} else {
default:
updateErr = store.UpdateProgress(r.Context(), profileID, item.MediaItemID, item.Position, item.Duration, thresholds)
}
@@ -0,0 +1,50 @@
package handlers
import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/userstore"
)
func TestClampEventAt(t *testing.T) {
now := time.Date(2026, 6, 19, 12, 0, 0, 0, time.UTC)
cases := []struct {
name string
client time.Time
want time.Time
}{
{"zero defaults to now", time.Time{}, now},
{"far future clamps to now", now.Add(48 * time.Hour), now},
{"just past skew clamps to now", now.Add(progressClockSkew + time.Minute), now},
{"within skew is kept", now.Add(time.Minute), now.Add(time.Minute)},
{"past is kept", now.Add(-3 * time.Hour), now.Add(-3 * time.Hour)},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := clampEventAt(tc.client, now); !got.Equal(tc.want) {
t.Fatalf("clampEventAt(%v) = %v, want %v", tc.client, got, tc.want)
}
})
}
}
func TestOfflineProgressState(t *testing.T) {
thresholds := userstore.ProgressThresholds{WatchedPct: 90, MinResumePct: 5}
// Below the min-resume floor → skipped.
if _, _, skip := userstore.ResolveProgressState(10, 1000, thresholds); !skip {
t.Fatal("tiny progress should be skipped")
}
// Above the watched threshold → completed latch, position reset.
pos, completed, skip := userstore.ResolveProgressState(950, 1000, thresholds)
if skip || !completed || pos != 0 {
t.Fatalf("watched item = (pos=%v completed=%v skip=%v), want (0 true false)", pos, completed, skip)
}
// Normal mid-progress → kept as-is.
pos, completed, skip = userstore.ResolveProgressState(500, 1000, thresholds)
if skip || completed || pos != 500 {
t.Fatalf("mid progress = (pos=%v completed=%v skip=%v), want (500 false false)", pos, completed, skip)
}
}
+1 -1
View File
@@ -182,7 +182,7 @@ func (h *PlaybackHandler) handleRealtimeClientMessage(sessionID string, data []b
}
switch record.Name {
case playback.CommandStop, playback.CommandTerminate:
err := h.stopPlaybackSessionByID(context.Background(), sessionID)
err := h.stopPlaybackSessionByID(context.Background(), sessionID, true)
if err != nil && !errors.Is(err, playback.ErrSessionNotFound) {
slog.Error("failed to stop playback after realtime completion", "session", sessionID, "playback_session_id", sessionID, "error", err)
}
+67 -34
View File
@@ -32,6 +32,15 @@ type StreamHandler struct {
EventsHub *evt.Hub
AdminStore PlaybackAdminStore
SessionSyncer PlaybackSessionSyncer
// TM is the shared transcode/reconstruct manager (same instance as the
// PlaybackHandler's). It lets a direct/remux stream rebuild its playback
// Session from the recipe card after a server restart instead of 404-ing.
// May be nil (tests / minimal setups) — reconstruct is then simply off.
TM *playback.TranscodeManager
// JWTSecret verifies the stream token carried on the serve URL (?st=), which
// is the reconstruction descriptor for direct/remux after a restart. Empty
// disables token-based reconstruct (tests / minimal setups).
JWTSecret string
// PlaybackConfig returns the current playback config; read it through
// ffmpegPath(). May be nil (tests).
PlaybackConfig func() config.PlaybackConfig
@@ -54,6 +63,10 @@ func NewStreamHandler(sessionMgr SessionManagerInterface, fileResolver FilePathR
return &StreamHandler{
sessionMgr: sessionMgr,
fileResolver: fileResolver,
// A bare manager (no recipe store) behaves as "no reconstruct" — plain
// GetSession + ownership — so HandleStream has a single code path. The
// router overwrites this with the shared manager to enable reconstruct.
TM: playback.NewTranscodeManager(),
}
}
@@ -75,14 +88,22 @@ func (h *StreamHandler) HandleStream(w http.ResponseWriter, r *http.Request) {
}
setPlaybackSessionLogContext(r, sessionID)
session, err := h.sessionMgr.GetSession(sessionID)
if err != nil {
// Look up the session, reconstructing it from the recipe card on a not-found
// miss (e.g. after a server restart) so a direct/remux stream resumes instead
// of 404-ing. The client re-supplies its position (HTTP Range for direct, the
// ?seek= query for remux), so no runtime beyond the Session needs rebuilding.
// Without a token (or signing secret) reconstruct is off, collapsing to a
// plain GetSession + ownership check.
card := streamCardFromToken(r.URL.Query().Get(streamTokenParam), sessionID, h.JWTSecret)
session, status := h.TM.LoadOrReconstructSession(r.Context(), h.sessionMgr.GetSession, sessionID, userID, card)
switch status {
case playback.SessionMissing:
writePlaybackSessionNotFound(w)
return
}
// Verify session ownership.
if session.UserID != userID {
case playback.SessionLoadFailed:
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load playback session")
return
case playback.SessionForbidden:
writeError(w, http.StatusForbidden, "forbidden", "Session belongs to another user")
return
}
@@ -240,37 +261,49 @@ func (h *StreamHandler) HandleSubtitle(w http.ResponseWriter, r *http.Request) {
// Check downloaded subtitles (from S3).
if h.SubtitleRepo != nil && h.S3Client != nil {
downloaded, err := h.SubtitleRepo.ListDownloadedSubtitles(r.Context(), file.ID)
if err == nil {
downloadedIndex := embeddedIndex - len(file.SubtitleTracks)
if downloadedIndex >= 0 && downloadedIndex < len(downloaded) {
dl := downloaded[downloadedIndex]
data, err := h.S3Client.GetObject(r.Context(), h.S3Bucket, dl.S3Key)
if err != nil {
writeError(w, http.StatusBadGateway, "s3_error", "Failed to load subtitle from storage")
return
}
if err != nil {
// A DB failure here must not masquerade as "track not found":
// surface it as an internal error (with a server-side signal)
// so the real failure is diagnosable instead of looking like an
// intermittent 404 to the client.
slog.ErrorContext(r.Context(), "list downloaded subtitles failed",
"file_id", file.ID,
"track", trackIndex,
"error", err,
)
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list downloaded subtitles")
return
}
// Serve ASS/SSA downloaded subtitles as raw data.
if playback.IsASS(string(dl.Format)) {
playback.ServeSubtitle(w, data, "ass")
return
}
// If the subtitle is already VTT, serve directly.
if dl.Format == subtitles.FormatVTT {
playback.ServeSubtitle(w, data, "vtt")
return
}
// Convert to VTT using the playback conversion pipeline.
vttData, err := playback.ConvertToVTT(data, string(dl.Format))
if err != nil {
writeError(w, http.StatusInternalServerError, "convert_error", "Failed to convert subtitle")
return
}
playback.ServeSubtitle(w, vttData, "vtt")
downloadedIndex := embeddedIndex - len(file.SubtitleTracks)
if downloadedIndex >= 0 && downloadedIndex < len(downloaded) {
dl := downloaded[downloadedIndex]
data, err := h.S3Client.GetObject(r.Context(), h.S3Bucket, dl.S3Key)
if err != nil {
writeError(w, http.StatusBadGateway, "s3_error", "Failed to load subtitle from storage")
return
}
// Serve ASS/SSA downloaded subtitles as raw data.
if playback.IsASS(string(dl.Format)) {
playback.ServeSubtitle(w, data, "ass")
return
}
// If the subtitle is already VTT, serve directly.
if dl.Format == subtitles.FormatVTT {
playback.ServeSubtitle(w, data, "vtt")
return
}
// Convert to VTT using the playback conversion pipeline.
vttData, err := playback.ConvertToVTT(data, string(dl.Format))
if err != nil {
writeError(w, http.StatusInternalServerError, "convert_error", "Failed to convert subtitle")
return
}
playback.ServeSubtitle(w, vttData, "vtt")
return
}
}
+52
View File
@@ -2,12 +2,15 @@ package handlers
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/go-chi/chi/v5"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
@@ -121,6 +124,55 @@ func TestHandleStream_KeepsSessionWhenLookupFailsForNonMissingReason(t *testing.
}
}
// TestHandleSubtitle_ListDownloadedSubtitlesErrorReturns500 pins the fix for
// issue #248: a failure listing downloaded subtitles must surface as a 500 with
// an "internal_error" code, not be swallowed and reported to the client as a
// generic "Subtitle track not found" 404 (which made a real backing-store
// failure look like an intermittent client-side subtitle bug).
func TestHandleSubtitle_ListDownloadedSubtitlesErrorReturns500(t *testing.T) {
// No external or embedded tracks, so track index 0 falls through to the
// downloaded-subtitle branch that queries the repository.
file := &models.MediaFile{
ID: 42,
ContentID: "movie-1",
FilePath: "/tmp/movie.mkv",
Duration: 3600,
}
baseMgr := playback.NewSessionManager(0, 0)
session, err := baseMgr.StartSession(1, "profile-1", 42, playback.PlayDirect, false)
if err != nil {
t.Fatalf("StartSession: %v", err)
}
handler := NewStreamHandler(baseMgr, testPlaybackFileResolver{file: file})
handler.SubtitleRepo = &handlerMockSubtitleRepo{listErr: errors.New("db unavailable")}
handler.S3Client = newMockS3ClientForHandler()
handler.S3Bucket = "test-bucket"
req := httptest.NewRequest(http.MethodGet, "/api/v1/stream/"+session.ID+"/subtitles/0.vtt", nil)
req = req.WithContext(newAuthorizedPlaybackContext())
routeCtx := chi.NewRouteContext()
routeCtx.URLParams.Add("session_id", session.ID)
routeCtx.URLParams.Add("track", "0.vtt")
req = req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, routeCtx))
rr := httptest.NewRecorder()
handler.HandleSubtitle(rr, req)
if rr.Code != http.StatusInternalServerError {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
var body struct {
Error string `json:"error"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &body); err != nil {
t.Fatalf("decode error body: %v (body = %s)", err, rr.Body.String())
}
if body.Error != "internal_error" {
t.Fatalf("error code = %q, want %q (body = %s)", body.Error, "internal_error", rr.Body.String())
}
}
func TestHandleTransportStartFailure_KeepsSessionForNonMissingError(t *testing.T) {
filePath := writePlaybackTestMediaFile(t, "movie.mkv")
file := &models.MediaFile{
@@ -239,6 +239,9 @@ type handlerMockSubtitleRepo struct {
subtitles map[int]*subtitles.DownloadedSubtitle
nextID int
byKey map[string]*subtitles.DownloadedSubtitle
// listErr, when set, is returned by ListDownloadedSubtitles to simulate a
// backing-store failure.
listErr error
}
func newMockSubtitleRepoForHandler() *handlerMockSubtitleRepo {
@@ -265,6 +268,9 @@ func (m *handlerMockSubtitleRepo) GetDownloadedSubtitle(_ context.Context, id in
}
func (m *handlerMockSubtitleRepo) ListDownloadedSubtitles(context.Context, int) ([]subtitles.DownloadedSubtitle, error) {
if m.listErr != nil {
return nil, m.listErr
}
return nil, nil
}
+58 -15
View File
@@ -7,6 +7,7 @@ import (
"io"
"net/http"
"sync"
"sync/atomic"
"time"
"github.com/Silo-Server/silo-server/internal/access"
@@ -110,9 +111,16 @@ type watchTogetherPingMessage struct {
ClientSentAt string `json:"client_sent_at"`
}
// watchTogetherRoomConn serializes every write to the underlying gorilla
// connection. gorilla/websocket does not support concurrent writers, and room
// broadcasts arrive from other members' goroutines, so all writes — including
// pong/error replies from the read loop — must go through this wrapper.
type watchTogetherRoomConn struct {
conn *websocket.Conn
writeMu sync.Mutex
// pingSentAtNano is the send time of the most recent protocol-level ping,
// used to measure round-trip latency when the pong arrives.
pingSentAtNano atomic.Int64
}
func (c *watchTogetherRoomConn) WriteJSON(v any) error {
@@ -130,9 +138,35 @@ func (c *watchTogetherRoomConn) Close() error {
func (c *watchTogetherRoomConn) WritePing() error {
c.writeMu.Lock()
defer c.writeMu.Unlock()
c.pingSentAtNano.Store(time.Now().UnixNano())
return writeWebSocketControl(c.conn, websocket.PingMessage, nil)
}
// TakePingSentAt returns and clears the send time of the last unanswered
// protocol-level ping.
func (c *watchTogetherRoomConn) TakePingSentAt() time.Time {
nano := c.pingSentAtNano.Swap(0)
if nano == 0 {
return time.Time{}
}
return time.Unix(0, nano)
}
func (c *watchTogetherRoomConn) WriteError(code, message string) {
_ = c.WriteJSON(map[string]string{
"type": "error",
"code": code,
"message": message,
})
}
func (c *watchTogetherRoomConn) writeRoomClosed(reason string) {
_ = c.WriteJSON(map[string]string{
"type": "room_closed",
"reason": reason,
})
}
func NewWatchTogetherHandler(
service *watchtogether.Service,
scopeResolver WatchTogetherScopeResolver,
@@ -653,19 +687,33 @@ func (h *WatchTogetherHandler) HandleRoomWebSocket(w http.ResponseWriter, r *htt
reg, snapshot, err := h.Service.Connect(ctx, roomID, claims.UserID, profileID, realtimeConn)
if err != nil {
// Terminal failures use room_closed so clients stop reconnecting
// instead of retrying a room that will never come back.
if errors.Is(err, watchtogether.ErrRoomNotFound) {
writeWebSocketError(conn, "not_found", "Room not found")
realtimeConn.writeRoomClosed("not_found")
} else if errors.Is(err, watchtogether.ErrRoomClosed) {
writeWebSocketError(conn, "gone", "Room is no longer active")
realtimeConn.writeRoomClosed("ended")
} else {
writeWebSocketError(conn, "internal_error", "Failed to connect room socket")
realtimeConn.WriteError("internal_error", "Failed to connect room socket")
}
return
}
defer h.Service.Disconnect(reg, false)
configureWebSocket(conn)
// Measure round-trip latency from protocol-level ping/pong on the server
// clock; client-reported timestamps are subject to clock skew and cannot
// be trusted for command scheduling.
conn.SetPongHandler(func(string) error {
_ = conn.SetReadDeadline(time.Now().Add(wsPingInterval + wsPongTimeout))
if sentAt := realtimeConn.TakePingSentAt(); !sentAt.IsZero() {
_ = h.Service.HandlePingForConnection(ctx, reg, claims.UserID, profileID, time.Since(sentAt).Milliseconds())
}
return nil
})
startWebSocketPingLoop(ctx, realtimeConn.WritePing)
// Prime an RTT sample right away instead of waiting for the first tick.
_ = realtimeConn.WritePing()
if err := realtimeConn.WriteJSON(map[string]any{
"type": "snapshot",
@@ -680,15 +728,15 @@ func (h *WatchTogetherHandler) HandleRoomWebSocket(w http.ResponseWriter, r *htt
return
}
if err := h.handleRoomClientMessage(ctx, conn, reg, claims.UserID, profileID, data); err != nil {
writeWebSocketError(conn, "bad_request", err.Error())
if err := h.handleRoomClientMessage(ctx, realtimeConn, reg, claims.UserID, profileID, data); err != nil {
realtimeConn.WriteError("bad_request", err.Error())
}
}
}
func (h *WatchTogetherHandler) handleRoomClientMessage(
ctx context.Context,
conn *websocket.Conn,
rc *watchTogetherRoomConn,
reg *watchtogether.Registration,
userID int,
profileID string,
@@ -768,16 +816,11 @@ func (h *WatchTogetherHandler) handleRoomClientMessage(
if err := json.Unmarshal(data, &msg); err != nil {
return err
}
clientSentAt, err := time.Parse(time.RFC3339Nano, msg.ClientSentAt)
if err != nil {
return errors.New("client_sent_at must be RFC3339Nano")
}
// The echoed timestamps only serve the client's clock-offset estimate;
// latency for command scheduling is measured server-side from
// protocol-level ping/pong.
now := time.Now().UTC()
pingMS := now.Sub(clientSentAt.UTC()).Milliseconds()
if pingMS > 0 {
_ = h.Service.HandlePingForConnection(ctx, reg, userID, profileID, pingMS)
}
return writeWebSocketJSON(conn, map[string]string{
return rc.WriteJSON(map[string]string{
"type": "pong",
"client_sent_at": msg.ClientSentAt,
"server_received_at": now.Format(time.RFC3339Nano),
+73 -15
View File
@@ -32,7 +32,7 @@ import (
"github.com/Silo-Server/silo-server/internal/catalogseed"
"github.com/Silo-Server/silo-server/internal/clientip"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/download"
"github.com/Silo-Server/silo-server/internal/downloads"
evt "github.com/Silo-Server/silo-server/internal/events"
"github.com/Silo-Server/silo-server/internal/historyimport"
"github.com/Silo-Server/silo-server/internal/intromarkers"
@@ -132,8 +132,9 @@ type Dependencies struct {
ActivityLogRepo *activitylog.Repo
OpsLogRepo *opslog.Repo
FFmpegLogSink playback.FFmpegLogSink
RedisClient *redis.Client // for session listing (may be nil)
TaskManager *taskmanager.TaskManager // task manager (may be nil)
RedisClient *redis.Client // for session listing (may be nil)
TaskManager *taskmanager.TaskManager // task manager (may be nil)
ArtifactManager *downloads.ArtifactManager // download prepare-to-file pipeline (may be nil)
AdminJobCancelRegistry *adminjob.CancelRegistry
IntroRepository *intromarkers.Repository
IntroAnalyzer *intromarkers.Analyzer
@@ -449,14 +450,10 @@ func NewRouter(deps Dependencies) chi.Router {
ebookAnnotationStore = handlers.NewPGEbookReaderAnnotationStore(deps.DB)
browseRepo := catalog.NewBrowseRepository(deps.DB)
itemRepo = catalog.NewItemRepository(deps.DB)
catalogSearchSettings, err := catalog.LoadCatalogSearchSettings(context.Background(), settingsRepo)
if err != nil {
slog.Warn("catalog search: failed to load settings; using postgres", "err", err)
catalogSearchSettings = catalog.DefaultCatalogSearchSettings()
}
searchIndexEvents := catalog.NewSearchIndexEventRepository(deps.DB)
catalogSearchService = catalog.NewCatalogSearchServiceFromSettings(
catalogSearchSettings,
catalogSearchService = catalog.NewCatalogSearchService(
context.Background(),
settingsRepo,
itemRepo,
searchIndexEvents,
deps.CatalogSearchVectorizer,
@@ -751,6 +748,13 @@ func NewRouter(deps Dependencies) chi.Router {
playbackHandler.SessionSyncer = deps.SessionSyncer
}
if streamHandler != nil {
// Share the playback handler's transcode/reconstruct manager so a
// direct/remux stream can rebuild its session from the token recipe
// after a restart (same manager, same SessionManager).
streamHandler.TM = playbackHandler.TranscodeManager()
if deps.Config != nil {
streamHandler.JWTSecret = deps.Config.Auth.JWTSecret
}
streamHandler.AdminStore = playbackAdminStore
streamHandler.EventsHub = deps.EventsHub
streamHandler.SessionSyncer = deps.SessionSyncer
@@ -819,9 +823,9 @@ func NewRouter(deps Dependencies) chi.Router {
watchtogether.NewRepository(deps.DB),
deps.SessionMgr,
deps.FileRepo,
playbackHandler.CommandDispatcher,
watchtogether.NewCatalogSelectionResolver(detailSvc),
watchtogether.NewSuggestionRepository(deps.DB),
watchtogether.NewProfileNameResolver(deps.UserStoreProvider),
),
viewerResolver,
roomTokenService,
@@ -1353,18 +1357,18 @@ func NewRouter(deps Dependencies) chi.Router {
// Build download handler.
var downloadHandler *handlers.DownloadHandler
if deps.DB != nil && deps.FileRepo != nil && deps.Config != nil {
downloadRepo := download.NewRepository(deps.DB)
downloadBandwidth := download.NewBandwidthManager(
downloadRepo := downloads.NewRepository(deps.DB)
downloadBandwidth := downloads.NewBandwidthManager(
deps.Config.Download.ServerBandwidthBPS,
deps.Config.Download.UserBandwidthBPS,
)
downloadLimiter := download.NewQuantityLimiter(
downloadLimiter := downloads.NewQuantityLimiter(
downloadRepo,
deps.Config.Download.MaxConcurrentPerUser,
deps.Config.Download.MaxPerPeriod,
deps.Config.Download.PeriodDuration,
)
downloadSvc := download.NewService(
downloadSvc := downloads.NewService(
downloadRepo,
downloadBandwidth,
downloadLimiter,
@@ -1376,7 +1380,31 @@ func NewRouter(deps Dependencies) chi.Router {
settingsRepo,
&deps.Config.Download,
)
if detailSvc != nil {
// Offline manifest + artwork/subtitle proxies (Phase 2). subtitleManager
// may be nil when subtitles are unconfigured; pass a nil interface so the
// downloaded-subtitle path reports unavailable instead of panicking.
var subtitleSource downloads.SubtitleSource
if subtitleManager != nil {
subtitleSource = subtitleManager
}
downloadSvc.SetOfflineDeps(detailSvc, subtitleSource, nil)
}
if deps.ArtifactManager != nil {
// Prepare-to-file pipeline (Phase 3): remux/transcode-to-single-file.
downloadSvc.SetArtifactManager(deps.ArtifactManager)
}
// Series monitoring (auto-download subscriptions). Client-pull only:
// devices sync on app open / background refresh; there is no server
// background worker.
downloadSvc.SetSubscriptions(downloads.NewSubscriptionRepository(deps.DB))
downloadHandler = handlers.NewDownloadHandler(downloadSvc)
if profileHandler != nil {
// Profiles may live outside Postgres (sqlite userdb backend), so
// deleting one cannot FK-cascade the shared user_devices table;
// purge the device library (and its downloads) in-app instead.
profileHandler.DeviceLibraryPurger = downloadRepo
}
} else {
downloadHandler = handlers.NewDownloadHandler(nil)
}
@@ -1630,6 +1658,7 @@ func NewRouter(deps Dependencies) chi.Router {
}
notificationsHandler := handlers.NewNotificationsHandler(deps.Notifications, deps.EventsHub)
r.With(apimw.RequireProfile).Post("/events/ws-ticket", notificationsHandler.HandleMintWSTicket)
r.With(apimw.RequireProfile).Post("/devices/push/apple", notificationsHandler.HandleRegisterApplePushDevice)
// Discord DM channel: the linked identity and mode hang off
// the login account, not a profile, so these stay outside
// the RequireProfile subrouter below (static paths coexist
@@ -1648,6 +1677,7 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/capability", notificationsHandler.HandleCapability)
r.Get("/preferences", notificationsHandler.HandleGetPreferences)
r.Put("/preferences", notificationsHandler.HandleUpdatePreferences)
r.Get("/push/apple/display/{delivery_id}", notificationsHandler.HandleApplePushDisplay)
r.Get("/email-preferences", notificationsHandler.HandleGetEmailPreferences)
r.Put("/email-preferences", notificationsHandler.HandleUpdateEmailPreferences)
r.Put("/email-preferences/address", notificationsHandler.HandleRequestEmailAddress)
@@ -2051,6 +2081,8 @@ func NewRouter(deps Dependencies) chi.Router {
r.Get("/metadata/ai/status", metadataAIHandler.HandleStatus)
if itemRepo != nil {
metadataAIHandler.ItemAccess = itemRepo
metadataAIHandler.SeasonLookup = seasonRepo
metadataAIHandler.EpisodeLookup = episodeRepo
r.Post("/items/{id}/translate-description", metadataAIHandler.HandleTranslateOnView)
}
} else {
@@ -2153,10 +2185,27 @@ func NewRouter(deps Dependencies) chi.Router {
// Download routes.
r.Route("/downloads", func(r chi.Router) {
r.Get("/capability", downloadHandler.HandleCapability)
r.Post("/", downloadHandler.HandleCreateDownload)
r.Get("/", downloadHandler.HandleListDownloads)
// Series monitoring (auto-download) subscriptions.
r.Post("/subscriptions", downloadHandler.HandleCreateSubscription)
r.Post("/subscriptions/sync", downloadHandler.HandleSyncSubscriptions)
r.Get("/subscriptions", downloadHandler.HandleListSubscriptions)
r.Get("/subscriptions/{id}", downloadHandler.HandleGetSubscription)
r.Patch("/subscriptions/{id}", downloadHandler.HandlePatchSubscription)
r.Delete("/subscriptions/{id}", downloadHandler.HandleDeleteSubscription)
r.Get("/batches/{batch_id}/manifests", downloadHandler.HandleBatchManifests)
r.Patch("/{id}", downloadHandler.HandlePatchDownload)
r.Delete("/{id}", downloadHandler.HandleDeleteDownload)
// GET+HEAD: background download stacks probe with HEAD
// before issuing ranged GETs; http.ServeContent handles
// HEAD natively.
r.Get("/{id}/file", downloadHandler.HandleDownloadFile)
r.Head("/{id}/file", downloadHandler.HandleDownloadFile)
r.Get("/{id}/manifest", downloadHandler.HandleManifest)
r.Get("/{id}/artwork/{kind}", downloadHandler.HandleArtwork)
r.Get("/{id}/subtitles/{ref}", downloadHandler.HandleSubtitle)
})
r.Get("/direct-download", downloadHandler.HandleDirectDownload)
r.Head("/direct-download", downloadHandler.HandleDirectDownload)
@@ -2305,6 +2354,15 @@ func NewRouter(deps Dependencies) chi.Router {
if discordNotificationsHandler != nil {
r.Post("/notifications/discord/test", discordNotificationsHandler.HandleAdminTest)
}
if deps.Notifications != nil || settingsRepo != nil {
applePushHandler := handlers.NewAdminApplePushHandler(deps.Notifications, settingsRepo)
if deps.Notifications != nil {
r.Post("/notifications/push/apple/test", applePushHandler.HandleTest)
}
if settingsRepo != nil {
r.Post("/notifications/push/relay/register", applePushHandler.HandleRegisterRelay)
}
}
if deps.Notifications != nil && deps.Notifications.ServerChannels != nil {
serverChannelsHandler := handlers.NewAdminServerChannelsHandler(deps.Notifications)
r.Route("/notifications/server-channels", func(r chi.Router) {
+151
View File
@@ -124,6 +124,157 @@ func (r *BrowseRepository) browse(ctx context.Context, filters BrowseFilters, in
}, nil
}
// BrowseRecentlyAddedAcrossLibraries serves a recently_added browse spanning
// multiple libraries by running the proven single-library index path once per
// library and merging the already-sorted results in memory. Each per-library
// query filters on exactly one media_folder_id, so it takes the
// singleLibraryNoDedup fast path and walks idx_item_libraries_folder_seen_content
// as a top-N index scan (~1ms). N small walks replace one multi-library
// MIN(first_seen_at) + GROUP BY scan of the whole catalog (HashAggregate over
// ~180k groups + top-N heapsort, ~2s with a wide projection and cold cache).
//
// It only serves offset 0 — the /Items/Latest hot path. A per-library top-Limit
// fetch cannot satisfy a deep global offset, so callers must route offset>0
// pages to BrowsePage; passing a positive Offset here is an error rather than a
// silently-wrong page.
//
// Cross-library dedup is performed app-side: a content_id present in more than
// one library keeps the EARLIEST first_seen_at, preserving the
// MIN(first_seen_at) "added at" semantics of the multi-library GROUP BY path.
// The reported Total is a lower bound (exact when the page is the whole result),
// chosen so HasMore remains correct for offset 0.
//
// Known limitation vs the exact GROUP BY query, bounded to content_ids that
// live in 2+ libraries: such a dup is only deduped against the slices actually
// fetched, so if its true MIN(first_seen_at) library does not rank it in that
// library's top-Limit, the merge sees only a later first_seen_at and may rank it
// too recent (or drop it if it is outside top-Limit in every library holding it).
// This is acceptable for an approximate "recently added" rail; do not reuse this
// path where exact MIN ranking across libraries is required.
func (r *BrowseRepository) BrowseRecentlyAddedAcrossLibraries(ctx context.Context, base BrowseFilters, libraryIDs []int) (*BrowseResult, error) {
if base.Offset > 0 {
return nil, fmt.Errorf("browse recently added across libraries: offset %d unsupported, use BrowsePage", base.Offset)
}
limit := base.Limit
if limit <= 0 {
limit = 20
}
// Each per-library subquery is clamped to MaxLimit by BrowsePage, but the
// merged page is trimmed to this `limit`; clamp it too so the final result
// honors the same BrowseFilters.MaxLimit contract as the single-library path.
if base.MaxLimit > 0 && limit > base.MaxLimit {
limit = base.MaxLimit
}
perLibrary := make([][]*models.MediaItem, 0, len(libraryIDs))
anyLibraryHasMore := false
for _, id := range libraryIDs {
sub := base
sub.LibraryID = id
// Clearing the multi-library scopes is what selects the
// singleLibraryNoDedup fast path in buildBrowsePlan.
sub.LibraryIDs = nil
sub.DisabledLibraryIDs = nil
sub.Offset = 0
sub.Limit = limit
res, err := r.BrowsePage(ctx, sub, false)
if err != nil {
return nil, fmt.Errorf("browse recently added in library %d: %w", id, err)
}
if res == nil {
continue
}
perLibrary = append(perLibrary, res.Items)
if res.HasMore {
anyLibraryHasMore = true
}
}
merged := mergeRecentlyAddedItems(perLibrary)
// More results exist when the merged set already overflows the page, or when
// any single library still has rows behind its own top-Limit window (which
// implies that library alone holds more than `limit` items).
hasMore := anyLibraryHasMore || len(merged) > limit
total := len(merged)
if hasMore && total <= limit {
// A library has rows beyond the page but the merged set fit exactly;
// nudge the total so offset-0 HasMore stays true for total-driven callers.
total = limit + 1
}
if len(merged) > limit {
merged = merged[:limit]
}
return &BrowseResult{Items: merged, Total: total, HasMore: hasMore}, nil
}
// mergeRecentlyAddedItems merges per-library recently_added result slices — each
// already sorted by first_seen_at DESC, content_id ASC — into a single globally
// ordered slice. Duplicate content_ids are collapsed to one entry whose AddedAt
// is the EARLIEST across libraries, mirroring MIN(first_seen_at) from the
// multi-library GROUP BY path. The result is sorted by AddedAt DESC, content_id
// ASC and is NOT truncated; callers trim to the page size and derive HasMore.
func mergeRecentlyAddedItems(perLibrary [][]*models.MediaItem) []*models.MediaItem {
byID := make(map[string]*models.MediaItem)
order := make([]*models.MediaItem, 0)
for _, items := range perLibrary {
for _, it := range items {
if it == nil {
continue
}
existing, ok := byID[it.ContentID]
if !ok {
byID[it.ContentID] = it
order = append(order, it)
continue
}
// Keep the earliest first_seen_at to preserve MIN semantics.
if addedAtBefore(it.AddedAt, existing.AddedAt) {
existing.AddedAt = it.AddedAt
}
}
}
// Re-sort after dedup because collapsing to MIN(first_seen_at) can move an
// item earlier than its position in any single per-library slice.
slices.SortStableFunc(order, compareRecentlyAdded)
return order
}
// compareRecentlyAdded orders items by AddedAt DESC, then content_id ASC,
// matching "ORDER BY mil.first_seen_at DESC, mi.content_id ASC". A nil AddedAt
// sorts last (treated as the oldest).
func compareRecentlyAdded(a, b *models.MediaItem) int {
at, bt := a.AddedAt, b.AddedAt
switch {
case at == nil && bt == nil:
// fall through to the content_id tie-break
case at == nil:
return 1
case bt == nil:
return -1
default:
if at.After(*bt) {
return -1
}
if at.Before(*bt) {
return 1
}
}
return strings.Compare(a.ContentID, b.ContentID)
}
// addedAtBefore reports whether a is strictly earlier than b, treating nil as
// the oldest possible value so a real timestamp always wins the MIN.
func addedAtBefore(a, b *time.Time) bool {
if a == nil {
return b != nil
}
if b == nil {
return false
}
return a.Before(*b)
}
// browseQueryPlan captures the rendered SQL fragments and bound args for a
// browse data query. It is produced by buildBrowsePlan and consumed both by
// browse() (to execute) and by tests (to inspect the emitted SQL without a
@@ -0,0 +1,108 @@
package catalog
import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/models"
)
func tsPtr(t time.Time) *time.Time { return &t }
func contentIDs(items []*models.MediaItem) []string {
ids := make([]string, len(items))
for i, it := range items {
ids[i] = it.ContentID
}
return ids
}
func TestMergeRecentlyAddedItems_OrdersByAddedAtDescThenContentID(t *testing.T) {
base := time.Date(2026, 6, 1, 0, 0, 0, 0, time.UTC)
// Two already-sorted per-library slices (added_at DESC, content_id ASC).
libA := []*models.MediaItem{
{ContentID: "a3", AddedAt: tsPtr(base.Add(3 * time.Hour))},
{ContentID: "a1", AddedAt: tsPtr(base.Add(1 * time.Hour))},
}
libB := []*models.MediaItem{
{ContentID: "b4", AddedAt: tsPtr(base.Add(4 * time.Hour))},
{ContentID: "b2", AddedAt: tsPtr(base.Add(2 * time.Hour))},
}
got := contentIDs(mergeRecentlyAddedItems([][]*models.MediaItem{libA, libB}))
want := []string{"b4", "a3", "b2", "a1"}
if len(got) != len(want) {
t.Fatalf("merged length = %d, want %d (%v)", len(got), len(want), got)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("merged order = %v, want %v", got, want)
}
}
}
func TestMergeRecentlyAddedItems_TieBreaksByContentIDAsc(t *testing.T) {
at := time.Date(2026, 6, 1, 12, 0, 0, 0, time.UTC)
libA := []*models.MediaItem{{ContentID: "zeta", AddedAt: tsPtr(at)}}
libB := []*models.MediaItem{{ContentID: "alpha", AddedAt: tsPtr(at)}}
got := contentIDs(mergeRecentlyAddedItems([][]*models.MediaItem{libA, libB}))
want := []string{"alpha", "zeta"}
for i := range want {
if got[i] != want[i] {
t.Fatalf("tie-break order = %v, want %v", got, want)
}
}
}
func TestMergeRecentlyAddedItems_DedupsKeepingEarliestAddedAt(t *testing.T) {
early := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)
late := time.Date(2026, 6, 1, 0, 0, 0, 0, time.UTC)
other := time.Date(2026, 3, 1, 0, 0, 0, 0, time.UTC)
// "dup" appears in both libraries with different first_seen_at. The merge
// must collapse it to one entry carrying the EARLIEST timestamp (MIN
// semantics), which also moves it after "other" in DESC order.
libA := []*models.MediaItem{
{ContentID: "dup", AddedAt: tsPtr(late)},
}
libB := []*models.MediaItem{
{ContentID: "other", AddedAt: tsPtr(other)},
{ContentID: "dup", AddedAt: tsPtr(early)},
}
merged := mergeRecentlyAddedItems([][]*models.MediaItem{libA, libB})
if got := len(merged); got != 2 {
t.Fatalf("merged length = %d, want 2 (dup must collapse): %v", got, contentIDs(merged))
}
// other (March) should now precede dup (January).
want := []string{"other", "dup"}
for i := range want {
if merged[i].ContentID != want[i] {
t.Fatalf("dedup order = %v, want %v", contentIDs(merged), want)
}
}
var dup *models.MediaItem
for _, it := range merged {
if it.ContentID == "dup" {
dup = it
}
}
if dup.AddedAt == nil || !dup.AddedAt.Equal(early) {
t.Fatalf("dup AddedAt = %v, want earliest %v", dup.AddedAt, early)
}
}
func TestMergeRecentlyAddedItems_NilAddedAtSortsLast(t *testing.T) {
at := time.Date(2026, 6, 1, 0, 0, 0, 0, time.UTC)
libA := []*models.MediaItem{{ContentID: "nil1", AddedAt: nil}}
libB := []*models.MediaItem{{ContentID: "dated", AddedAt: tsPtr(at)}}
got := contentIDs(mergeRecentlyAddedItems([][]*models.MediaItem{libA, libB}))
want := []string{"dated", "nil1"}
for i := range want {
if got[i] != want[i] {
t.Fatalf("nil-handling order = %v, want %v", got, want)
}
}
}
@@ -0,0 +1,166 @@
package catalog
import (
"context"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
)
func newRestampTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
// TestItemUpsertRestampsDefaultMetadataLanguage covers the persistence half of
// issue #211: when a refresh adopts a new canonical language, the upsert must
// accept the incoming non-empty default_metadata_language instead of pinning
// to the stamp written at first match. An empty incoming value must still
// keep the existing stamp (scanner skeleton writes send the zero value).
func TestItemUpsertRestampsDefaultMetadataLanguage(t *testing.T) {
ctx := context.Background()
pool := newRestampTestPool(t)
repo := NewItemRepository(pool)
contentID := fmt.Sprintf("restamp-item-%d", time.Now().UnixNano())
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = $1`, contentID)
})
base := &models.MediaItem{
ContentID: contentID,
Type: "movie",
Title: "Gammel Titel",
Status: "matched",
DefaultMetadataLanguage: "zh",
Studios: []string{},
Networks: []string{},
Countries: []string{},
Genres: []string{},
}
if err := repo.Upsert(ctx, base); err != nil {
t.Fatalf("initial upsert: %v", err)
}
restamped := *base
restamped.Title = "Ny Titel"
restamped.DefaultMetadataLanguage = "da"
if err := repo.Upsert(ctx, &restamped); err != nil {
t.Fatalf("restamp upsert: %v", err)
}
got, err := repo.GetByID(ctx, contentID)
if err != nil {
t.Fatalf("get item: %v", err)
}
if got.DefaultMetadataLanguage != "da" {
t.Errorf("default_metadata_language = %q, want da (incoming non-empty stamp must win)", got.DefaultMetadataLanguage)
}
emptied := restamped
emptied.DefaultMetadataLanguage = ""
if err := repo.Upsert(ctx, &emptied); err != nil {
t.Fatalf("empty-language upsert: %v", err)
}
got, err = repo.GetByID(ctx, contentID)
if err != nil {
t.Fatalf("get item after empty upsert: %v", err)
}
if got.DefaultMetadataLanguage != "da" {
t.Errorf("default_metadata_language = %q, want da kept when incoming is empty", got.DefaultMetadataLanguage)
}
}
// TestSeasonAndEpisodeUpsertRestampDefaultMetadataLanguage verifies the same
// restamp-on-upsert semantics for the season and episode tables, which carry
// their own default_metadata_language pins.
func TestSeasonAndEpisodeUpsertRestampDefaultMetadataLanguage(t *testing.T) {
ctx := context.Background()
pool := newRestampTestPool(t)
suffix := time.Now().UnixNano()
seriesID := fmt.Sprintf("restamp-series-%d", suffix)
seasonID := fmt.Sprintf("restamp-season-%d", suffix)
episodeID := fmt.Sprintf("restamp-episode-%d", suffix)
t.Cleanup(func() {
// Seasons and episodes cascade from the series row (FK ON DELETE
// CASCADE), so a single delete cleans up everything seeded here.
_, _ = pool.Exec(ctx, `DELETE FROM media_items WHERE content_id = $1`, seriesID)
})
itemRepo := NewItemRepository(pool)
if err := itemRepo.Upsert(ctx, &models.MediaItem{
ContentID: seriesID,
Type: "series",
Title: "Restamp Series",
Status: "matched",
DefaultMetadataLanguage: "zh",
Studios: []string{},
Networks: []string{},
Countries: []string{},
Genres: []string{},
}); err != nil {
t.Fatalf("seed series: %v", err)
}
seasonRepo := NewSeasonRepository(pool)
season := &models.Season{
ContentID: seasonID,
SeriesID: seriesID,
SeasonNumber: 1,
Title: "Sæson 1",
DefaultMetadataLanguage: "zh",
}
if err := seasonRepo.Upsert(ctx, season); err != nil {
t.Fatalf("seed season: %v", err)
}
season.DefaultMetadataLanguage = "da"
if err := seasonRepo.Upsert(ctx, season); err != nil {
t.Fatalf("restamp season: %v", err)
}
var seasonLang string
if err := pool.QueryRow(ctx, `SELECT default_metadata_language FROM seasons WHERE content_id = $1`, seasonID).Scan(&seasonLang); err != nil {
t.Fatalf("read season language: %v", err)
}
if seasonLang != "da" {
t.Errorf("season default_metadata_language = %q, want da", seasonLang)
}
episodeRepo := NewEpisodeRepository(pool)
episode := &models.Episode{
ContentID: episodeID,
SeriesID: seriesID,
SeasonID: seasonID,
SeasonNumber: 1,
EpisodeNumber: 1,
Title: "Afsnit 1",
DefaultMetadataLanguage: "zh",
}
if err := episodeRepo.Upsert(ctx, episode); err != nil {
t.Fatalf("seed episode: %v", err)
}
episode.DefaultMetadataLanguage = "da"
if err := episodeRepo.Upsert(ctx, episode); err != nil {
t.Fatalf("restamp episode: %v", err)
}
var episodeLang string
if err := pool.QueryRow(ctx, `SELECT default_metadata_language FROM episodes WHERE content_id = $1`, episodeID).Scan(&episodeLang); err != nil {
t.Fatalf("read episode language: %v", err)
}
if episodeLang != "da" {
t.Errorf("episode default_metadata_language = %q, want da", episodeLang)
}
}
+326 -52
View File
@@ -677,6 +677,55 @@ func (s *DetailService) PendingTranslationLanguage(ctx context.Context, item *mo
return ""
}
loc, err := s.itemLocRepo.Get(ctx, item.ContentID, language)
if err != nil {
return ""
}
return pendingTranslationLanguageWith(item, language, loc)
}
// pendingTranslationLanguageWith is the shared core of PendingTranslationLanguage:
// it decides the pending-translation language for an item given a pre-resolved
// presentation language and the item's localization row (nil when none exists).
// The batch detail path reuses it after a single bulk localization lookup so it
// yields an identical result to the per-item method.
func pendingTranslationLanguageWith(item *models.MediaItem, language string, loc *models.MediaItemLocalization) string {
if item == nil || strings.TrimSpace(item.Overview) == "" || language == "" || sameMetadataLanguage(item.DefaultMetadataLanguage, language) {
return ""
}
if loc != nil && loc.Overview != "" {
return ""
}
return language
}
// PendingSeasonTranslationLanguage is the season-row equivalent of
// PendingTranslationLanguage.
func (s *DetailService) PendingSeasonTranslationLanguage(ctx context.Context, season *models.Season, filter AccessFilter) string {
if season == nil || strings.TrimSpace(season.Overview) == "" || s.seasonLocRepo == nil {
return ""
}
language, err := s.resolvePresentationLanguage(ctx, filter)
if err != nil || language == "" || sameMetadataLanguage(season.DefaultMetadataLanguage, language) {
return ""
}
loc, err := s.seasonLocRepo.Get(ctx, season.ContentID, language)
if err != nil || (loc != nil && loc.Overview != "") {
return ""
}
return language
}
// PendingEpisodeTranslationLanguage is the episode-row equivalent of
// PendingTranslationLanguage.
func (s *DetailService) PendingEpisodeTranslationLanguage(ctx context.Context, episode *models.Episode, filter AccessFilter) string {
if episode == nil || strings.TrimSpace(episode.Overview) == "" || s.episodeLocRepo == nil {
return ""
}
language, err := s.resolvePresentationLanguage(ctx, filter)
if err != nil || language == "" || sameMetadataLanguage(episode.DefaultMetadataLanguage, language) {
return ""
}
loc, err := s.episodeLocRepo.Get(ctx, episode.ContentID, language)
if err != nil || (loc != nil && loc.Overview != "") {
return ""
}
@@ -710,10 +759,22 @@ func (s *DetailService) LocalizeItemModel(ctx context.Context, item *models.Medi
return cloneMediaItem(item), err
}
loc, err := s.itemLocRepo.Get(ctx, item.ContentID, language)
if err != nil || loc == nil {
if err != nil {
return cloneMediaItem(item), err
}
return applyItemLocalization(item, loc), nil
return s.localizeItemModelWith(item, language, loc), nil
}
// localizeItemModelWith applies a pre-resolved localization to item, returning a
// clone when there is nothing to localize (no language, base language already
// matches, no localization repo, or no localization row). Shared core of
// LocalizeItemModel so the batch detail path produces identical models after a
// single bulk localization lookup.
func (s *DetailService) localizeItemModelWith(item *models.MediaItem, language string, loc *models.MediaItemLocalization) *models.MediaItem {
if language == "" || sameMetadataLanguage(item.DefaultMetadataLanguage, language) || s.itemLocRepo == nil || loc == nil {
return cloneMediaItem(item)
}
return applyItemLocalization(item, loc)
}
// LocalizeItemModels applies presentation-language localization to a batch of
@@ -842,7 +903,7 @@ func (s *DetailService) GetItemDetail(ctx context.Context, contentID string, fil
if err := s.validatePresentationItemAccess(ctx, filter, contentID); err != nil {
return nil, err
}
return s.buildMediaItemDetail(ctx, item, contentID, filter)
return s.buildMediaItemDetail(ctx, item, contentID, filter, nil)
case !errors.Is(err, ErrItemNotFound):
return nil, err
}
@@ -971,6 +1032,164 @@ func (s *DetailService) GetEpisodeDetailsForSeries(
return result, nil
}
// fileVersionBatchFetcher is the optional batch form of FileVersionFetcher.
// When the configured fetcher implements it, GetItemDetailsByIDs loads every
// page item's media files in one query instead of one per item. Fetchers that
// don't implement it (e.g. some test fakes) transparently fall back to the
// per-item GetByContentID path inside buildMediaItemDetail.
type fileVersionBatchFetcher interface {
ListByContentIDs(ctx context.Context, contentIDs []string) (map[string][]*models.MediaFile, error)
}
// GetItemDetailsByIDs returns full item details for the given content IDs,
// batching the shared per-item lookups that GetItemDetail otherwise issues
// one-at-a-time: row load (GetByIDs), access checks (EnsureAccessibleIDs +
// presentation-library membership), localization, credits, media files, and
// work summaries. The detail produced for each id is byte-for-byte identical to
// calling GetItemDetail(id): the same builder (buildMediaItemDetail) assembles
// it, only fed pre-resolved inputs via itemDetailPrefetch. Any input building
// block missing a batch form falls back to its inline per-item lookup, so the
// output never drifts from the unbatched path.
//
// IDs that are not item-typed media (e.g. episodes or seasons — handled by
// GetEpisodeDetailsForSeries / GetItemDetail respectively), are access-filtered,
// or fail to build are simply absent from the result map, mirroring
// GetItemDetail returning an error so callers fall back to list-level rendering.
func (s *DetailService) GetItemDetailsByIDs(ctx context.Context, contentIDs []string, filter AccessFilter) (map[string]*ItemDetail, error) {
result := make(map[string]*ItemDetail, len(contentIDs))
if len(contentIDs) == 0 {
return result, nil
}
items, err := s.itemRepo.GetByIDs(ctx, contentIDs)
if err != nil {
return nil, err
}
if len(items) == 0 {
return result, nil
}
foundIDs := make([]string, 0, len(items))
for _, item := range items {
foundIDs = append(foundIDs, item.ContentID)
}
// Access checks, batched to match GetItemDetail's per-item EnsureAccessible
// and validatePresentationItemAccess exactly.
accessible, err := s.itemRepo.EnsureAccessibleIDs(ctx, foundIDs, filter)
if err != nil {
return nil, err
}
presentationOK := func(string) bool { return true }
if filter.PresentationLibraryID != nil {
membership, err := s.itemRepo.GetItemsInLibrary(ctx, foundIDs, *filter.PresentationLibraryID)
if err != nil {
return nil, err
}
presentationOK = func(id string) bool { return membership[id] }
}
visible := make([]*models.MediaItem, 0, len(items))
visibleIDs := make([]string, 0, len(items))
nonSeriesIDs := make([]string, 0, len(items))
for _, item := range items {
if !accessible[item.ContentID] || !presentationOK(item.ContentID) {
continue
}
visible = append(visible, item)
visibleIDs = append(visibleIDs, item.ContentID)
if item.Type != "series" {
nonSeriesIDs = append(nonSeriesIDs, item.ContentID)
}
}
if len(visible) == 0 {
return result, nil
}
// Localization: resolve the presentation language once, then one bulk
// localization lookup. A resolution failure surfaces the same wrapped error
// GetItemDetail would produce from buildMediaItemDetail, so the caller
// degrades to the per-item path.
language, err := s.resolvePresentationLanguage(ctx, filter)
if err != nil {
return nil, fmt.Errorf("localizing item detail: %w", err)
}
var locByID map[string]*models.MediaItemLocalization
if language != "" && s.itemLocRepo != nil {
locByID, err = s.itemLocRepo.GetByContentIDs(ctx, visibleIDs, language)
if err != nil {
return nil, fmt.Errorf("localizing item detail: %w", err)
}
}
// Credits for the whole page in one query.
var creditsByID map[string][]models.ItemPerson
if s.personRepo != nil {
creditsByID, err = s.personRepo.ListForItems(ctx, visibleIDs)
if err != nil {
return nil, err
}
}
// Media files (playable versions) for non-series items in one query, when the
// fetcher supports batching.
var filesByID map[string][]*models.MediaFile
batchFetcher, haveFileBatch := s.fileFetcher.(fileVersionBatchFetcher)
if haveFileBatch && len(nonSeriesIDs) > 0 {
filesByID, err = batchFetcher.ListByContentIDs(ctx, nonSeriesIDs)
if err != nil {
return nil, err
}
}
// Work summaries for the whole page in one query, when the provider supports
// batching (ListSummariesForContentIDs is the documented batch equivalent of
// GetSummaryForContentID).
var workSummaries map[string]*WorkSummary
batchSummary, haveWorkSummaryBatch := s.workSummary.(WorkSummaryBatchProvider)
if haveWorkSummaryBatch {
workSummaries, err = batchSummary.ListSummariesForContentIDs(ctx, visibleIDs, filter)
if err != nil {
return nil, err
}
}
for _, item := range visible {
id := item.ContentID
loc := locByID[id]
pending := ""
if s.itemLocRepo != nil {
pending = pendingTranslationLanguageWith(item, language, loc)
}
pf := &itemDetailPrefetch{
haveLocalization: true,
pendingTranslation: pending,
localizedItem: s.localizeItemModelWith(item, language, loc),
haveCredits: s.personRepo != nil,
}
if s.personRepo != nil {
pf.castCredits, pf.crewCredits = splitCastCrew(s.personCredits(ctx, creditsByID[id]))
}
if item.Type != "series" && haveFileBatch {
pf.haveFiles = true
pf.files = filesByID[id]
}
if haveWorkSummaryBatch {
pf.haveWorkSummary = true
pf.workSummary = workSummaries[id]
}
detail, err := s.buildMediaItemDetail(ctx, item, id, filter, pf)
if err != nil {
// Skip rather than fail the batch; the caller falls back to list
// rendering for any id missing from the result, matching the
// per-item path where one item's build error never breaks the page.
continue
}
result[id] = detail
}
return result, nil
}
// fetchCredits returns cast and crew credits for the given content ID.
func (s *DetailService) fetchCredits(ctx context.Context, contentID string) ([]CastCredit, []CrewCredit) {
if s.personRepo == nil {
@@ -984,14 +1203,45 @@ func (s *DetailService) fetchCredits(ctx context.Context, contentID string) ([]C
return splitCastCrew(credits)
}
func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.MediaItem, contentID string, filter AccessFilter) (*ItemDetail, error) {
pendingTranslation := s.PendingTranslationLanguage(ctx, item, filter)
localizedItem, err := s.LocalizeItemModel(ctx, item, filter)
if err != nil {
return nil, fmt.Errorf("localizing item detail: %w", err)
// itemDetailPrefetch carries per-item lookups that GetItemDetailsByIDs resolves
// once for a whole page so buildMediaItemDetail can skip the equivalent per-item
// queries. Each piece is opt-in via its "have" flag: when unset, the builder
// falls back to its normal inline lookup, so the result is byte-for-byte
// identical to the unbatched GetItemDetail path regardless of which pieces were
// prefetched. A nil prefetch reproduces the original per-item behavior exactly.
type itemDetailPrefetch struct {
haveLocalization bool
pendingTranslation string
localizedItem *models.MediaItem
haveCredits bool
castCredits []CastCredit
crewCredits []CrewCredit
haveFiles bool
files []*models.MediaFile
haveWorkSummary bool
workSummary *WorkSummary
}
func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.MediaItem, contentID string, filter AccessFilter, pf *itemDetailPrefetch) (*ItemDetail, error) {
var pendingTranslation string
if pf != nil && pf.haveLocalization {
pendingTranslation = pf.pendingTranslation
item = pf.localizedItem
} else {
pendingTranslation = s.PendingTranslationLanguage(ctx, item, filter)
localizedItem, err := s.LocalizeItemModel(ctx, item, filter)
if err != nil {
return nil, fmt.Errorf("localizing item detail: %w", err)
}
item = localizedItem
}
var castCredits []CastCredit
var crewCredits []CrewCredit
if pf != nil && pf.haveCredits {
castCredits, crewCredits = pf.castCredits, pf.crewCredits
} else {
castCredits, crewCredits = s.fetchCredits(ctx, contentID)
}
item = localizedItem
castCredits, crewCredits := s.fetchCredits(ctx, contentID)
detail := &ItemDetail{
ContentID: item.ContentID,
Type: item.Type,
@@ -1042,9 +1292,15 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
// For series, each episode file shares the series content_id, so
// GetByContentID would return every episode — not alternate encodings.
if item.Type != "series" {
files, err := s.fileFetcher.GetByContentID(ctx, contentID)
if err != nil {
return nil, fmt.Errorf("fetching file versions: %w", err)
var files []*models.MediaFile
if pf != nil && pf.haveFiles {
files = pf.files
} else {
fetched, err := s.fileFetcher.GetByContentID(ctx, contentID)
if err != nil {
return nil, fmt.Errorf("fetching file versions: %w", err)
}
files = fetched
}
files = FilterMediaFilesByAccess(files, filter)
@@ -1115,7 +1371,11 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
}
}
applyWorkSummary(ctx, detail, s.workSummary, filter)
if pf != nil && pf.haveWorkSummary {
applyWorkSummaryValue(detail, pf.workSummary)
} else {
applyWorkSummary(ctx, detail, s.workSummary, filter)
}
return detail, nil
}
@@ -1124,7 +1384,17 @@ func applyWorkSummary(ctx context.Context, detail *ItemDetail, provider WorkSumm
return
}
summary, err := provider.GetSummaryForContentID(ctx, detail.ContentID, filter)
if err != nil || summary == nil {
if err != nil {
return
}
applyWorkSummaryValue(detail, summary)
}
// applyWorkSummaryValue copies a resolved work summary onto the detail. Shared by
// the per-item (applyWorkSummary) and batched (GetItemDetailsByIDs) paths so a
// summary fetched in bulk applies identically. A nil summary is a no-op.
func applyWorkSummaryValue(detail *ItemDetail, summary *WorkSummary) {
if detail == nil || summary == nil {
return
}
detail.WorkID = summary.WorkID
@@ -1755,6 +2025,7 @@ func clearSentinel(s string) string {
}
func (s *DetailService) buildSeasonDetail(ctx context.Context, season *models.Season, filter AccessFilter) (*ItemDetail, error) {
pendingTranslation := s.PendingSeasonTranslationLanguage(ctx, season, filter)
localizedSeason, err := s.LocalizeSeasonModel(ctx, season, filter)
if err != nil {
return nil, fmt.Errorf("localizing season detail: %w", err)
@@ -1790,22 +2061,23 @@ func (s *DetailService) buildSeasonDetail(ctx context.Context, season *models.Se
seasonNumber := season.SeasonNumber
castCredits, crewCredits := s.fetchCredits(ctx, season.SeriesID)
detail := &ItemDetail{
ContentID: season.ContentID,
Type: "season",
Title: title,
Overview: season.Overview,
PosterThumbhash: season.PosterThumbhash,
BackdropThumbhash: series.BackdropThumbhash,
SeriesID: season.SeriesID,
SeriesTitle: series.Title,
SeasonNumber: &seasonNumber,
EpisodeCount: &episodeCount,
IsSpecials: season.SeasonNumber == 0,
Cast: castCredits,
Crew: crewCredits,
Versions: []FileVersion{},
PlaybackVariants: []PlaybackVariant{},
Subtitles: []SubtitleInfo{},
ContentID: season.ContentID,
Type: "season",
Title: title,
Overview: season.Overview,
PendingTranslationLanguage: pendingTranslation,
PosterThumbhash: season.PosterThumbhash,
BackdropThumbhash: series.BackdropThumbhash,
SeriesID: season.SeriesID,
SeriesTitle: series.Title,
SeasonNumber: &seasonNumber,
EpisodeCount: &episodeCount,
IsSpecials: season.SeasonNumber == 0,
Cast: castCredits,
Crew: crewCredits,
Versions: []FileVersion{},
PlaybackVariants: []PlaybackVariant{},
Subtitles: []SubtitleInfo{},
}
if season.AirDate != nil {
airDate := season.AirDate.Format("2006-01-02")
@@ -1818,6 +2090,7 @@ func (s *DetailService) buildSeasonDetail(ctx context.Context, season *models.Se
}
func (s *DetailService) buildEpisodeDetail(ctx context.Context, episode *models.Episode, seriesCtx *seriesDetailContext, filter AccessFilter) (*ItemDetail, error) {
pendingTranslation := s.PendingEpisodeTranslationLanguage(ctx, episode, filter)
localizedEpisode, err := s.LocalizeEpisodeModel(ctx, episode, filter)
if err != nil {
return nil, fmt.Errorf("localizing episode detail: %w", err)
@@ -1828,27 +2101,28 @@ func (s *DetailService) buildEpisodeDetail(ctx context.Context, episode *models.
seasonNumber := episode.SeasonNumber
episodeNumber := episode.EpisodeNumber
detail := &ItemDetail{
ContentID: episode.ContentID,
Type: "episode",
Title: episode.Title,
Overview: episode.Overview,
Runtime: episode.Runtime,
RatingIMDB: episode.RatingIMDB,
RatingTMDB: episode.RatingTMDB,
ImdbID: episode.ImdbID,
TmdbID: episode.TmdbID,
TvdbID: episode.TvdbID,
PosterThumbhash: episode.StillThumbhash,
BackdropThumbhash: series.BackdropThumbhash,
SeriesID: episode.SeriesID,
SeriesTitle: series.Title,
SeasonNumber: &seasonNumber,
EpisodeNumber: &episodeNumber,
Cast: seriesCtx.castCredits,
Crew: seriesCtx.crewCredits,
Versions: []FileVersion{},
PlaybackVariants: []PlaybackVariant{},
Subtitles: []SubtitleInfo{},
ContentID: episode.ContentID,
Type: "episode",
Title: episode.Title,
Overview: episode.Overview,
PendingTranslationLanguage: pendingTranslation,
Runtime: episode.Runtime,
RatingIMDB: episode.RatingIMDB,
RatingTMDB: episode.RatingTMDB,
ImdbID: episode.ImdbID,
TmdbID: episode.TmdbID,
TvdbID: episode.TvdbID,
PosterThumbhash: episode.StillThumbhash,
BackdropThumbhash: series.BackdropThumbhash,
SeriesID: episode.SeriesID,
SeriesTitle: series.Title,
SeasonNumber: &seasonNumber,
EpisodeNumber: &episodeNumber,
Cast: seriesCtx.castCredits,
Crew: seriesCtx.crewCredits,
Versions: []FileVersion{},
PlaybackVariants: []PlaybackVariant{},
Subtitles: []SubtitleInfo{},
}
if episode.AirDate != nil {
airDate := episode.AirDate.Format("2006-01-02")
@@ -0,0 +1,281 @@
package catalog
import (
"context"
"fmt"
"os"
"reflect"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
)
// batchEquivFileFetcher is a FileVersionFetcher (and the optional batch
// fileVersionBatchFetcher) backed by one in-memory map. Serving both the
// per-item GetByContentID and the batched ListByContentIDs from the same
// backing data is exactly how the real scanner.FileRepository behaves: the two
// methods return the same files for a given content ID. This lets the
// equivalence test prove that GetItemDetailsByIDs routes prefetched files into
// buildMediaItemDetail identically to the per-item GetItemDetail path, without
// pulling the scanner package in (which would create an import cycle).
type batchEquivFileFetcher struct {
files map[string][]*models.MediaFile
}
func (f *batchEquivFileFetcher) GetByContentID(_ context.Context, contentID string) ([]*models.MediaFile, error) {
return f.files[contentID], nil
}
func (f *batchEquivFileFetcher) GetByEpisodeID(_ context.Context, _ string) ([]*models.MediaFile, error) {
return nil, nil
}
func (f *batchEquivFileFetcher) ListByContentIDs(_ context.Context, contentIDs []string) (map[string][]*models.MediaFile, error) {
out := make(map[string][]*models.MediaFile, len(contentIDs))
for _, id := range contentIDs {
if files, ok := f.files[id]; ok {
out[id] = files
}
}
return out, nil
}
// batchEquivWorkSummaryProvider implements both the per-item
// WorkSummaryProvider and the batched WorkSummaryBatchProvider from one backing
// map, the same contract literaryworks.Repository satisfies in production. As
// with files, this proves the batch path's work-summary routing matches the
// per-item path without importing literaryworks (import cycle).
type batchEquivWorkSummaryProvider struct {
summaries map[string]*WorkSummary
}
func (p *batchEquivWorkSummaryProvider) GetSummaryForContentID(_ context.Context, contentID string, _ AccessFilter) (*WorkSummary, error) {
return p.summaries[contentID], nil
}
func (p *batchEquivWorkSummaryProvider) ListSummariesForContentIDs(_ context.Context, contentIDs []string, _ AccessFilter) (map[string]*WorkSummary, error) {
out := make(map[string]*WorkSummary, len(contentIDs))
for _, id := range contentIDs {
if s, ok := p.summaries[id]; ok {
out[id] = s
}
}
return out, nil
}
func newBatchEquivTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
var tableName *string
if err := pool.QueryRow(ctx, `SELECT to_regclass('public.media_items')::text`).Scan(&tableName); err != nil {
t.Fatalf("check media_items table: %v", err)
}
if tableName == nil || *tableName == "" {
t.Skip("test database has not applied base schema")
}
return pool
}
func batchEquivExec(t *testing.T, pool *pgxpool.Pool, sql string, args ...any) {
t.Helper()
if _, err := pool.Exec(context.Background(), sql, args...); err != nil {
t.Fatalf("seed (%s): %v", sql, err)
}
}
// TestGetItemDetailsByIDs_MatchesGetItemDetail exercises the REAL
// DetailService.GetItemDetailsByIDs against the REAL per-item GetItemDetail over
// shared in-memory/real repositories, asserting per-id deep equality of the
// returned *ItemDetail.
//
// Unlike the jellycompat handler test (which stubs the content service so both
// paths return the same canned *ItemDetail), this drives the genuinely-batched
// repository code: ItemRepository.GetByIDs vs GetByID, EnsureAccessibleIDs vs
// EnsureAccessible, MediaItemLocalizationRepository.GetByContentIDs vs Get, and
// PersonRepository.ListForItems vs ListForItem all run against a real Postgres.
// Media files and work summaries route through interface fakes that serve the
// batch and per-item method shapes from one backing map (mirroring the real
// FileRepository / literaryworks.Repository), so the file/work-summary
// prefetch wiring is compared too.
//
// Equivalence axes covered, with a mixed movie+movie+series dataset:
// - access checks (EnsureAccessibleIDs vs EnsureAccessible), incl. an
// access-filtered R-rated item asserted ABSENT from the batch map and
// rejected by per-item GetItemDetail;
// - localization: a movie with an fr localization row (localized title/
// overview path) alongside a movie that has none (nil short-circuit) and
// a pending-translation-language case;
// - credits ordering (ListForItems vs ListForItem) on the first movie;
// - media files (ListByContentIDs vs GetByContentID) on the first movie;
// - work summary (ListSummariesForContentIDs vs GetSummaryForContentID) on
// the series.
func TestGetItemDetailsByIDs_MatchesGetItemDetail(t *testing.T) {
pool := newBatchEquivTestPool(t)
ctx := context.Background()
suffix := time.Now().UnixNano()
movieA := fmt.Sprintf("batch-equiv-movie-a-%d", suffix)
movieB := fmt.Sprintf("batch-equiv-movie-b-%d", suffix)
series := fmt.Sprintf("batch-equiv-series-%d", suffix)
movieR := fmt.Sprintf("batch-equiv-movie-r-%d", suffix)
personActor1 := suffix
personActor2 := suffix + 1
personDirector := suffix + 2
t.Cleanup(func() {
ids := []string{movieA, movieB, series, movieR}
batchEquivExec(t, pool, `DELETE FROM item_people WHERE content_id = ANY($1)`, ids)
batchEquivExec(t, pool, `DELETE FROM people WHERE id = ANY($1)`, []int64{personActor1, personActor2, personDirector})
batchEquivExec(t, pool, `DELETE FROM media_item_localizations WHERE content_id = ANY($1)`, ids)
batchEquivExec(t, pool, `DELETE FROM media_items WHERE content_id = ANY($1)`, ids)
})
insertItem := func(contentID, mediaType, title, overview, rating string) {
batchEquivExec(t, pool, `
INSERT INTO media_items (content_id, type, title, genres, overview, content_rating, default_metadata_language)
VALUES ($1, $2, $3, '{}'::text[], $4, $5, 'en')
`, contentID, mediaType, title, overview, rating)
}
// movieA: localized into fr, credited, has files. content_rating PG -> visible.
insertItem(movieA, "movie", "Movie A", "Movie A overview (en)", "PG")
// movieB: overview present, default lang en, NO fr localization row -> the
// pending-translation-language path must fire and match between paths.
insertItem(movieB, "movie", "Movie B", "Movie B overview (en)", "PG")
// series: has a work summary, no playable files (series skip the file path).
insertItem(series, "series", "Series One", "Series overview (en)", "PG")
// movieR: R-rated -> filtered out by MaxContentRating=PG-13 on BOTH paths.
insertItem(movieR, "movie", "Movie R", "Movie R overview (en)", "R")
// fr localization for movieA only.
batchEquivExec(t, pool, `
INSERT INTO media_item_localizations (
content_id, language, title, sort_title, overview, tagline,
poster_path, poster_source_path, poster_thumbhash,
backdrop_path, backdrop_source_path, backdrop_thumbhash,
logo_path, logo_source_path, overview_source, tagline_source
)
VALUES ($1, 'fr', 'Film A', '', 'Synopsis du film A (fr)', '',
'', '', '', '', '', '', '', '', 'manual', 'manual')
`, movieA)
// Credits on movieA: two actors (cast) + one director (crew). Seeded with
// sort_order so ListForItems / ListForItem ordering parity is observable.
insertPerson := func(id int64, name string) {
batchEquivExec(t, pool, `INSERT INTO people (id, name) VALUES ($1, $2)`, id, name)
}
insertPerson(personActor1, "Lead Actor")
insertPerson(personActor2, "Supporting Actor")
insertPerson(personDirector, "The Director")
insertCredit := func(id, personID int64, kind models.PersonKind, character string, order int) {
batchEquivExec(t, pool, `
INSERT INTO item_people (id, person_id, content_id, kind, character, sort_order)
VALUES ($1, $2, $3, $4, $5, $6)
`, id, personID, movieA, int16(kind), character, order)
}
insertCredit(personActor1, personActor1, models.PersonKindActor, "Hero", 0)
insertCredit(personActor2, personActor2, models.PersonKindActor, "Sidekick", 1)
insertCredit(personDirector, personDirector, models.PersonKindDirector, "", 0)
fileFetcher := &batchEquivFileFetcher{files: map[string][]*models.MediaFile{
movieA: {
{ID: 9001, ContentID: movieA, FilePath: "/media/movie-a-1080p.mkv", Container: "mkv", Resolution: "1080p", Duration: 6000, FileSize: 5_000_000},
{ID: 9002, ContentID: movieA, FilePath: "/media/movie-a-720p.mkv", Container: "mkv", Resolution: "720p", Duration: 6000, FileSize: 2_000_000},
},
}}
workSummaries := &batchEquivWorkSummaryProvider{summaries: map[string]*WorkSummary{
series: {
WorkID: "work-series-1",
Title: "Series One Work",
Formats: []WorkFormatSummary{
{Type: "series", ContentID: series, LibraryID: 3},
},
},
}}
newService := func() *DetailService {
svc := NewDetailService(
NewItemRepository(pool),
NewEpisodeRepository(pool),
NewSeasonRepository(pool),
NewPersonRepository(pool),
fileFetcher,
)
svc.SetWorkSummaryProvider(workSummaries)
return svc
}
// Single shared access filter for BOTH paths: PG-13 ceiling (so PG items are
// visible, the R item is not) and an explicit fr presentation language (so
// the localization path actually runs rather than short-circuiting).
filter := AccessFilter{
PresentationLanguage: "fr",
MaxContentRating: "PG-13",
UserID: 1,
ProfileID: "profile-1",
}
visibleIDs := []string{movieA, movieB, series}
allIDs := []string{movieA, movieB, series, movieR}
batch, err := newService().GetItemDetailsByIDs(ctx, allIDs, filter)
if err != nil {
t.Fatalf("GetItemDetailsByIDs: %v", err)
}
// Access-control parity: the R-rated item must be absent from the batch map
// and rejected by per-item GetItemDetail, i.e. EnsureAccessibleIDs and
// EnsureAccessible agree.
if _, present := batch[movieR]; present {
t.Fatalf("access-filtered item %s present in batch result", movieR)
}
if _, err := newService().GetItemDetail(ctx, movieR, filter); err == nil {
t.Fatalf("per-item GetItemDetail(%s) succeeded, want access rejection", movieR)
}
for _, id := range visibleIDs {
batchDetail, ok := batch[id]
if !ok {
t.Fatalf("id %s missing from batch result", id)
}
perItem, err := newService().GetItemDetail(ctx, id, filter)
if err != nil {
t.Fatalf("GetItemDetail(%s): %v", id, err)
}
if !reflect.DeepEqual(batchDetail, perItem) {
t.Fatalf("detail mismatch for %s:\nbatch: %#v\nperItem: %#v", id, batchDetail, perItem)
}
}
// Guard that the distinguishing data actually landed (otherwise the deep
// equality above could pass vacuously on empty details).
if got := batch[movieA]; got.Title != "Film A" || got.Overview != "Synopsis du film A (fr)" {
t.Fatalf("movieA not localized into fr: title=%q overview=%q", got.Title, got.Overview)
}
if got := batch[movieA]; len(got.Cast) != 2 || len(got.Crew) != 1 {
t.Fatalf("movieA credits = %d cast / %d crew, want 2/1", len(got.Cast), len(got.Crew))
}
if got := batch[movieA]; got.Cast[0].Character != "Hero" || got.Cast[1].Character != "Sidekick" {
t.Fatalf("movieA cast order = %q,%q, want Hero,Sidekick", got.Cast[0].Character, got.Cast[1].Character)
}
if got := batch[movieA]; len(got.Versions) != 2 {
t.Fatalf("movieA versions = %d, want 2 (file prefetch not applied)", len(got.Versions))
}
if got := batch[movieB]; got.PendingTranslationLanguage != "fr" {
t.Fatalf("movieB pending translation language = %q, want fr", got.PendingTranslationLanguage)
}
if got := batch[series]; got.WorkID != "work-series-1" {
t.Fatalf("series work summary not applied: WorkID=%q", got.WorkID)
}
}
@@ -108,6 +108,9 @@ var SensitiveSettingKeys = map[string]bool{
// value by the notifications system; clients receive the public half via
// the capability endpoint, never from the settings store).
"notifications.web_push.vapid_keypair": true,
// Silo push relay bearer credential for APNs/FCM delivery.
"notifications.push_relay_api_key": true,
}
// EncryptedSettingsRepo decorates a raw settings store, transparently
@@ -158,6 +158,7 @@ func TestSensitiveSettingKeys_Audited(t *testing.T) {
"discord.client_secret",
"discord.bot_token",
"notifications.web_push.vapid_keypair",
"notifications.push_relay_api_key",
}
for _, k := range mustHave {
if !SensitiveSettingKeys[k] {
@@ -32,6 +32,31 @@ type episodeCatalogUserStatePlan struct {
requireProgressRatio bool
}
// applyEpisodeCatalogAccessFilter applies the section access constraints the
// episode_catalog_entries table supports. The table has no `type` column, so
// the compat audiobook/podcast media-type exclusion cannot be expressed here:
// emitting `NOT (ece.type = ANY(...))` raised SQLSTATE 42703 and degraded the
// whole query to empty. The exclusion is therefore dropped on this path.
//
// Dropping it is NOT a no-op masquerading as cosmetic: episode_catalog_entries
// is NOT episodes-of-TV-series only — episode_libraries is populated from any
// media_files row with a non-null episode_id, so podcast episodes
// (media_items.type = 'podcast') also land here. What keeps non-TV episodes out
// of episode-scoped results is library scoping plus the `si.type = 'series'`
// guard in episodeCatalogSelectBody (hydration), not the media-type clause.
// content_rating filtering is preserved here (that column exists).
//
// Because hydration's si.type='series' guard drops non-series rows AFTER the
// page/count scans select them, the scan and hydration would otherwise diverge:
// pages could under-fill and TotalRecordCount could count rows that never
// render. episodeCatalogSeriesParentGuard re-expresses that same constraint at
// the entry-scan level so both queries stay aligned with hydration.
func applyEpisodeCatalogAccessFilter(access AccessFilter, whereParts *[]string, args *[]any, argIdx *int) {
access.ExcludedMediaTypes = nil // access is a value param; caller unaffected
ApplySectionAccessFilter("ece", access, whereParts, args, argIdx)
*whereParts = append(*whereParts, episodeCatalogSeriesParentGuard)
}
func (e *QueryExecutor) tryEpisodeCatalogUserStatePreviewPage(
ctx context.Context,
def QueryDefinition,
@@ -84,7 +109,7 @@ func (e *QueryExecutor) tryEpisodeCatalogUserStatePreviewPage(
argIdx++
}
ApplySectionAccessFilter("ece", access, &whereParts, &args, &argIdx)
applyEpisodeCatalogAccessFilter(access, &whereParts, &args, &argIdx)
if e.SnapshotAt != nil {
whereParts = append(whereParts, fmt.Sprintf("ece.episode_created_at <= $%d", argIdx))
@@ -242,7 +267,7 @@ func (e *QueryExecutor) tryEpisodeCatalogEntriesPreviewPage(
argIdx++
}
ApplySectionAccessFilter("ece", access, &whereParts, &args, &argIdx)
applyEpisodeCatalogAccessFilter(access, &whereParts, &args, &argIdx)
if e.SnapshotAt != nil {
whereParts = append(whereParts, fmt.Sprintf("ece.episode_created_at <= $%d", argIdx))
@@ -45,6 +45,31 @@ func TestEpisodeCatalogSelectBodyExposesQualifiedColumns(t *testing.T) {
}
}
// TestEpisodeCatalogSelectBodyExcludesNonSeriesEpisodes guards the invariant
// that the episode catalog only ever hydrates episodes whose parent is a TV
// series. episode_libraries (and episode_catalog_entries) also contain podcast
// episodes (media_items.type = 'podcast'); the `si.type = 'series'` predicate in
// episodeCatalogSelectBody is what keeps those non-TV rows out of episode-scoped
// results regardless of which library a query resolves. Dropping it would let
// podcast/audiobook episodes surface in a TV "episode" rail (fail-open).
func TestEpisodeCatalogSelectBodyExcludesNonSeriesEpisodes(t *testing.T) {
if !strings.Contains(episodeCatalogBaseRelation, "si.type = 'series'") {
t.Fatalf("episodeCatalogBaseRelation must guard si.type = 'series' so non-TV "+
"(podcast/audiobook) episodes never hydrate; got %q", episodeCatalogBaseRelation)
}
// The library-scoped relation and the hydration relation render from the same
// template, so the guard must travel with every rendering.
scoped, _, handled := episodeCatalogBaseRelationForLibraries([]int{2}, nil, 1)
if !handled {
t.Fatal("expected episode library scope to be handled")
}
if !strings.Contains(scoped, "si.type = 'series'") {
t.Fatalf("library-scoped episode relation must keep the si.type = 'series' "+
"guard; got %q", scoped)
}
}
// columnsReferencedOnAlias returns every distinct column referenced as
// "<alias>.<column>" within projection (e.g. mi.air_timezone -> air_timezone).
func columnsReferencedOnAlias(projection, alias string) map[string]bool {
+31 -1
View File
@@ -5,6 +5,18 @@ import (
"strings"
)
// episodeCatalogSelectBody is the hand-written subquery that hydrates episode
// rows for the "episode" media scope, aliased "mi" so the shared catalog
// projection can read it. The `si.type = 'series'` predicate is load-bearing,
// not cosmetic: episode_libraries (and thus episode_catalog_entries) is
// populated from every media_files row with a non-null episode_id, including
// podcast episodes (media_items.type = 'podcast'). Without this guard those
// non-TV episodes would hydrate into episode-scoped results, e.g. when a
// library-scoped episode query happens to resolve a podcast/audiobook folder.
// Restricting hydration to series-parented episodes keeps the episode catalog
// to genuine TV episodes on every surface (native and Jellyfin-compat). Real
// TV episodes always hang off a type='series' parent, so this never
// over-filters legitimate episode sections.
const episodeCatalogSelectBody = `(
SELECT
e.content_id,
@@ -63,9 +75,27 @@ const episodeCatalogSelectBody = `(
FROM episodes e
JOIN media_items si ON si.content_id = e.series_id
LEFT JOIN seasons s ON s.content_id = e.season_id
WHERE %s
WHERE (%s)
AND si.type = 'series'
) mi`
// episodeCatalogSeriesParentGuard mirrors the load-bearing `si.type = 'series'`
// predicate in episodeCatalogSelectBody, but expressed against an entry-scan row
// (ece.episode_id) instead of the hydration join. episode_catalog_entries rows
// can reference episodes parented to non-series media_items (e.g. podcast
// episodes), which hydration silently drops. Applying this same constraint to
// the page and count scans keeps them aligned with hydration so pages never
// under-fill and TotalRecordCount never counts rows that can't hydrate. It binds
// no parameters, so it can be appended to any whereParts list without touching
// the argument index.
const episodeCatalogSeriesParentGuard = `EXISTS (
SELECT 1
FROM episodes e
JOIN media_items si ON si.content_id = e.series_id
WHERE e.content_id = ece.episode_id
AND si.type = 'series'
)`
const episodeCatalogActiveLibraryExists = `EXISTS (
SELECT 1
FROM episode_libraries el
+46 -2
View File
@@ -320,7 +320,7 @@ func (r *EpisodeRepository) Upsert(ctx context.Context, ep *models.Episode) erro
ON CONFLICT (series_id, season_number, episode_number) DO UPDATE SET
season_id = COALESCE(EXCLUDED.season_id, episodes.season_id),
title = EXCLUDED.title,
default_metadata_language = COALESCE(NULLIF(episodes.default_metadata_language, ''), EXCLUDED.default_metadata_language),
default_metadata_language = COALESCE(NULLIF(EXCLUDED.default_metadata_language, ''), episodes.default_metadata_language),
overview = EXCLUDED.overview,
air_date = EXCLUDED.air_date,
runtime = EXCLUDED.runtime,
@@ -503,7 +503,7 @@ func (r *EpisodeRepository) BulkUpsert(ctx context.Context, seriesID string, epi
ON CONFLICT (series_id, season_number, episode_number) DO UPDATE SET
season_id = COALESCE(EXCLUDED.season_id, episodes.season_id),
title = EXCLUDED.title,
default_metadata_language = COALESCE(NULLIF(episodes.default_metadata_language, ''), EXCLUDED.default_metadata_language),
default_metadata_language = COALESCE(NULLIF(EXCLUDED.default_metadata_language, ''), episodes.default_metadata_language),
overview = EXCLUDED.overview,
air_date = EXCLUDED.air_date,
runtime = EXCLUDED.runtime,
@@ -827,6 +827,50 @@ func (r *EpisodeRepository) ListBySeason(ctx context.Context, seriesID string, s
return scanEpisodes(rows)
}
// ListAdjacentInSeries returns the episode at (seasonNumber, episodeNumber)
// together with its immediate previous and next available neighbors within the
// series, in natural (season, episode) order. It is the bounded backing query
// for Jellyfin's AdjacentTo request (Wholphin autoplay/skip): instead of
// materializing every episode of the series, each neighbor is found with a
// bounded index range scan backed by the episodes_series_season_episode_key
// UNIQUE index (series_id, season_number, episode_number). Neighbors are
// resolved across season boundaries, so the result is at most three rows.
func (r *EpisodeRepository) ListAdjacentInSeries(ctx context.Context, seriesID string, seasonNumber, episodeNumber int) ([]*models.Episode, error) {
query := `SELECT ` + episodeColumns + ` FROM (
(SELECT ` + episodeColumns + `
FROM episodes
WHERE series_id = $1
AND (season_number, episode_number) < ($2, $3)
AND ` + episodeAvailabilityPredicate + `
ORDER BY season_number DESC, episode_number DESC
LIMIT 1)
UNION ALL
(SELECT ` + episodeColumns + `
FROM episodes
WHERE series_id = $1
AND season_number = $2
AND episode_number = $3
AND ` + episodeAvailabilityPredicate + `)
UNION ALL
(SELECT ` + episodeColumns + `
FROM episodes
WHERE series_id = $1
AND (season_number, episode_number) > ($2, $3)
AND ` + episodeAvailabilityPredicate + `
ORDER BY season_number ASC, episode_number ASC
LIMIT 1)
) AS adjacent
ORDER BY season_number ASC, episode_number ASC`
rows, err := r.pool.Query(ctx, query, seriesID, seasonNumber, episodeNumber)
if err != nil {
return nil, fmt.Errorf("listing adjacent episodes in series: %w", err)
}
defer rows.Close()
return scanEpisodes(rows)
}
// ListBySeasonID returns all episodes for a specific season row, ordered by
// episode number.
func (r *EpisodeRepository) ListBySeasonID(ctx context.Context, seasonID string) ([]*models.Episode, error) {
+64 -1
View File
@@ -467,7 +467,7 @@ func (r *ItemRepository) upsert(ctx context.Context, execer itemExecer, item *mo
type = EXCLUDED.type,
title = EXCLUDED.title,
sort_title = EXCLUDED.sort_title,
default_metadata_language = COALESCE(NULLIF(media_items.default_metadata_language, ''), EXCLUDED.default_metadata_language),
default_metadata_language = COALESCE(NULLIF(EXCLUDED.default_metadata_language, ''), media_items.default_metadata_language),
original_title = EXCLUDED.original_title,
year = EXCLUDED.year,
genres = EXCLUDED.genres,
@@ -1333,6 +1333,69 @@ func (r *ItemRepository) EnsureAccessible(ctx context.Context, contentID string,
return nil
}
// EnsureAccessibleIDs is the batch form of EnsureAccessible: it returns the set
// of content IDs from the input that the viewer may access, applying the exact
// same predicates (library allow/deny via the single JOIN form, max content
// rating, excluded media types). An item is accessible iff at least one
// media_items row — joined per library link when a library filter is active —
// satisfies the predicates, matching EnsureAccessible's `LIMIT 1` semantics.
// IDs absent from the returned map are not accessible (mirrors EnsureAccessible
// returning ErrItemNotFound). Used by GetItemDetailsByIDs to avoid a per-item
// EnsureAccessible fan-out.
func (r *ItemRepository) EnsureAccessibleIDs(ctx context.Context, contentIDs []string, filter AccessFilter) (map[string]bool, error) {
result := make(map[string]bool, len(contentIDs))
if len(contentIDs) == 0 {
return result, nil
}
// An empty (non-nil) allowlist means "no libraries allowed" — nothing is
// accessible, exactly as EnsureAccessible returns ErrItemNotFound.
if filter.AllowedLibraryIDs != nil && len(filter.AllowedLibraryIDs) == 0 {
return result, nil
}
var conditions []string
var args []any
argIdx := 1
fromClause := "media_items mi"
conditions = append(conditions, fmt.Sprintf("mi.content_id = ANY($%d)", argIdx))
args = append(args, contentIDs)
argIdx++
needsLibJoin := filter.AllowedLibraryIDs != nil || len(filter.DisabledLibraryIDs) > 0
if filter.AllowedLibraryIDs != nil {
conditions = append(conditions, fmt.Sprintf("mil.media_folder_id = ANY($%d)", argIdx))
args = append(args, filter.AllowedLibraryIDs)
argIdx++
}
if len(filter.DisabledLibraryIDs) > 0 {
conditions = append(conditions, fmt.Sprintf("NOT (mil.media_folder_id = ANY($%d))", argIdx))
args = append(args, filter.DisabledLibraryIDs)
argIdx++
}
if needsLibJoin {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
}
applyAccessFilter("mi", AccessFilter{MaxContentRating: filter.MaxContentRating, ExcludedMediaTypes: filter.ExcludedMediaTypes}, &conditions, &args, &argIdx)
query := fmt.Sprintf("SELECT DISTINCT mi.content_id FROM %s WHERE %s", fromClause, strings.Join(conditions, " AND "))
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("checking items access: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, fmt.Errorf("scanning item access row: %w", err)
}
result[id] = true
}
return result, rows.Err()
}
// GetItemsInLibrary returns a membership map for the provided content IDs within
// a single library. Episode content IDs are matched directly against
// episode_libraries so compat callers can filter mixed item/episode batches.
+1
View File
@@ -199,6 +199,7 @@ func buildListNextUpQuery(q NextUpQuery, limit int) (string, []interface{}) {
WHERE uwp2.user_id = $1
AND uwp2.profile_id = $2
AND uwp2.media_item_id = e2.content_id
AND (uwp2.completed = TRUE OR uwp2.position_seconds > 0)
)
ORDER BY e2.season_number, e2.episode_number
LIMIT 1
+1
View File
@@ -20,6 +20,7 @@ func TestBuildListNextUpQuery_PrefersRecentCompletedOverOlderPartialProgress(t *
"uwp_ip.updated_at > ce.updated_at",
"FROM user_history_hidden_items hhi",
"uwp.updated_at <= hhi.hidden_before",
"AND (uwp2.completed = TRUE OR uwp2.position_seconds > 0)",
"ORDER BY e.series_id, uwp.updated_at DESC, e.season_number DESC, e.episode_number DESC",
"LIMIT $3",
}
+230
View File
@@ -0,0 +1,230 @@
package catalog
import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestConflictingExternalID(t *testing.T) {
p := models.Person{TmdbID: "111", ImdbID: "nm222"}
tests := []struct {
name string
constraint string
wantField string
wantValue string
wantOK bool
}{
{"tmdb", "idx_people_tmdb_id", "tmdb_id", "111", true},
{"imdb", "idx_people_imdb_id", "imdb_id", "nm222", true},
{"unknown constraint", "idx_people_name", "", "", false},
{"empty constraint", "", "", "", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
field, value, ok := conflictingExternalID(tt.constraint, p)
if field != tt.wantField || value != tt.wantValue || ok != tt.wantOK {
t.Fatalf("conflictingExternalID(%q) = (%q,%q,%v), want (%q,%q,%v)",
tt.constraint, field, value, ok, tt.wantField, tt.wantValue, tt.wantOK)
}
})
}
// A blank id must never be reported as the conflict source.
if _, _, ok := conflictingExternalID("idx_people_tmdb_id", models.Person{}); ok {
t.Fatal("expected ok=false when the conflicting field is empty")
}
}
func TestExternalIDsCompatible(t *testing.T) {
tests := []struct {
name string
a, b models.Person
want bool
}{
{
name: "tmdb-only vs imdb-only (the core dedup case)",
a: models.Person{TmdbID: "111", ImdbID: "nm9"},
b: models.Person{ImdbID: "nm9"},
want: true,
},
{
name: "identical ids",
a: models.Person{TmdbID: "111", ImdbID: "nm9"},
b: models.Person{TmdbID: "111", ImdbID: "nm9"},
want: true,
},
{
name: "conflicting tmdb ids",
a: models.Person{TmdbID: "111", ImdbID: "nm9"},
b: models.Person{TmdbID: "999", ImdbID: "nm9"},
want: false,
},
{
name: "conflicting tvdb ids",
a: models.Person{ImdbID: "nm9", TvdbID: "5"},
b: models.Person{ImdbID: "nm9", TvdbID: "6"},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := externalIDsCompatible(tt.a, tt.b); got != tt.want {
t.Fatalf("externalIDsCompatible = %v, want %v", got, tt.want)
}
})
}
}
func TestCanMergePeople(t *testing.T) {
tests := []struct {
name string
a, b models.Person
want bool
}{
{
name: "same name, compatible ids -> merge",
a: models.Person{Name: "Jane Doe", TmdbID: "111"},
b: models.Person{Name: "Jane Doe", ImdbID: "nm9"},
want: true,
},
{
name: "case/space-insensitive name match -> merge",
a: models.Person{Name: " jane doe ", TmdbID: "111"},
b: models.Person{Name: "Jane Doe", ImdbID: "nm9"},
want: true,
},
{
name: "different names, compatible ids -> do not merge",
a: models.Person{Name: "Jane Doe", TmdbID: "111"},
b: models.Person{Name: "John Smith", ImdbID: "nm9"},
want: false,
},
{
name: "same name but contradictory ids -> do not merge",
a: models.Person{Name: "Jane Doe", TmdbID: "111"},
b: models.Person{Name: "Jane Doe", TmdbID: "999"},
want: false,
},
{
name: "empty name on one side -> do not merge",
a: models.Person{Name: "", TmdbID: "111"},
b: models.Person{Name: "Jane Doe", ImdbID: "nm9"},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := canMergePeople(tt.a, tt.b); got != tt.want {
t.Fatalf("canMergePeople = %v, want %v", got, tt.want)
}
})
}
}
func TestMergePersonFields(t *testing.T) {
birth := time.Date(1980, 1, 2, 0, 0, 0, 0, time.UTC)
t.Run("backfills only empty fields, survivor wins", func(t *testing.T) {
dst := models.Person{
Name: "Jane Doe",
TmdbID: "111",
Bio: "survivor bio",
}
src := models.Person{
Name: "Janey",
TmdbID: "999", // must NOT overwrite a populated survivor id
ImdbID: "nm9", // fills the gap
Bio: "partner bio",
Birthplace: "Springfield",
BirthDate: &birth,
PhotoPath: "/p.jpg",
}
mergePersonFields(&dst, src)
if dst.Name != "Jane Doe" {
t.Errorf("Name overwritten: %q", dst.Name)
}
if dst.TmdbID != "111" {
t.Errorf("populated TmdbID overwritten: %q", dst.TmdbID)
}
if dst.ImdbID != "nm9" {
t.Errorf("empty ImdbID not backfilled: %q", dst.ImdbID)
}
if dst.Bio != "survivor bio" {
t.Errorf("populated Bio overwritten: %q", dst.Bio)
}
if dst.Birthplace != "Springfield" {
t.Errorf("empty Birthplace not backfilled: %q", dst.Birthplace)
}
if dst.BirthDate == nil || !dst.BirthDate.Equal(birth) {
t.Errorf("nil BirthDate not backfilled: %v", dst.BirthDate)
}
if dst.PhotoPath != "/p.jpg" {
t.Errorf("empty PhotoPath not backfilled: %q", dst.PhotoPath)
}
})
t.Run("backfills name when survivor is blank", func(t *testing.T) {
dst := models.Person{TmdbID: "111"}
src := models.Person{Name: "Jane Doe"}
mergePersonFields(&dst, src)
if dst.Name != "Jane Doe" {
t.Errorf("blank survivor Name not backfilled: %q", dst.Name)
}
})
t.Run("photo trio is folded together", func(t *testing.T) {
dst := models.Person{} // no photo at all
src := models.Person{
PhotoPath: "/p.jpg",
PhotoSourcePath: "tmdb://p",
PhotoThumbhash: "hash",
}
mergePersonFields(&dst, src)
if dst.PhotoPath != "/p.jpg" || dst.PhotoSourcePath != "tmdb://p" || dst.PhotoThumbhash != "hash" {
t.Errorf("photo trio not folded together: %+v", dst)
}
// A survivor that already has a photo keeps its own trio intact.
keep := models.Person{PhotoPath: "/keep.jpg"}
mergePersonFields(&keep, src)
if keep.PhotoPath != "/keep.jpg" || keep.PhotoSourcePath != "" || keep.PhotoThumbhash != "" {
t.Errorf("survivor photo trio was disturbed: %+v", keep)
}
})
}
func TestExternalIDValueAndSet(t *testing.T) {
p := models.Person{TmdbID: "111", ImdbID: "nm9", TvdbID: "5", PlexGUID: "g"}
for field, want := range map[string]string{
"tmdb_id": "111", "imdb_id": "nm9", "tvdb_id": "5", "plex_guid": "g", "nope": "",
} {
if got := externalIDValue(p, field); got != want {
t.Errorf("externalIDValue(%q) = %q, want %q", field, got, want)
}
}
// Setting a field to a new value replaces only that column.
updated := p
setExternalIDField(&updated, "imdb_id", "nm12345")
if updated.ImdbID != "nm12345" {
t.Errorf("setExternalIDField did not set imdb_id: %q", updated.ImdbID)
}
if updated.TmdbID != "111" {
t.Errorf("setExternalIDField touched the wrong field: %q", updated.TmdbID)
}
// Setting a field to "" blanks it, matching the old clear behavior.
cleared := p
setExternalIDField(&cleared, "imdb_id", "")
if cleared.ImdbID != "" {
t.Errorf("setExternalIDField did not blank imdb_id: %q", cleared.ImdbID)
}
if cleared.TmdbID != "111" {
t.Errorf("setExternalIDField touched the wrong field: %q", cleared.TmdbID)
}
}
+387 -10
View File
@@ -2,7 +2,9 @@ package catalog
import (
"context"
"errors"
"fmt"
"log/slog"
"strconv"
"strings"
"time"
@@ -493,6 +495,13 @@ func (r *PersonRepository) GetByName(ctx context.Context, name string) (*models.
}
// Search finds persons by name substring (case-insensitive), ordered by name.
//
// The predicate is `name ILIKE '%term%'` rather than `LOWER(name) LIKE ...` so
// the pg_trgm GIN index idx_people_name_trgm can serve it: for a rare 3+ char
// term that index turns a ~300-400ms full name-index scan into a ~10ms bitmap
// scan. ILIKE is itself case-insensitive, so this stays equivalent to the prior
// LOWER(name) comparison (including its existing treatment of % and _ in the
// term as LIKE wildcards).
func (r *PersonRepository) Search(ctx context.Context, query string, limit int) ([]models.Person, error) {
if limit <= 0 {
limit = 20
@@ -500,7 +509,7 @@ func (r *PersonRepository) Search(ctx context.Context, query string, limit int)
rows, err := r.pool.Query(ctx, `
SELECT id, name, sort_name, bio, birth_date, death_date, birthplace, homepage,
photo_path, photo_source_path, photo_thumbhash, tmdb_id, imdb_id, tvdb_id, plex_guid, created_at, updated_at
FROM people WHERE LOWER(name) LIKE '%' || LOWER($1) || '%'
FROM people WHERE name ILIKE '%' || $1 || '%'
ORDER BY name LIMIT $2`, query, limit,
)
if err != nil {
@@ -521,15 +530,114 @@ func (r *PersonRepository) Search(ctx context.Context, query string, limit int)
return people, rows.Err()
}
// maxPersonConflictResolutions bounds how many external-id collisions a single
// Update is allowed to resolve before giving up. One per unique external-id
// field is the natural ceiling (today only tmdb_id and imdb_id are unique).
const maxPersonConflictResolutions = 4
// Update updates all non-key fields on a person.
//
// A person's resolved external ids (tmdb_id / imdb_id) can collide with another
// row that turns out to be the same human discovered through a different id —
// e.g. one credit ingested with only a tmdb_id and another with only an imdb_id,
// later reconciled when a refresh fills in the missing cross-id. Rather than
// failing that write (SQLSTATE 23505) and letting the refresh worker retry the
// same row forever, Update detects the collision and resolves it: it merges the
// two people when they are the same human, or drops just the conflicting id when
// they are not, then retries the write so updated_at advances and the row leaves
// the stale set.
//
// The common, no-collision path runs as a single plain transaction. Only when a
// 23505 surfaces does Update fall back to updateResolvingConflicts, which does
// the whole reconciliation — merge or drop, plus the field write and reindex —
// inside one transaction so it commits atomically (or not at all). Returns
// pgx.ErrNoRows if the row no longer exists (e.g. merged away concurrently).
func (r *PersonRepository) Update(ctx context.Context, p models.Person) error {
err := r.applyUpdate(ctx, p)
if err == nil || !isDuplicateKeyError(err) {
return err
}
return r.updateResolvingConflicts(ctx, p)
}
// applyUpdate writes all non-key fields on a person and enqueues a search
// reindex for the items they appear in, all in one transaction.
func (r *PersonRepository) applyUpdate(ctx context.Context, p models.Person) error {
tx, err := r.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("begin person update tx: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
_, err = tx.Exec(ctx, `
affected, err := execPersonUpdate(ctx, tx, p)
if err != nil {
return err
}
if affected == 0 {
// The row was deleted out from under us (e.g. merged away concurrently).
return pgx.ErrNoRows
}
if err := reindexPersonItems(ctx, tx, p.ID); err != nil {
return err
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit person update tx: %w", err)
}
return nil
}
// updateResolvingConflicts performs the person write inside a single transaction,
// reconciling external-id collisions (merge or drop) as they arise. The write is
// retried after each resolution via a savepoint so a failed attempt does not
// poison the transaction; the whole thing commits atomically once the write
// lands. Bounded by maxPersonConflictResolutions so it cannot spin.
func (r *PersonRepository) updateResolvingConflicts(ctx context.Context, p models.Person) error {
tx, err := r.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("begin person merge tx: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
var affected int64
for attempt := 0; ; attempt++ {
affected, err = tryPersonUpdate(ctx, tx, p)
if err == nil {
break
}
if !isDuplicateKeyError(err) || attempt >= maxPersonConflictResolutions {
return err
}
field, value, ok := conflictingExternalID(extractConstraint(err), p)
if !ok {
// Unique violation on a constraint we do not know how to reconcile.
return err
}
resolved, resolveErr := r.resolveExternalIDConflict(ctx, tx, &p, field, value)
if resolveErr != nil {
return resolveErr
}
if !resolved {
return err
}
}
if affected == 0 {
// Survivor was deleted concurrently (e.g. merged into another row); there
// is nothing left to persist for this id.
return pgx.ErrNoRows
}
if err := reindexPersonItems(ctx, tx, p.ID); err != nil {
return err
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit person merge tx: %w", err)
}
return nil
}
// execPersonUpdate writes all non-key person fields and returns the rows affected.
func execPersonUpdate(ctx context.Context, tx pgx.Tx, p models.Person) (int64, error) {
tag, err := tx.Exec(ctx, `
UPDATE people SET name=$2, sort_name=$3, bio=$4, birth_date=$5, death_date=$6,
birthplace=$7, homepage=$8, photo_path=$9, photo_source_path=$10, photo_thumbhash=$11,
tmdb_id=$12, imdb_id=$13, tvdb_id=$14, plex_guid=$15, updated_at=now()
@@ -539,30 +647,299 @@ func (r *PersonRepository) Update(ctx context.Context, p models.Person) error {
p.TmdbID, p.ImdbID, p.TvdbID, p.PlexGUID,
)
if err != nil {
return err
return 0, err
}
return tag.RowsAffected(), nil
}
// tryPersonUpdate runs execPersonUpdate inside a savepoint so a unique violation
// rolls back only the attempt, leaving the surrounding transaction usable.
func tryPersonUpdate(ctx context.Context, tx pgx.Tx, p models.Person) (int64, error) {
sp, err := tx.Begin(ctx)
if err != nil {
return 0, fmt.Errorf("begin person update savepoint: %w", err)
}
affected, err := execPersonUpdate(ctx, sp, p)
if err != nil {
_ = sp.Rollback(ctx)
return 0, err
}
if err := sp.Commit(ctx); err != nil {
return 0, fmt.Errorf("release person update savepoint: %w", err)
}
return affected, nil
}
// reindexPersonItems enqueues a catalog-search reindex for every item the person
// appears in. After a merge the survivor's link set already includes the merged
// person's items, so this covers all items whose denormalized people text changed.
func reindexPersonItems(ctx context.Context, tx pgx.Tx, personID int64) error {
rows, err := tx.Query(ctx, `
SELECT DISTINCT content_id
FROM item_people
WHERE person_id = $1
`, p.ID)
`, personID)
if err != nil {
return fmt.Errorf("listing items linked to person %d: %w", p.ID, err)
return fmt.Errorf("listing items linked to person %d: %w", personID, err)
}
contentIDs, err := pgx.CollectRows(rows, pgx.RowTo[string])
if err != nil {
return fmt.Errorf("collecting items linked to person %d: %w", p.ID, err)
return fmt.Errorf("collecting items linked to person %d: %w", personID, err)
}
if err := EnqueueSearchIndexUpserts(ctx, tx, contentIDs); err != nil {
return fmt.Errorf("enqueueing catalog search person update for person %d: %w", p.ID, err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit person update tx: %w", err)
return fmt.Errorf("enqueueing catalog search person update for person %d: %w", personID, err)
}
return nil
}
// resolveExternalIDConflict reconciles a unique-violation on (field=value) raised
// while updating *p, operating within the caller's transaction tx. It mutates *p
// in place (folding in the partner's data, or dropping the conflicting id) and
// reports whether the caller should retry the write. resolved=false means the
// conflict could not be reconciled and the original error should be surfaced.
func (r *PersonRepository) resolveExternalIDConflict(
ctx context.Context,
tx pgx.Tx,
p *models.Person,
field string,
value string,
) (resolved bool, err error) {
if value == "" {
return false, nil
}
// Identify the row that currently owns the colliding id.
var partnerID int64
lookup := fmt.Sprintf("SELECT id FROM people WHERE %s = $1", field)
switch scanErr := tx.QueryRow(ctx, lookup, value).Scan(&partnerID); {
case errors.Is(scanErr, pgx.ErrNoRows):
// The conflicting owner vanished (concurrent merge); retry the plain write.
return true, nil
case scanErr != nil:
return false, fmt.Errorf("locate person owning %s=%q: %w", field, value, scanErr)
}
if partnerID == p.ID {
// Self-conflict is impossible under the unique index; bail rather than loop.
return false, nil
}
// Lock both rows in a deterministic order so concurrent merges cannot deadlock.
people, err := scanPeopleByIDsForUpdate(ctx, tx, p.ID, partnerID)
if err != nil {
return false, err
}
if _, ok := people[p.ID]; !ok {
// The survivor was deleted under us (concurrent merge); retry the plain
// write, which will affect zero rows and surface as pgx.ErrNoRows.
return true, nil
}
partner, ok := people[partnerID]
if !ok {
// Partner already gone; retry the plain write.
return true, nil
}
// Re-confirm the partner still holds the colliding value after locking.
if externalIDValue(partner, field) != value {
return true, nil
}
if !canMergePeople(*p, partner) {
// Not confidently the same human (contradictory ids, or names disagree):
// restore the survivor's currently-persisted value for this field so the
// retried write becomes a no-op on that column. This lets the write commit
// instead of looping forever, without destructively deleting a
// possibly-distinct person and without blanking an id the survivor already
// held (blanking would silently drop a previously-valid provider id, e.g.
// on the admin PATCH path that mutates an existing id into a colliding one).
// Writing the row's own current value back can never violate the unique
// index, so this field will not re-trigger the conflict.
setExternalIDField(p, field, externalIDValue(people[p.ID], field))
slog.Warn("person merge: declining to merge, preserving survivor's existing id",
"person_id", p.ID, "partner_id", partnerID, "field", field, "value", value,
"person_name", p.Name, "partner_name", partner.Name)
return true, nil
}
// Fold the partner's ids and any metadata this write lacks onto the survivor,
// so the retried write persists data only the partner had.
mergePersonFields(p, partner)
// Repoint the partner's credits onto the survivor, skipping links that would
// duplicate an existing (content_id, kind, character) credit on the survivor.
if _, err := tx.Exec(ctx, `
UPDATE item_people ip SET person_id = $1
WHERE ip.person_id = $2
AND NOT EXISTS (
SELECT 1 FROM item_people e
WHERE e.person_id = $1
AND e.content_id = ip.content_id
AND e.kind = ip.kind
AND e.character = ip.character
)`, p.ID, partnerID); err != nil {
return false, fmt.Errorf("repoint item_people from %d to %d: %w", partnerID, p.ID, err)
}
// Delete the partner; any leftover duplicate credits cascade away.
if _, err := tx.Exec(ctx, `DELETE FROM people WHERE id = $1`, partnerID); err != nil {
return false, fmt.Errorf("delete merged person %d: %w", partnerID, err)
}
slog.Info("person merge: folded duplicate person",
"survivor_id", p.ID, "merged_id", partnerID, "field", field, "value", value)
return true, nil
}
// scanPeopleByIDsForUpdate loads the given people, locking them FOR UPDATE in
// ascending id order so concurrent merges acquire locks consistently.
func scanPeopleByIDsForUpdate(ctx context.Context, tx pgx.Tx, ids ...int64) (map[int64]models.Person, error) {
rows, err := tx.Query(ctx, `
SELECT id, name, sort_name, bio, birth_date, death_date, birthplace, homepage,
photo_path, photo_source_path, photo_thumbhash, tmdb_id, imdb_id, tvdb_id, plex_guid, created_at, updated_at
FROM people
WHERE id = ANY($1)
ORDER BY id
FOR UPDATE`, ids)
if err != nil {
return nil, fmt.Errorf("lock people for merge: %w", err)
}
defer rows.Close()
out := make(map[int64]models.Person, len(ids))
for rows.Next() {
var p models.Person
if err := rows.Scan(&p.ID, &p.Name, &p.SortName, &p.Bio, &p.BirthDate, &p.DeathDate, &p.Birthplace, &p.Homepage,
&p.PhotoPath, &p.PhotoSourcePath, &p.PhotoThumbhash, &p.TmdbID, &p.ImdbID, &p.TvdbID, &p.PlexGUID, &p.CreatedAt, &p.UpdatedAt,
); err != nil {
return nil, fmt.Errorf("scan locked person: %w", err)
}
out[p.ID] = p
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate locked people: %w", err)
}
return out, nil
}
// conflictingExternalID maps a violated unique constraint to the person field
// and value that caused it. ok is false for constraints Update cannot reconcile.
func conflictingExternalID(constraint string, p models.Person) (field, value string, ok bool) {
switch constraint {
case "idx_people_tmdb_id":
return "tmdb_id", p.TmdbID, p.TmdbID != ""
case "idx_people_imdb_id":
return "imdb_id", p.ImdbID, p.ImdbID != ""
default:
return "", "", false
}
}
// externalIDValue returns the value of the named external-id column on p.
func externalIDValue(p models.Person, field string) string {
switch field {
case "tmdb_id":
return p.TmdbID
case "imdb_id":
return p.ImdbID
case "tvdb_id":
return p.TvdbID
case "plex_guid":
return p.PlexGUID
default:
return ""
}
}
// setExternalIDField sets the named external-id column on p to value.
func setExternalIDField(p *models.Person, field, value string) {
switch field {
case "tmdb_id":
p.TmdbID = value
case "imdb_id":
p.ImdbID = value
case "tvdb_id":
p.TvdbID = value
case "plex_guid":
p.PlexGUID = value
}
}
// externalIDsCompatible reports whether a and b can be the same person: no
// external-id field where both hold a different non-empty value. A field where
// one side is empty (or both match) is compatible.
func externalIDsCompatible(a, b models.Person) bool {
return idsCompatible(a.TmdbID, b.TmdbID) &&
idsCompatible(a.ImdbID, b.ImdbID) &&
idsCompatible(a.TvdbID, b.TvdbID) &&
idsCompatible(a.PlexGUID, b.PlexGUID)
}
func idsCompatible(x, y string) bool {
return x == "" || y == "" || x == y
}
// canMergePeople reports whether two rows that collided on a shared external id
// are confidently the same human and may be folded together. A shared id is a
// strong signal, but providers occasionally hand the same id to two distinct
// people; requiring the names to agree as well guards against destructively
// deleting a genuinely different person. When this returns false the caller
// drops the conflicting id instead (non-destructive).
func canMergePeople(a, b models.Person) bool {
return externalIDsCompatible(a, b) && personNamesMatch(a.Name, b.Name)
}
func personNamesMatch(a, b string) bool {
a = strings.TrimSpace(a)
b = strings.TrimSpace(b)
if a == "" || b == "" {
return false
}
return strings.EqualFold(a, b)
}
// mergePersonFields fills empty external ids and metadata on dst from src,
// preserving dst's existing values. dst is the refreshed survivor, so its
// populated fields win; src (the row being merged away) only backfills gaps.
func mergePersonFields(dst *models.Person, src models.Person) {
if dst.Name == "" {
dst.Name = src.Name
}
if dst.TmdbID == "" {
dst.TmdbID = src.TmdbID
}
if dst.ImdbID == "" {
dst.ImdbID = src.ImdbID
}
if dst.TvdbID == "" {
dst.TvdbID = src.TvdbID
}
if dst.PlexGUID == "" {
dst.PlexGUID = src.PlexGUID
}
if dst.Bio == "" {
dst.Bio = src.Bio
}
if dst.Birthplace == "" {
dst.Birthplace = src.Birthplace
}
if dst.Homepage == "" {
dst.Homepage = src.Homepage
}
if dst.SortName == "" {
dst.SortName = src.SortName
}
if dst.BirthDate == nil {
dst.BirthDate = src.BirthDate
}
if dst.DeathDate == nil {
dst.DeathDate = src.DeathDate
}
if dst.PhotoPath == "" {
dst.PhotoPath = src.PhotoPath
dst.PhotoSourcePath = src.PhotoSourcePath
dst.PhotoThumbhash = src.PhotoThumbhash
}
}
func (r *PersonRepository) UpdatePhotoIfSourceMatches(ctx context.Context, personID int64, sourcePath, cachedPath, thumbhash string) (bool, error) {
if r == nil || r.pool == nil {
return false, pgx.ErrNoRows
+2 -2
View File
@@ -112,7 +112,7 @@ func (r *SeasonRepository) Upsert(ctx context.Context, s *models.Season) error {
)
ON CONFLICT (series_id, season_number) DO UPDATE SET
title = EXCLUDED.title,
default_metadata_language = COALESCE(NULLIF(seasons.default_metadata_language, ''), EXCLUDED.default_metadata_language),
default_metadata_language = COALESCE(NULLIF(EXCLUDED.default_metadata_language, ''), seasons.default_metadata_language),
overview = EXCLUDED.overview,
air_date = EXCLUDED.air_date,
poster_path = EXCLUDED.poster_path,
@@ -198,7 +198,7 @@ func (r *SeasonRepository) BulkUpsert(ctx context.Context, seasons []*models.Sea
)
ON CONFLICT (series_id, season_number) DO UPDATE SET
title = EXCLUDED.title,
default_metadata_language = COALESCE(NULLIF(seasons.default_metadata_language, ''), EXCLUDED.default_metadata_language),
default_metadata_language = COALESCE(NULLIF(EXCLUDED.default_metadata_language, ''), seasons.default_metadata_language),
overview = EXCLUDED.overview,
air_date = EXCLUDED.air_date,
poster_path = EXCLUDED.poster_path,
+4 -1
View File
@@ -1828,7 +1828,10 @@ func bulkUpdatePeople(ctx context.Context, tx pgx.Tx, people map[int64]*importPe
FROM (VALUES `,
rows,
7,
nil,
// A standalone VALUES clause has no target column to infer types
// from, so v.id must be cast explicitly or it defaults to text and
// the join predicate becomes bigint = text (SQLSTATE 42883).
map[int]string{0: "::bigint"},
`) AS v(id, tmdb_id, imdb_id, tvdb_id, plex_guid, photo_path, photo_thumbhash)
WHERE p.id = v.id`,
nil,
+63
View File
@@ -0,0 +1,63 @@
package catalogseed
import (
"context"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func newCatalogSeedTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
return pool
}
func TestBulkUpdatePeopleComparesIDsAsBigint(t *testing.T) {
ctx := context.Background()
pool := newCatalogSeedTestPool(t)
tx, err := pool.Begin(ctx)
if err != nil {
t.Fatalf("begin tx: %v", err)
}
defer func() { _ = tx.Rollback(ctx) }()
personID := time.Now().UnixNano()
if _, err := tx.Exec(ctx, `
INSERT INTO people (id, name, sort_name, tmdb_id)
VALUES ($1, 'Import Test Person', 'import test person', '111')
`, personID); err != nil {
t.Fatalf("seed person: %v", err)
}
people := map[int64]*importPersonState{
personID: {
ID: personID,
Name: "Import Test Person",
TmdbID: "222",
NeedsUpdate: true,
},
}
if err := bulkUpdatePeople(ctx, tx, people); err != nil {
t.Fatalf("bulkUpdatePeople: %v", err)
}
var tmdbID string
if err := tx.QueryRow(ctx, `SELECT tmdb_id FROM people WHERE id = $1`, personID).Scan(&tmdbID); err != nil {
t.Fatalf("read back person: %v", err)
}
if tmdbID != "222" {
t.Fatalf("tmdb_id = %q, want %q", tmdbID, "222")
}
}
+12 -1
View File
@@ -303,6 +303,13 @@ type DownloadConfig struct {
MaxConcurrentPerUser int `yaml:"-"` // max simultaneous downloads per user (0 = unlimited)
MaxPerPeriod int `yaml:"-"` // max downloads per user per period (0 = unlimited)
PeriodDuration time.Duration `yaml:"-"` // rolling window for MaxPerPeriod
// Downloads v2 (offline sync for mobile). The prepare-to-file pipeline that
// consumes these ships in Phase 3; they are parsed default-off from Phase 0.
TranscodeEnabled bool `yaml:"-"` // server gate for transcode-to-file (default false)
ArtifactDir string `yaml:"-"` // prepared-artifact output volume ("" = default under the transcode dir)
MaxConcurrentPrepares int `yaml:"-"` // encode/remux worker-pool size (default 2)
ArtifactMaxBytes int64 `yaml:"-"` // LRU eviction budget for prepared artifacts (0 = unlimited)
}
// MetadataConfig holds metadata pipeline settings.
@@ -365,6 +372,10 @@ var defaultJellyfinCompatServerID = uuid.NewSHA1(
[]byte("https://silo.local/jellycompat"),
).String()
// DefaultTranscodeDir is the fallback playback.transcode_dir; download
// artifacts default to a sibling directory (see downloads.effectiveArtifactDir).
const DefaultTranscodeDir = "/tmp/silo-transcode"
const DefaultJellyfinCompatEmulatedServerVersion = "10.12.0"
const DefaultJellyfinWebVersion = "10.11.6"
const DefaultJellyfinWebInstallDir = "/var/lib/silo/compat/jellyfin-web"
@@ -432,7 +443,7 @@ func setDefaults() *configRaw {
},
Playback: PlaybackConfig{
FFmpegPath: "/usr/lib/jellyfin-ffmpeg/ffmpeg",
TranscodeDir: "/tmp/silo-transcode",
TranscodeDir: DefaultTranscodeDir,
HWAccel: "auto",
ChapterThumbnailWorkers: 1,
ChapterThumbnailExecution: "local",
+26 -1
View File
@@ -302,7 +302,7 @@ func LoadFromDB(m map[string]string) (*Config, error) {
// Playback
cfg.Playback.FFmpegPath = stringOr(m, "playback.ffmpeg_path", "/usr/lib/jellyfin-ffmpeg/ffmpeg")
cfg.Playback.TranscodeDir = stringOr(m, "playback.transcode_dir", "/tmp/silo-transcode")
cfg.Playback.TranscodeDir = stringOr(m, "playback.transcode_dir", DefaultTranscodeDir)
cfg.Playback.HWAccel = stringOr(m, "playback.hw_accel", "auto")
cfg.Playback.HWDevice = stringOr(m, "playback.hw_device", "")
chapterThumbnailWorkers, err := intOr(m, "playback.chapter_thumbnail_workers", 1)
@@ -565,5 +565,30 @@ func LoadFromDB(m map[string]string) (*Config, error) {
cfg.Download.MaxPerPeriod = maxPerPeriod
cfg.Download.PeriodDuration = periodDuration
// Downloads v2 (offline sync). Default-off; the prepare-to-file pipeline that
// consumes these ships in Phase 3.
downloadTranscodeEnabled, err := boolOr(m, "download.transcode_enabled", false)
if err != nil {
return nil, err
}
maxConcurrentPrepares, err := intOr(m, "download.max_concurrent_prepares", 2)
if err != nil {
return nil, err
}
artifactMaxBytes, err := int64Or(m, "download.artifact_max_bytes", 0)
if err != nil {
return nil, err
}
if maxConcurrentPrepares < 0 {
return nil, fmt.Errorf("invalid value for %q: must be non-negative", "download.max_concurrent_prepares")
}
if artifactMaxBytes < 0 {
return nil, fmt.Errorf("invalid value for %q: must be non-negative", "download.artifact_max_bytes")
}
cfg.Download.TranscodeEnabled = downloadTranscodeEnabled
cfg.Download.ArtifactDir = stringOr(m, "download.artifact_dir", "")
cfg.Download.MaxConcurrentPrepares = maxConcurrentPrepares
cfg.Download.ArtifactMaxBytes = artifactMaxBytes
return cfg, nil
}
+2
View File
@@ -42,6 +42,8 @@ func TestRestartRequired(t *testing.T) {
{"overlays.enabled", false},
{"markers.mode", false},
{"download.enabled", false},
{"download.transcode_enabled", false},
{"download.max_concurrent_prepares", false},
{"allow_4k_transcode", false},
{"defaults.card_overlays", false},
// Unknown keys default to live.
-50
View File
@@ -1,50 +0,0 @@
package download
import (
"errors"
"time"
)
// Download status constants.
const (
StatusQueued = "queued"
StatusDownloading = "downloading"
StatusCompleted = "completed"
StatusFailed = "failed"
StatusCancelled = "cancelled"
)
// Download kind constants.
const (
KindDirect = "direct"
KindQueued = "queued"
)
// Sentinel errors.
var (
ErrNotFound = errors.New("download not found")
ErrDownloadNotAllowed = errors.New("user is not allowed to download")
ErrFeatureDisabled = errors.New("downloads are disabled")
ErrConcurrentLimitReached = errors.New("concurrent download limit reached")
ErrPeriodLimitReached = errors.New("download period limit reached")
ErrDownloadNotActive = errors.New("download is not in an active state")
ErrStatusConflict = errors.New("download status transition conflict")
)
// Download represents a row in the downloads table.
type Download struct {
ID string
UserID int
MediaFileID int
ContentID string
EpisodeID string
BatchID string
Kind string // direct or queued
Status string // queued, downloading, completed, failed, cancelled
FileSize int64
BytesSent int64
ErrorMessage string
CreatedAt time.Time
UpdatedAt time.Time
CompletedAt *time.Time
}
-239
View File
@@ -1,239 +0,0 @@
package download
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
const downloadColumns = `id, user_id, media_file_id, content_id, episode_id, batch_id,
kind, status, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at`
// Repository provides CRUD operations for the downloads table.
type Repository struct {
pool *pgxpool.Pool
}
// NewRepository creates a new Repository backed by the given pool.
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
}
func scanDownload(row pgx.Row) (*Download, error) {
var d Download
var episodeID, batchID *string
err := row.Scan(
&d.ID, &d.UserID, &d.MediaFileID, &d.ContentID, &episodeID, &batchID,
&d.Kind, &d.Status, &d.FileSize, &d.BytesSent, &d.ErrorMessage,
&d.CreatedAt, &d.UpdatedAt, &d.CompletedAt,
)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("scanning download: %w", err)
}
if episodeID != nil {
d.EpisodeID = *episodeID
}
if batchID != nil {
d.BatchID = *batchID
}
return &d, nil
}
func scanDownloads(rows pgx.Rows) ([]*Download, error) {
var downloads []*Download
for rows.Next() {
var d Download
var episodeID, batchID *string
err := rows.Scan(
&d.ID, &d.UserID, &d.MediaFileID, &d.ContentID, &episodeID, &batchID,
&d.Kind, &d.Status, &d.FileSize, &d.BytesSent, &d.ErrorMessage,
&d.CreatedAt, &d.UpdatedAt, &d.CompletedAt,
)
if err != nil {
return nil, fmt.Errorf("scanning download row: %w", err)
}
if episodeID != nil {
d.EpisodeID = *episodeID
}
if batchID != nil {
d.BatchID = *batchID
}
downloads = append(downloads, &d)
}
return downloads, rows.Err()
}
// Create inserts a new download record.
func (r *Repository) Create(ctx context.Context, d *Download) error {
_, err := r.pool.Exec(ctx,
`INSERT INTO downloads (id, user_id, media_file_id, content_id, episode_id, batch_id,
kind, status, file_size, bytes_sent, error_message, created_at, updated_at, completed_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)`,
d.ID, d.UserID, d.MediaFileID, d.ContentID,
nilIfEmpty(d.EpisodeID), nilIfEmpty(d.BatchID),
d.Kind, d.Status, d.FileSize, d.BytesSent, d.ErrorMessage,
d.CreatedAt, d.UpdatedAt, d.CompletedAt,
)
if err != nil {
return fmt.Errorf("inserting download: %w", err)
}
return nil
}
// CreateBatch inserts multiple download records in a single transaction.
func (r *Repository) CreateBatch(ctx context.Context, downloads []*Download) error {
tx, err := r.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("beginning batch insert: %w", err)
}
defer tx.Rollback(ctx)
for _, d := range downloads {
_, err := tx.Exec(ctx,
`INSERT INTO downloads (id, user_id, media_file_id, content_id, episode_id, batch_id,
kind, status, file_size, bytes_sent, error_message, created_at, updated_at, completed_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)`,
d.ID, d.UserID, d.MediaFileID, d.ContentID,
nilIfEmpty(d.EpisodeID), nilIfEmpty(d.BatchID),
d.Kind, d.Status, d.FileSize, d.BytesSent, d.ErrorMessage,
d.CreatedAt, d.UpdatedAt, d.CompletedAt,
)
if err != nil {
return fmt.Errorf("inserting batch download: %w", err)
}
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("committing batch insert: %w", err)
}
return nil
}
// GetByID retrieves a download by its ID.
func (r *Repository) GetByID(ctx context.Context, id string) (*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads WHERE id = $1`
return scanDownload(r.pool.QueryRow(ctx, query, id))
}
// ListByUser returns all downloads for a user, most recent first.
func (r *Repository) ListByUser(ctx context.Context, userID int) ([]*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE user_id = $1
ORDER BY created_at DESC`
rows, err := r.pool.Query(ctx, query, userID)
if err != nil {
return nil, fmt.Errorf("listing downloads for user: %w", err)
}
defer rows.Close()
result, err := scanDownloads(rows)
if err != nil {
return nil, fmt.Errorf("scanning download rows: %w", err)
}
return result, nil
}
// CountActiveByUser returns the number of active (queued or downloading) downloads for a user.
func (r *Repository) CountActiveByUser(ctx context.Context, userID int) (int, error) {
var count int
err := r.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM downloads WHERE user_id = $1 AND status IN ('queued', 'downloading')`,
userID,
).Scan(&count)
if err != nil {
return 0, fmt.Errorf("counting active downloads: %w", err)
}
return count, nil
}
// CountByUserSince returns the number of successful downloads created since the given time.
// Cancelled and failed downloads are excluded so transient failures don't consume quota.
func (r *Repository) CountByUserSince(ctx context.Context, userID int, since time.Time) (int, error) {
var count int
err := r.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM downloads WHERE user_id = $1 AND created_at >= $2 AND status NOT IN ('cancelled', 'failed')`,
userID, since,
).Scan(&count)
if err != nil {
return 0, fmt.Errorf("counting downloads in period: %w", err)
}
return count, nil
}
// UpdateStatus sets the status and optionally the bytes_sent and completed_at fields.
func (r *Repository) UpdateStatus(ctx context.Context, id, status string, bytesSent int64, completedAt *time.Time) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = $1, bytes_sent = $2, completed_at = $3, updated_at = now() WHERE id = $4`,
status, bytesSent, completedAt, id,
)
if err != nil {
return fmt.Errorf("updating download status: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// TransitionStatus atomically transitions a download from expectedStatus to newStatus.
// Returns ErrStatusConflict if the row is not in expectedStatus (another request
// already transitioned it).
func (r *Repository) TransitionStatus(ctx context.Context, id, expectedStatus, newStatus string, bytesSent int64, completedAt *time.Time) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = $1, bytes_sent = $2, completed_at = $3, updated_at = now()
WHERE id = $4 AND status = $5`,
newStatus, bytesSent, completedAt, id, expectedStatus,
)
if err != nil {
return fmt.Errorf("transitioning download status: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrStatusConflict
}
return nil
}
// Delete removes a download record. Returns ErrNotFound if the row doesn't exist
// or doesn't belong to the given user.
func (r *Repository) Delete(ctx context.Context, id string, userID int) error {
tag, err := r.pool.Exec(ctx,
`DELETE FROM downloads WHERE id = $1 AND user_id = $2`, id, userID,
)
if err != nil {
return fmt.Errorf("deleting download: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// CancelByID sets a download to cancelled if it is still queued or downloading.
func (r *Repository) CancelByID(ctx context.Context, id string, userID int) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = 'cancelled', updated_at = now()
WHERE id = $1 AND user_id = $2 AND status IN ('queued', 'downloading')`,
id, userID,
)
if err != nil {
return fmt.Errorf("cancelling download: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
func nilIfEmpty(s string) *string {
if s == "" {
return nil
}
return &s
}
-518
View File
@@ -1,518 +0,0 @@
package download
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/idgen"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
// FileResolver looks up media files by various keys.
type FileResolver interface {
GetByID(ctx context.Context, id int) (*models.MediaFile, error)
GetByContentID(ctx context.Context, contentID string) ([]*models.MediaFile, error)
GetByEpisodeID(ctx context.Context, episodeID string) ([]*models.MediaFile, error)
}
// ItemResolver looks up media items.
type ItemResolver interface {
GetByID(ctx context.Context, contentID string) (*models.MediaItem, error)
}
// EpisodeResolver lists episodes for a series.
type EpisodeResolver interface {
ListBySeries(ctx context.Context, seriesID string) ([]*models.Episode, error)
}
// UserResolver looks up users.
type UserResolver interface {
GetByID(ctx context.Context, id int) (*models.User, error)
}
// ItemAccessChecker checks library/content-rating access.
type ItemAccessChecker interface {
EnsureAccessible(ctx context.Context, contentID string, filter catalog.AccessFilter) error
}
// SettingsReader loads all server settings as a flat map.
type SettingsReader interface {
GetAll(ctx context.Context) (map[string]string, error)
}
const configCacheTTL = 30 * time.Second
// Service orchestrates download permission checks, quota enforcement,
// file resolution, and file serving.
type Service struct {
repo *Repository
bandwidth *BandwidthManager
limiter *QuantityLimiter
fileRepo FileResolver
itemRepo ItemResolver
episodeRepo EpisodeResolver
userRepo UserResolver
itemAccess ItemAccessChecker
settings SettingsReader
cfgMu sync.RWMutex
cfg config.DownloadConfig
cfgLoadedAt time.Time
}
// NewService creates a new download service with the given dependencies.
func NewService(
repo *Repository,
bandwidth *BandwidthManager,
limiter *QuantityLimiter,
fileRepo FileResolver,
itemRepo ItemResolver,
episodeRepo EpisodeResolver,
userRepo UserResolver,
itemAccess ItemAccessChecker,
settings SettingsReader,
initialCfg *config.DownloadConfig,
) *Service {
s := &Service{
repo: repo,
bandwidth: bandwidth,
limiter: limiter,
fileRepo: fileRepo,
itemRepo: itemRepo,
episodeRepo: episodeRepo,
userRepo: userRepo,
itemAccess: itemAccess,
settings: settings,
}
if initialCfg != nil {
s.cfg = *initialCfg
s.cfgLoadedAt = time.Now()
}
return s
}
// loadConfig returns the current download config, refreshing from DB if stale.
func (s *Service) loadConfig(ctx context.Context) config.DownloadConfig {
s.cfgMu.RLock()
if time.Since(s.cfgLoadedAt) < configCacheTTL {
cfg := s.cfg
s.cfgMu.RUnlock()
return cfg
}
s.cfgMu.RUnlock()
s.cfgMu.Lock()
defer s.cfgMu.Unlock()
// Double-check after acquiring write lock.
if time.Since(s.cfgLoadedAt) < configCacheTTL {
return s.cfg
}
if s.settings == nil {
return s.cfg
}
allSettings, err := s.settings.GetAll(ctx)
if err != nil {
slog.Warn("failed to reload download config from DB, using cached", "error", err)
return s.cfg
}
newFullCfg, err := config.LoadFromDB(allSettings)
if err != nil {
slog.Warn("failed to parse download config from DB, using cached", "error", err)
return s.cfg
}
oldCfg := s.cfg
s.cfg = newFullCfg.Download
s.cfgLoadedAt = time.Now()
// Update bandwidth manager if limits changed.
if s.bandwidth != nil && (oldCfg.ServerBandwidthBPS != s.cfg.ServerBandwidthBPS || oldCfg.UserBandwidthBPS != s.cfg.UserBandwidthBPS) {
s.bandwidth.Reload(s.cfg.ServerBandwidthBPS, s.cfg.UserBandwidthBPS)
slog.Info("download bandwidth config reloaded", "server_bps", s.cfg.ServerBandwidthBPS, "user_bps", s.cfg.UserBandwidthBPS)
}
// Update quantity limiter if limits changed.
if s.limiter != nil && (oldCfg.MaxConcurrentPerUser != s.cfg.MaxConcurrentPerUser || oldCfg.MaxPerPeriod != s.cfg.MaxPerPeriod || oldCfg.PeriodDuration != s.cfg.PeriodDuration) {
s.limiter.Reload(s.cfg.MaxConcurrentPerUser, s.cfg.MaxPerPeriod, s.cfg.PeriodDuration)
slog.Info("download quantity limits reloaded", "max_concurrent", s.cfg.MaxConcurrentPerUser, "max_per_period", s.cfg.MaxPerPeriod, "period", s.cfg.PeriodDuration)
}
return s.cfg
}
// CreateRequest holds the parameters for creating a download.
type CreateRequest struct {
ContentID string
EpisodeID string
FileID int
}
// CreateQueued creates a queued download for a single item (movie or episode).
func (s *Service) CreateQueued(ctx context.Context, userID int, req CreateRequest, filter catalog.AccessFilter) (*Download, error) {
if err := s.checkEnabled(ctx); err != nil {
return nil, err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return nil, fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return nil, ErrDownloadNotAllowed
}
file, err := s.resolveFile(ctx, req)
if err != nil {
return nil, err
}
if err := s.itemAccess.EnsureAccessible(ctx, file.ContentID, filter); err != nil {
return nil, err
}
if err := s.limiter.Check(ctx, userID, 1); err != nil {
return nil, err
}
id, err := idgen.NextID()
if err != nil {
return nil, fmt.Errorf("generating download ID: %w", err)
}
now := time.Now()
d := &Download{
ID: id,
UserID: userID,
MediaFileID: file.ID,
ContentID: file.ContentID,
EpisodeID: file.EpisodeID,
Kind: KindQueued,
Status: StatusQueued,
FileSize: file.FileSize,
CreatedAt: now,
UpdatedAt: now,
}
if err := s.repo.Create(ctx, d); err != nil {
return nil, err
}
return d, nil
}
// CreateQueuedBatch creates download records for all episodes in a series.
// Returns the created downloads and a shared batch ID.
func (s *Service) CreateQueuedBatch(ctx context.Context, userID int, seriesContentID string, filter catalog.AccessFilter) ([]*Download, string, error) {
if err := s.checkEnabled(ctx); err != nil {
return nil, "", err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return nil, "", fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return nil, "", ErrDownloadNotAllowed
}
item, err := s.itemRepo.GetByID(ctx, seriesContentID)
if err != nil {
return nil, "", fmt.Errorf("loading series: %w", err)
}
if item.Type != "series" {
return nil, "", fmt.Errorf("content_id is not a series")
}
if err := s.itemAccess.EnsureAccessible(ctx, seriesContentID, filter); err != nil {
return nil, "", err
}
episodes, err := s.episodeRepo.ListBySeries(ctx, seriesContentID)
if err != nil {
return nil, "", fmt.Errorf("listing episodes: %w", err)
}
var downloads []*Download
now := time.Now()
for _, ep := range episodes {
files, err := s.fileRepo.GetByEpisodeID(ctx, ep.ContentID)
if err != nil {
return nil, "", fmt.Errorf("resolving files for episode %s: %w", ep.ContentID, err)
}
if len(files) == 0 {
continue // skip episodes with no files
}
file := pickBestFile(files)
id, err := idgen.NextID()
if err != nil {
return nil, "", fmt.Errorf("generating download ID: %w", err)
}
downloads = append(downloads, &Download{
ID: id,
UserID: userID,
MediaFileID: file.ID,
ContentID: seriesContentID,
EpisodeID: ep.ContentID,
Kind: KindQueued,
Status: StatusQueued,
FileSize: file.FileSize,
CreatedAt: now,
UpdatedAt: now,
})
}
if len(downloads) == 0 {
return nil, "", fmt.Errorf("no downloadable episodes found")
}
if err := s.limiter.Check(ctx, userID, len(downloads)); err != nil {
return nil, "", err
}
batchID, err := idgen.NextID()
if err != nil {
return nil, "", fmt.Errorf("generating batch ID: %w", err)
}
for _, d := range downloads {
d.BatchID = batchID
}
if err := s.repo.CreateBatch(ctx, downloads); err != nil {
return nil, "", err
}
return downloads, batchID, nil
}
// ServeDirect validates permissions and serves a file directly for browser download.
// No persistent download record is created.
func (s *Service) ServeDirect(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, fileID int, filter catalog.AccessFilter) error {
if err := s.checkEnabled(ctx); err != nil {
return err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return ErrDownloadNotAllowed
}
file, err := s.fileRepo.GetByID(ctx, fileID)
if err != nil {
return fmt.Errorf("loading media file: %w", err)
}
if file == nil || file.MissingSince != nil {
return catalog.ErrItemNotFound
}
if err := s.itemAccess.EnsureAccessible(ctx, file.ContentID, filter); err != nil {
return err
}
return s.serveFileDownload(ctx, w, r, file, userID)
}
// ServeFile serves a queued download's file, verifying ownership and current policy.
func (s *Service) ServeFile(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, downloadID string) error {
// Re-check policy — admin may have disabled downloads or revoked permission
// after this download was queued.
if err := s.checkEnabled(ctx); err != nil {
return err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return ErrDownloadNotAllowed
}
dl, err := s.repo.GetByID(ctx, downloadID)
if err != nil {
return err
}
if dl.UserID != userID {
return ErrNotFound // don't reveal existence
}
if dl.Status == StatusCancelled || dl.Status == StatusFailed {
return fmt.Errorf("download is %s: %w", dl.Status, ErrDownloadNotActive)
}
file, err := s.fileRepo.GetByID(ctx, dl.MediaFileID)
if err != nil {
return fmt.Errorf("loading media file: %w", err)
}
if file == nil || file.MissingSince != nil {
return catalog.ErrItemNotFound
}
// Atomically transition queued → downloading. If another request already
// claimed this download, return a conflict error.
if dl.Status == StatusQueued {
if err := s.repo.TransitionStatus(ctx, dl.ID, StatusQueued, StatusDownloading, 0, nil); err != nil {
if errors.Is(err, ErrStatusConflict) {
return fmt.Errorf("download already in progress: %w", ErrDownloadNotActive)
}
slog.Warn("failed to transition download to downloading", "download_id", dl.ID, "error", err)
}
}
if err := s.serveFileDownload(ctx, w, r, file, userID); err != nil {
if updateErr := s.repo.UpdateStatus(ctx, dl.ID, StatusFailed, 0, nil); updateErr != nil {
slog.Error("failed to mark download as failed", "download_id", dl.ID, "error", updateErr)
}
return err
}
now := time.Now()
if err := s.repo.UpdateStatus(ctx, dl.ID, StatusCompleted, file.FileSize, &now); err != nil {
slog.Error("failed to mark download as completed", "download_id", dl.ID, "error", err)
}
return nil
}
// List returns all downloads for a user.
func (s *Service) List(ctx context.Context, userID int) ([]*Download, error) {
return s.repo.ListByUser(ctx, userID)
}
// Cancel cancels a queued/active download or deletes a completed/failed one.
func (s *Service) Cancel(ctx context.Context, userID int, downloadID string) error {
dl, err := s.repo.GetByID(ctx, downloadID)
if err != nil {
return err
}
if dl.UserID != userID {
return ErrNotFound
}
switch dl.Status {
case StatusQueued, StatusDownloading:
return s.repo.CancelByID(ctx, downloadID, userID)
default:
return s.repo.Delete(ctx, downloadID, userID)
}
}
func (s *Service) checkEnabled(ctx context.Context) error {
cfg := s.loadConfig(ctx)
if !cfg.Enabled {
return ErrFeatureDisabled
}
return nil
}
func (s *Service) resolveFile(ctx context.Context, req CreateRequest) (*models.MediaFile, error) {
if req.FileID > 0 {
file, err := s.fileRepo.GetByID(ctx, req.FileID)
if err != nil {
return nil, fmt.Errorf("loading media file: %w", err)
}
if file == nil || file.MissingSince != nil {
return nil, catalog.ErrItemNotFound
}
return file, nil
}
var files []*models.MediaFile
var err error
if req.EpisodeID != "" {
files, err = s.fileRepo.GetByEpisodeID(ctx, req.EpisodeID)
} else {
files, err = s.fileRepo.GetByContentID(ctx, req.ContentID)
}
if err != nil {
return nil, fmt.Errorf("resolving files: %w", err)
}
if len(files) == 0 {
return nil, catalog.ErrItemNotFound
}
return pickBestFile(files), nil
}
func (s *Service) serveFileDownload(ctx context.Context, w http.ResponseWriter, r *http.Request, file *models.MediaFile, userID int) error {
f, err := os.Open(file.FilePath)
if err != nil {
if os.IsNotExist(err) {
return catalog.ErrItemNotFound
}
return fmt.Errorf("opening file: %w", err)
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return fmt.Errorf("stat file: %w", err)
}
filename := sanitizeFilename(filepath.Base(file.FilePath))
w.Header().Set("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, filename))
w.Header().Set("Content-Type", playback.MimeFromExtension(file.FilePath))
var reader io.ReadSeeker = f
if s.bandwidth != nil {
reader = s.bandwidth.ThrottledReader(ctx, f, userID)
}
http.ServeContent(w, r, stat.Name(), stat.ModTime(), reader)
return nil
}
// pickBestFile selects the highest-resolution file from a list.
func pickBestFile(files []*models.MediaFile) *models.MediaFile {
if len(files) == 1 {
return files[0]
}
best := files[0]
for _, f := range files[1:] {
if resolutionRank(f.Resolution) > resolutionRank(best.Resolution) {
best = f
}
}
return best
}
func resolutionRank(res string) int {
switch strings.ToLower(res) {
case "2160p":
return 4
case "1080p":
return 3
case "720p":
return 2
case "480p":
return 1
default:
return 0
}
}
func sanitizeFilename(name string) string {
return strings.Map(func(r rune) rune {
switch r {
case '/', '\\', '"', '<', '>', '|', '?', '*', ':':
return '_'
}
return r
}, name)
}
+93
View File
@@ -0,0 +1,93 @@
package downloads
import (
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"path/filepath"
"time"
"github.com/Silo-Server/silo-server/internal/config"
)
// Artifact status constants (download_artifacts.status).
const (
ArtifactQueued = "queued"
ArtifactRunning = "running"
ArtifactReady = "ready"
ArtifactFailed = "failed"
)
// ErrNoArtifactJob is returned by the queue when no claimable job exists.
var ErrNoArtifactJob = errors.New("no claimable artifact job")
// Artifact is a prepared (remux/transcode) file, deduplicated by
// (media_file_id, format, params_hash), and a row in the durable encode queue.
type Artifact struct {
ID string
MediaFileID int
Format string // remux | transcode
ParamsHash string
Container string
CodecVideo string
CodecAudio string
Resolution string
AudioTrackIndex int
TargetBitrateKbps int
OutputPath string
FileSize int64
Status string
ErrorMessage string
Attempts int
MaxAttempts int
LeaseOwner string
LeaseExpiresAt *time.Time
NextRetryAt *time.Time
CreatedAt time.Time
CompletedAt *time.Time
LastUsedAt time.Time
}
// paramsHash is the dedup key for an encode target:
// sha256(format | container | codec_video | codec_audio | resolution | audio_track_index | bitrate | subtitle_burn_in).
func paramsHash(format, container, codecVideo, codecAudio, resolution string, audioTrackIndex, targetBitrateKbps int, subtitleBurnIn bool) string {
sum := sha256.Sum256([]byte(fmt.Sprintf("%s|%s|%s|%s|%s|%d|%d|%t",
format, container, codecVideo, codecAudio, resolution, audioTrackIndex, targetBitrateKbps, subtitleBurnIn)))
return hex.EncodeToString(sum[:])
}
// defaultTranscodeDir is config's TranscodeDir default and roots
// prepared download artifacts when neither download.artifact_dir nor the
// transcode dir is configured. Keeping it absolute avoids writing artifacts
// relative to the process working directory.
const defaultTranscodeDir = config.DefaultTranscodeDir
// effectiveArtifactDir resolves where prepared artifacts are written: the
// configured download.artifact_dir when set, otherwise a dedicated directory
// alongside the transcode dir. The result is always rooted at a real volume,
// never "" (which would land in the process cwd).
//
// Artifacts live as a SIBLING of the transcode dir, never inside it:
// CleanupOrphanedTranscodeDirs deletes every non-active subdirectory of the
// transcode dir, so an artifact dir nested under it would be wiped on the next
// transcode sweep.
func effectiveArtifactDir(artifactDir, transcodeDir string) string {
if artifactDir != "" {
return artifactDir
}
if transcodeDir == "" {
transcodeDir = defaultTranscodeDir
}
return filepath.Join(filepath.Dir(transcodeDir), "silo-download-artifacts")
}
// artifactOutputPath derives a deterministic output path from
// (media_file_id, format, params_hash) so a reclaimed job targets the same file.
func artifactOutputPath(dir string, mediaFileID int, format, hash string) string {
short := hash
if len(short) > 16 {
short = short[:16]
}
return filepath.Join(dir, fmt.Sprintf("%d_%s_%s.mp4", mediaFileID, format, short))
}
+348
View File
@@ -0,0 +1,348 @@
package downloads
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
const artifactColumns = `id, media_file_id, format, params_hash, container, codec_video, codec_audio,
resolution, audio_track_index, target_bitrate_kbps, output_path, file_size, status, error_message,
attempts, max_attempts, lease_owner, lease_expires_at, next_retry_at,
created_at, completed_at, last_used_at`
// ArtifactRepository provides CRUD + durable-queue operations for
// download_artifacts.
type ArtifactRepository struct {
pool *pgxpool.Pool
}
// NewArtifactRepository creates an ArtifactRepository.
func NewArtifactRepository(pool *pgxpool.Pool) *ArtifactRepository {
return &ArtifactRepository{pool: pool}
}
func scanArtifact(row pgx.Row) (*Artifact, error) {
var a Artifact
var leaseOwner *string
if err := row.Scan(
&a.ID, &a.MediaFileID, &a.Format, &a.ParamsHash, &a.Container, &a.CodecVideo, &a.CodecAudio,
&a.Resolution, &a.AudioTrackIndex, &a.TargetBitrateKbps, &a.OutputPath, &a.FileSize, &a.Status, &a.ErrorMessage,
&a.Attempts, &a.MaxAttempts, &leaseOwner, &a.LeaseExpiresAt, &a.NextRetryAt,
&a.CreatedAt, &a.CompletedAt, &a.LastUsedAt,
); err != nil {
return nil, err
}
a.LeaseOwner = deref(leaseOwner)
return &a, nil
}
// EnsureQueued inserts the artifact if no row exists for its
// (media_file_id, format, params_hash), then returns the current row (existing
// or freshly queued) and whether it was newly created.
func (r *ArtifactRepository) EnsureQueued(ctx context.Context, a *Artifact) (*Artifact, bool, error) {
tag, err := r.pool.Exec(ctx,
`INSERT INTO download_artifacts
(id, media_file_id, format, params_hash, container, codec_video, codec_audio,
resolution, audio_track_index, target_bitrate_kbps, output_path, status, max_attempts)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, 'queued', $12)
ON CONFLICT (media_file_id, format, params_hash) DO NOTHING`,
a.ID, a.MediaFileID, a.Format, a.ParamsHash, a.Container, a.CodecVideo, a.CodecAudio,
a.Resolution, a.AudioTrackIndex, a.TargetBitrateKbps, a.OutputPath, a.MaxAttempts,
)
if err != nil {
return nil, false, fmt.Errorf("ensuring artifact: %w", err)
}
row, err := r.GetByKey(ctx, a.MediaFileID, a.Format, a.ParamsHash)
if err != nil {
return nil, false, err
}
return row, tag.RowsAffected() > 0, nil
}
// GetByID returns an artifact by id, or ErrNotFound.
func (r *ArtifactRepository) GetByID(ctx context.Context, id string) (*Artifact, error) {
a, err := scanArtifact(r.pool.QueryRow(ctx, `SELECT `+artifactColumns+` FROM download_artifacts WHERE id = $1`, id))
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("getting artifact: %w", err)
}
return a, nil
}
// GetByKey returns the artifact for a (media_file_id, format, params_hash), or ErrNotFound.
func (r *ArtifactRepository) GetByKey(ctx context.Context, mediaFileID int, format, paramsHash string) (*Artifact, error) {
a, err := scanArtifact(r.pool.QueryRow(ctx,
`SELECT `+artifactColumns+` FROM download_artifacts
WHERE media_file_id = $1 AND format = $2 AND params_hash = $3`,
mediaFileID, format, paramsHash,
))
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("getting artifact by key: %w", err)
}
return a, nil
}
// ClaimNext atomically claims one runnable job: a queued row whose backoff has
// elapsed, or a running row whose lease has expired (lease stealing). FOR UPDATE
// SKIP LOCKED makes concurrent workers (and nodes) safe without double-encoding.
// Returns ErrNoArtifactJob when nothing is claimable.
func (r *ArtifactRepository) ClaimNext(ctx context.Context, owner string, lease time.Duration) (*Artifact, error) {
leaseSecs := int(lease.Seconds())
if leaseSecs <= 0 {
leaseSecs = 60
}
a, err := scanArtifact(r.pool.QueryRow(ctx,
`UPDATE download_artifacts
SET status = 'running', lease_owner = $1, lease_expires_at = now() + make_interval(secs => $2),
attempts = attempts + 1
WHERE id = (
SELECT id FROM download_artifacts
WHERE (status = 'queued' AND (next_retry_at IS NULL OR next_retry_at <= now()))
OR (status = 'running' AND lease_expires_at < now())
ORDER BY created_at
LIMIT 1
FOR UPDATE SKIP LOCKED
)
RETURNING `+artifactColumns,
owner, leaseSecs,
))
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNoArtifactJob
}
return nil, fmt.Errorf("claiming artifact job: %w", err)
}
return a, nil
}
// Heartbeat extends a running job's lease while it is still owned by owner.
// Returns false when the lease was lost (another worker stole it).
func (r *ArtifactRepository) Heartbeat(ctx context.Context, id, owner string, lease time.Duration) (bool, error) {
leaseSecs := int(lease.Seconds())
if leaseSecs <= 0 {
leaseSecs = 60
}
tag, err := r.pool.Exec(ctx,
`UPDATE download_artifacts SET lease_expires_at = now() + make_interval(secs => $3)
WHERE id = $1 AND lease_owner = $2 AND status = 'running'`,
id, owner, leaseSecs,
)
if err != nil {
return false, fmt.Errorf("heartbeating artifact: %w", err)
}
return tag.RowsAffected() > 0, nil
}
// MarkReady transitions a job to ready, records its size/path, and clears the
// lease. The write is fenced on (lease_owner, status='running') so a worker that
// lost its lease — e.g. a slow encode whose lease expired and was reclaimed by
// another node — cannot flip a row it no longer owns. Returns false when the
// fence rejected the write (the lease was lost); the caller must then NOT flip
// linked downloads, leaving that to the current owner.
func (r *ArtifactRepository) MarkReady(ctx context.Context, id, owner, outputPath string, fileSize int64) (bool, error) {
tag, err := r.pool.Exec(ctx,
`UPDATE download_artifacts
SET status = 'ready', output_path = $2, file_size = $3, error_message = '',
completed_at = now(), last_used_at = now(),
lease_owner = NULL, lease_expires_at = NULL, next_retry_at = NULL
WHERE id = $1 AND lease_owner = $4 AND status = 'running'`,
id, outputPath, fileSize, owner,
)
if err != nil {
return false, fmt.Errorf("marking artifact ready: %w", err)
}
return tag.RowsAffected() > 0, nil
}
// MarkFailedOrRetry records a failed attempt. attempts was already incremented
// at claim time, so attempts >= max_attempts means terminal (status=failed);
// otherwise the row returns to queued behind a backoff gate. The write is fenced
// on (lease_owner, status='running'): terminal reports whether the job went
// terminal, and applied is false when the fence rejected the write (lease lost),
// in which case the caller must not fail linked downloads.
func (r *ArtifactRepository) MarkFailedOrRetry(ctx context.Context, id, owner, errMsg string, backoff time.Duration) (terminal bool, applied bool, err error) {
backoffSecs := int(backoff.Seconds())
if backoffSecs <= 0 {
backoffSecs = 30
}
err = r.pool.QueryRow(ctx,
`UPDATE download_artifacts
SET status = CASE WHEN attempts >= max_attempts THEN 'failed' ELSE 'queued' END,
error_message = $2,
next_retry_at = CASE WHEN attempts >= max_attempts THEN NULL ELSE now() + make_interval(secs => $3) END,
completed_at = CASE WHEN attempts >= max_attempts THEN now() ELSE NULL END,
lease_owner = NULL, lease_expires_at = NULL
WHERE id = $1 AND lease_owner = $4 AND status = 'running'
RETURNING status = 'failed'`,
id, errMsg, backoffSecs, owner,
).Scan(&terminal)
if errors.Is(err, pgx.ErrNoRows) {
return false, false, nil // lease lost; the current owner resolves the job
}
if err != nil {
return false, false, fmt.Errorf("marking artifact failed/retry: %w", err)
}
return terminal, true, nil
}
// reclaimedArtifact reports a row recovered by the startup sweep.
type reclaimedArtifact struct {
ID string
Terminal bool // true when the row exhausted attempts and is now failed
}
// ReclaimExpiredLeases resets running rows whose lease has expired: back to
// queued, or to failed when attempts are exhausted. This is the startup sweep
// that guarantees no crash can strand a job in `running` forever.
func (r *ArtifactRepository) ReclaimExpiredLeases(ctx context.Context) ([]reclaimedArtifact, error) {
rows, err := r.pool.Query(ctx,
`UPDATE download_artifacts
SET status = CASE WHEN attempts >= max_attempts THEN 'failed' ELSE 'queued' END,
lease_owner = NULL, lease_expires_at = NULL,
error_message = CASE WHEN attempts >= max_attempts THEN 'exceeded max attempts after lease expiry' ELSE error_message END,
completed_at = CASE WHEN attempts >= max_attempts THEN now() ELSE completed_at END
WHERE status = 'running' AND lease_expires_at < now()
RETURNING id, status = 'failed'`,
)
if err != nil {
return nil, fmt.Errorf("reclaiming expired leases: %w", err)
}
defer rows.Close()
var out []reclaimedArtifact
for rows.Next() {
var rc reclaimedArtifact
if err := rows.Scan(&rc.ID, &rc.Terminal); err != nil {
return nil, fmt.Errorf("scanning reclaimed artifact: %w", err)
}
out = append(out, rc)
}
return out, rows.Err()
}
// Requeue forces a ready/failed artifact back to queued (e.g. when its
// output_path is missing on disk). The deterministic output_path is preserved.
// Returns ErrNotFound when the row no longer exists (e.g. a concurrent sweep
// deleted it) so callers never keep using a dead artifact id.
func (r *ArtifactRepository) Requeue(ctx context.Context, id string) error {
tag, err := r.pool.Exec(ctx,
`UPDATE download_artifacts
SET status = 'queued', attempts = 0, error_message = '', next_retry_at = NULL,
lease_owner = NULL, lease_expires_at = NULL, completed_at = NULL
WHERE id = $1`,
id,
)
if err != nil {
return fmt.Errorf("requeuing artifact: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// TouchLastUsed bumps last_used_at for LRU accounting (called on serve).
func (r *ArtifactRepository) TouchLastUsed(ctx context.Context, id string) error {
_, err := r.pool.Exec(ctx, `UPDATE download_artifacts SET last_used_at = now() WHERE id = $1`, id)
if err != nil {
return fmt.Errorf("touching artifact: %w", err)
}
return nil
}
// ListReady returns ready artifacts ordered by least-recently-used first.
func (r *ArtifactRepository) ListReady(ctx context.Context) ([]*Artifact, error) {
rows, err := r.pool.Query(ctx,
`SELECT `+artifactColumns+` FROM download_artifacts WHERE status = 'ready' ORDER BY last_used_at ASC`)
if err != nil {
return nil, fmt.Errorf("listing ready artifacts: %w", err)
}
defer rows.Close()
return scanArtifacts(rows)
}
func scanArtifacts(rows pgx.Rows) ([]*Artifact, error) {
var out []*Artifact
for rows.Next() {
a, err := scanArtifact(rows)
if err != nil {
return nil, fmt.Errorf("scanning artifact row: %w", err)
}
out = append(out, a)
}
return out, rows.Err()
}
// TotalReadyBytes returns the sum of ready artifact sizes (LRU budget input).
func (r *ArtifactRepository) TotalReadyBytes(ctx context.Context) (int64, error) {
var total int64
if err := r.pool.QueryRow(ctx,
`SELECT COALESCE(SUM(file_size), 0) FROM download_artifacts WHERE status = 'ready'`).Scan(&total); err != nil {
return 0, fmt.Errorf("summing ready artifacts: %w", err)
}
return total, nil
}
// HasActiveLink reports whether any valid download row — managed or ephemeral
// (device-less web) — still references the artifact. Completed rows are
// retained because they remain re-downloadable handles; evicting an artifact a
// live row references would 404 a download the API advertises as servable.
func (r *ArtifactRepository) HasActiveLink(ctx context.Context, artifactID string) (bool, error) {
var exists bool
if err := r.pool.QueryRow(ctx,
`SELECT EXISTS(SELECT 1 FROM downloads
WHERE artifact_id = $1
AND status NOT IN ('cancelled','failed','revoked'))`,
artifactID,
).Scan(&exists); err != nil {
return false, fmt.Errorf("checking artifact links: %w", err)
}
return exists, nil
}
// ListFailedBefore returns terminally-failed artifacts cold since cutoff
// (last_used_at). Their linked downloads were already flipped to 'failed' by
// reconciliation, so the rows serve nothing and only block re-attempts.
func (r *ArtifactRepository) ListFailedBefore(ctx context.Context, cutoff time.Time) ([]*Artifact, error) {
rows, err := r.pool.Query(ctx,
`SELECT `+artifactColumns+` FROM download_artifacts
WHERE status = 'failed' AND last_used_at < $1`, cutoff)
if err != nil {
return nil, fmt.Errorf("listing failed artifacts: %w", err)
}
defer rows.Close()
return scanArtifacts(rows)
}
// ListUnlinkedReadyBefore returns ready artifacts referenced by NO download
// row at all (every linking row deleted) and unused since cutoff. These are
// pure orphans: nothing can ever serve them again.
func (r *ArtifactRepository) ListUnlinkedReadyBefore(ctx context.Context, cutoff time.Time) ([]*Artifact, error) {
rows, err := r.pool.Query(ctx,
`SELECT `+artifactColumns+` FROM download_artifacts a
WHERE a.status = 'ready' AND a.last_used_at < $1
AND NOT EXISTS (SELECT 1 FROM downloads d WHERE d.artifact_id = a.id)`, cutoff)
if err != nil {
return nil, fmt.Errorf("listing unlinked artifacts: %w", err)
}
defer rows.Close()
return scanArtifacts(rows)
}
// DeleteArtifact removes an artifact row.
func (r *ArtifactRepository) DeleteArtifact(ctx context.Context, id string) error {
_, err := r.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE id = $1`, id)
if err != nil {
return fmt.Errorf("deleting artifact: %w", err)
}
return nil
}
+257
View File
@@ -0,0 +1,257 @@
package downloads
import (
"context"
"errors"
"fmt"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/idgen"
)
func newArtifactTestRepo(t *testing.T) (*ArtifactRepository, *pgxpool.Pool, int) {
t.Helper()
pool := newDownloadsTestPool(t)
ctx := context.Background()
var present *string
if err := pool.QueryRow(ctx, `SELECT to_regclass('public.download_artifacts')::text`).Scan(&present); err != nil {
t.Fatalf("check download_artifacts: %v", err)
}
if present == nil {
t.Skip("download_artifacts migration has not been applied")
}
suffix := time.Now().UnixNano()
var folderID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_folders (type, name) VALUES ('movies', $1) RETURNING id`,
fmt.Sprintf("Artifacts Test %d", suffix),
).Scan(&folderID); err != nil {
t.Fatalf("seed folder: %v", err)
}
var fileID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_files (media_folder_id, file_path) VALUES ($1, $2) RETURNING id`,
folderID, fmt.Sprintf("/tmp/artifact-%d.mkv", suffix),
).Scan(&fileID); err != nil {
t.Fatalf("seed media file: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
return NewArtifactRepository(pool), pool, fileID
}
func newArtifact(t *testing.T, fileID int, hash string) *Artifact {
t.Helper()
id, err := idgen.NextID()
if err != nil {
t.Fatalf("id: %v", err)
}
return &Artifact{
ID: id, MediaFileID: fileID, Format: "transcode", ParamsHash: hash,
Container: "mp4", CodecVideo: "h264", CodecAudio: "aac", Resolution: "1080p",
AudioTrackIndex: -1, OutputPath: "/tmp/" + id + ".mp4", MaxAttempts: 3,
}
}
// TestArtifactQueueClaimAndLeaseRecovery is the Phase 3 / invariant-3 acceptance
// test: a crash mid-encode (an expired lease) is recovered on the next sweep so
// the job re-enqueues and reaches ready, and concurrent workers never claim the
// same job twice (no double-encode).
func TestArtifactQueueClaimAndLeaseRecovery(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
a := newArtifact(t, fileID, "hash-recovery")
row, created, err := repo.EnsureQueued(ctx, a)
if err != nil || !created || row.Status != ArtifactQueued {
t.Fatalf("EnsureQueued = (%+v, created=%v, %v), want new queued row", row, created, err)
}
// Dedup: a second ensure for the same key returns the same row, not a new one.
dup, created2, err := repo.EnsureQueued(ctx, newArtifact(t, fileID, "hash-recovery"))
if err != nil || created2 || dup.ID != row.ID {
t.Fatalf("dedup EnsureQueued = (%s, created=%v, %v), want existing %s", dup.ID, created2, err, row.ID)
}
// Worker 1 claims the job; worker 2 finds nothing (no double-encode).
claim, err := repo.ClaimNext(ctx, "worker-1", time.Minute)
if err != nil || claim.ID != row.ID || claim.Status != ArtifactRunning || claim.Attempts != 1 {
t.Fatalf("ClaimNext = (%+v, %v), want running attempts=1", claim, err)
}
if _, err := repo.ClaimNext(ctx, "worker-2", time.Minute); !errors.Is(err, ErrNoArtifactJob) {
t.Fatalf("second ClaimNext err = %v, want ErrNoArtifactJob", err)
}
// Simulate a crash: expire the lease, then run the startup sweep.
if _, err := pool.Exec(ctx, `UPDATE download_artifacts SET lease_expires_at = now() - interval '1 minute' WHERE id = $1`, row.ID); err != nil {
t.Fatalf("expire lease: %v", err)
}
reclaimed, err := repo.ReclaimExpiredLeases(ctx)
if err != nil {
t.Fatalf("reclaim: %v", err)
}
if len(reclaimed) != 1 || reclaimed[0].ID != row.ID || reclaimed[0].Terminal {
t.Fatalf("reclaimed = %+v, want one non-terminal %s", reclaimed, row.ID)
}
back, err := repo.GetByID(ctx, row.ID)
if err != nil || back.Status != ArtifactQueued {
t.Fatalf("after reclaim status = %v (%v), want queued (no permanent running)", back.Status, err)
}
// Another worker reclaims and completes it.
claim2, err := repo.ClaimNext(ctx, "worker-2", time.Minute)
if err != nil || claim2.ID != row.ID || claim2.Attempts != 2 {
t.Fatalf("reclaim ClaimNext = (%+v, %v), want attempts=2", claim2, err)
}
if applied, err := repo.MarkReady(ctx, row.ID, "worker-2", claim2.OutputPath, 4242); err != nil || !applied {
t.Fatalf("MarkReady = (%v, %v), want (true, nil)", applied, err)
}
done, err := repo.GetByKey(ctx, fileID, "transcode", "hash-recovery")
if err != nil || done.Status != ArtifactReady || done.FileSize != 4242 {
t.Fatalf("final = (%+v, %v), want ready size=4242", done, err)
}
}
// TestArtifactRetryUntilTerminal verifies attempt counting and backoff: a job
// retries behind its backoff gate until max_attempts, then goes terminal-failed.
func TestArtifactRetryUntilTerminal(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
row, _, err := repo.EnsureQueued(ctx, newArtifact(t, fileID, "hash-retry"))
if err != nil {
t.Fatalf("EnsureQueued: %v", err)
}
for attempt := 1; attempt <= 3; attempt++ {
claim, err := repo.ClaimNext(ctx, "worker", time.Minute)
if err != nil {
t.Fatalf("attempt %d ClaimNext: %v", attempt, err)
}
if claim.Attempts != attempt {
t.Fatalf("attempt %d: attempts = %d", attempt, claim.Attempts)
}
terminal, applied, err := repo.MarkFailedOrRetry(ctx, row.ID, "worker", "boom", 30*time.Second)
if err != nil || !applied {
t.Fatalf("attempt %d MarkFailedOrRetry = (%v, %v, %v)", attempt, terminal, applied, err)
}
if attempt < 3 {
if terminal {
t.Fatalf("attempt %d went terminal too early", attempt)
}
// Behind the backoff gate the job is not yet claimable.
if _, err := repo.ClaimNext(ctx, "worker", time.Minute); !errors.Is(err, ErrNoArtifactJob) {
t.Fatalf("attempt %d: job claimable during backoff", attempt)
}
if _, err := pool.Exec(ctx, `UPDATE download_artifacts SET next_retry_at = now() - interval '1 second' WHERE id = $1`, row.ID); err != nil {
t.Fatalf("clear backoff: %v", err)
}
} else if !terminal {
t.Fatalf("final attempt should be terminal")
}
}
failed, err := repo.GetByID(ctx, row.ID)
if err != nil || failed.Status != ArtifactFailed {
t.Fatalf("final status = %v (%v), want failed", failed.Status, err)
}
}
// TestArtifactMarkFencedByOwner verifies MarkReady/MarkFailedOrRetry only apply
// for the worker that currently holds the lease, so a worker whose lease was
// stolen (e.g. a slow encode reclaimed by another node) cannot flip a job it no
// longer owns — the double-encode guard behind invariant 3.
func TestArtifactMarkFencedByOwner(t *testing.T) {
repo, _, fileID := newArtifactTestRepo(t)
ctx := context.Background()
row, _, err := repo.EnsureQueued(ctx, newArtifact(t, fileID, "hash-fence"))
if err != nil {
t.Fatalf("EnsureQueued: %v", err)
}
if _, err := repo.ClaimNext(ctx, "owner-1", time.Minute); err != nil {
t.Fatalf("ClaimNext: %v", err)
}
// A non-owner cannot mark the job ready or failed.
if applied, err := repo.MarkReady(ctx, row.ID, "owner-2", "/tmp/x.mp4", 10); err != nil || applied {
t.Fatalf("MarkReady(non-owner) = (%v, %v), want (false, nil)", applied, err)
}
if _, applied, err := repo.MarkFailedOrRetry(ctx, row.ID, "owner-2", "boom", time.Second); err != nil || applied {
t.Fatalf("MarkFailedOrRetry(non-owner) applied = %v (%v), want false", applied, err)
}
// The job remains claimable-state 'running' and untouched.
mid, err := repo.GetByID(ctx, row.ID)
if err != nil || mid.Status != ArtifactRunning {
t.Fatalf("status after fenced writes = %v (%v), want running", mid.Status, err)
}
// The real owner succeeds.
if applied, err := repo.MarkReady(ctx, row.ID, "owner-1", "/tmp/x.mp4", 10); err != nil || !applied {
t.Fatalf("MarkReady(owner) = (%v, %v), want (true, nil)", applied, err)
}
}
// TestHasActiveLinkCoversEphemeralRows pins the eviction guard: an ephemeral
// (device-less web) download row must protect its artifact from LRU cleanup
// exactly like a managed row does, and terminal rows must not.
func TestHasActiveLinkCoversEphemeralRows(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
art := newArtifact(t, fileID, fmt.Sprintf("hash-link-%d", time.Now().UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("ensure artifact: %v", err)
}
var userID int
if err := pool.QueryRow(ctx,
`INSERT INTO users (username, role, download_allowed) VALUES ($1, 'user', true) RETURNING id`,
fmt.Sprintf("linkuser-%d", time.Now().UnixNano()),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
contentID := fmt.Sprintf("link-content-%d", time.Now().UnixNano())
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM downloads WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, userID)
})
dlRepo := NewRepository(pool)
now := time.Now()
dlID := fmt.Sprintf("dl-link-%d", now.UnixNano())
if err := dlRepo.Create(ctx, &Download{
ID: dlID, UserID: userID, MediaFileID: fileID, ContentID: contentID,
Kind: KindQueued, Status: StatusReady, Format: FormatTranscode,
ArtifactID: art.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create ephemeral download: %v", err)
}
active, err := repo.HasActiveLink(ctx, art.ID)
if err != nil {
t.Fatalf("HasActiveLink: %v", err)
}
if !active {
t.Fatal("ephemeral ready row must protect its artifact from eviction")
}
if _, err := pool.Exec(ctx, `UPDATE downloads SET status = 'cancelled' WHERE id = $1`, dlID); err != nil {
t.Fatalf("cancel download: %v", err)
}
active, err = repo.HasActiveLink(ctx, art.ID)
if err != nil {
t.Fatalf("HasActiveLink after cancel: %v", err)
}
if active {
t.Fatal("terminal-only links must not protect an artifact")
}
}
+155
View File
@@ -0,0 +1,155 @@
package downloads
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestParamsHashStableAndDistinct(t *testing.T) {
base := paramsHash("transcode", "mp4", "h264", "aac", "1080p", -1, 10000, false)
if base == "" || len(base) != 64 {
t.Fatalf("params hash should be a 64-char sha256 hex, got %q", base)
}
// Deterministic for identical inputs (dedup key).
if again := paramsHash("transcode", "mp4", "h264", "aac", "1080p", -1, 10000, false); again != base {
t.Fatalf("params hash not stable: %q != %q", base, again)
}
// Distinct when any parameter differs.
for _, other := range []string{
paramsHash("remux", "mp4", "h264", "aac", "1080p", -1, 10000, false),
paramsHash("transcode", "mp4", "hevc", "aac", "1080p", -1, 10000, false),
paramsHash("transcode", "mp4", "h264", "aac", "720p", -1, 10000, false),
paramsHash("transcode", "mp4", "h264", "aac", "1080p", 1, 10000, false),
paramsHash("transcode", "mp4", "h264", "aac", "1080p", -1, 5000, false),
paramsHash("transcode", "mp4", "h264", "aac", "1080p", -1, 10000, true),
} {
if other == base {
t.Fatalf("params hash collision: %q", other)
}
}
}
func TestArtifactOutputPathDeterministic(t *testing.T) {
p1 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef")
p2 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef")
if p1 != p2 {
t.Fatalf("output path not deterministic: %q != %q", p1, p2)
}
if !strings.HasPrefix(p1, "/var/artifacts/") || !strings.HasSuffix(p1, ".mp4") {
t.Fatalf("unexpected output path %q", p1)
}
if !strings.Contains(p1, "42_transcode_") {
t.Fatalf("output path missing identity components: %q", p1)
}
}
func TestEffectiveArtifactDir(t *testing.T) {
// Explicit config wins verbatim.
if got := effectiveArtifactDir("/data/artifacts", "/tmp/silo-transcode"); got != "/data/artifacts" {
t.Fatalf("explicit dir = %q, want /data/artifacts", got)
}
// Unset: a sibling of the transcode dir, never inside it (the transcode
// cleanup sweep would otherwise delete a nested artifact dir) and never the
// process cwd (a relative/empty path).
got := effectiveArtifactDir("", "/var/lib/silo/transcode")
if got != "/var/lib/silo/silo-download-artifacts" {
t.Fatalf("default dir = %q, want sibling of transcode dir", got)
}
if strings.HasPrefix(got, "/var/lib/silo/transcode/") {
t.Fatalf("default dir %q is nested under the transcode dir", got)
}
// Unset transcode dir falls back to the absolute default root, not "".
fallback := effectiveArtifactDir("", "")
if !strings.HasPrefix(fallback, "/") {
t.Fatalf("fallback dir %q is not absolute", fallback)
}
}
type fakeUserRepo struct{ user *models.User }
func (f fakeUserRepo) GetByID(context.Context, int) (*models.User, error) { return f.user, nil }
func TestCapabilityQualityPresetsGating(t *testing.T) {
newSvc := func(user *models.User, transcodeEnabled bool) *Service {
cfg := config.DownloadConfig{Enabled: true, TranscodeEnabled: transcodeEnabled}
return NewService(nil, nil, nil, nil, nil, nil, fakeUserRepo{user}, nil, nil, &cfg)
}
allowAll := &models.User{DownloadAllowed: true, DownloadTranscodeAllowed: true}
// No artifact pipeline wired → only original is fulfillable.
svc := newSvc(allowAll, true)
capInfo, err := svc.Capability(context.Background(), 1)
if err != nil {
t.Fatal(err)
}
if got := strings.Join(capInfo.QualityPresets, ","); got != "original" {
t.Fatalf("quality presets without pipeline = %q, want original", got)
}
// Pipeline wired + transcode server/user gates open → full bitrate ladder.
svc = newSvc(allowAll, true)
svc.SetArtifactManager(&ArtifactManager{})
capInfo, _ = svc.Capability(context.Background(), 1)
if got := strings.Join(capInfo.QualityPresets, ","); got != "original,20mbps,10mbps,5mbps,2mbps,1mbps" {
t.Fatalf("quality presets with pipeline = %q, want full ladder", got)
}
// Transcode gated off (user flag) → original only.
svc = newSvc(&models.User{DownloadAllowed: true, DownloadTranscodeAllowed: false}, true)
svc.SetArtifactManager(&ArtifactManager{})
capInfo, _ = svc.Capability(context.Background(), 1)
if got := strings.Join(capInfo.QualityPresets, ","); got != "original" {
t.Fatalf("quality presets with transcode gated = %q, want original", got)
}
// Download permission revoked → an EMPTY array, never nil: the capability
// contract documents quality_presets as an array, and a nil slice would
// serialize as JSON null and break typed clients.
svc = newSvc(&models.User{DownloadAllowed: false}, true)
capInfo, _ = svc.Capability(context.Background(), 1)
if capInfo.QualityPresets == nil {
t.Fatal("quality presets for a denied user must be an empty array, not nil")
}
if len(capInfo.QualityPresets) != 0 {
t.Fatalf("quality presets for a denied user = %v, want empty", capInfo.QualityPresets)
}
if b, err := json.Marshal(capInfo.QualityPresets); err != nil || string(b) != "[]" {
t.Fatalf("quality presets serialize to %s (%v), want []", b, err)
}
}
// TestTriggerDrainDoesNotBlockCaller pins the async-dispatch contract:
// Ensure runs on request goroutines and the kick drains the whole encode
// queue (ffmpeg included), so triggerDrain must return without waiting on it.
func TestTriggerDrainDoesNotBlockCaller(t *testing.T) {
m := &ArtifactManager{}
started := make(chan struct{})
release := make(chan struct{})
m.SetKick(func() {
close(started)
<-release
})
done := make(chan struct{})
go func() {
m.triggerDrain()
close(done)
}()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("triggerDrain blocked on the kick; it must dispatch asynchronously")
}
select {
case <-started:
case <-time.After(2 * time.Second):
t.Fatal("kick was never invoked")
}
close(release)
}
+582
View File
@@ -0,0 +1,582 @@
package downloads
import (
"context"
"errors"
"fmt"
"log/slog"
"os"
"sync"
"time"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/idgen"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
const (
artifactLease = 2 * time.Minute
artifactHeartbeat = 40 * time.Second
artifactMaxAttempts = 3
)
// EncodePreparer produces a single finalized file for an artifact. The default
// implementation calls playback.PrepareFile; tests substitute a fake.
type EncodePreparer interface {
PrepareFile(ctx context.Context, opts playback.TranscodeOpts, outputPath string) error
}
type playbackPreparer struct{}
func (playbackPreparer) PrepareFile(ctx context.Context, opts playback.TranscodeOpts, outputPath string) error {
return playback.PrepareFile(ctx, opts, outputPath)
}
// NewPlaybackPreparer returns the production EncodePreparer (ffmpeg-backed).
func NewPlaybackPreparer() EncodePreparer { return playbackPreparer{} }
// ArtifactNotifier publishes an event when a linked download changes state.
type ArtifactNotifier func(ctx context.Context, d *Download)
// ArtifactManager owns the durable encode queue: it ensures/deduplicates encode
// jobs, drains them through a bounded worker pool with leased heartbeats, and
// recovers stranded jobs on startup.
type ArtifactManager struct {
repo *ArtifactRepository
downloads *Repository
fileRepo FileResolver
preparer EncodePreparer
owner string
liveCfg func() *config.Config
notify ArtifactNotifier
mu sync.Mutex
kick func()
lastDiskSweep time.Time
lastStaleSweep time.Time
}
// maintenanceInterval spaces the disk-presence and stale-row sweeps: both are
// O(cache size) (stats / extra queries) and their failure modes self-heal, so
// running them on every 30s task tick is steady-state waste that grows with
// the artifact cache. The first run after startup always executes.
const maintenanceInterval = time.Hour
// maintenanceDue reports whether the sweep guarded by last is due, advancing
// the stamp when it is.
func (m *ArtifactManager) maintenanceDue(last *time.Time) bool {
m.mu.Lock()
defer m.mu.Unlock()
if !last.IsZero() && time.Since(*last) < maintenanceInterval {
return false
}
*last = time.Now()
return true
}
// NewArtifactManager constructs an ArtifactManager. liveCfg reads the current
// config (artifact dir, worker-pool size, byte budget, ffmpeg/hwaccel); owner is
// this node's id for lease ownership; notify (optional) publishes ready/failed.
func NewArtifactManager(
repo *ArtifactRepository,
downloadRepo *Repository,
fileRepo FileResolver,
preparer EncodePreparer,
owner string,
liveCfg func() *config.Config,
notify ArtifactNotifier,
) *ArtifactManager {
if preparer == nil {
preparer = playbackPreparer{}
}
if owner == "" {
owner = "node"
}
return &ArtifactManager{
repo: repo, downloads: downloadRepo, fileRepo: fileRepo, preparer: preparer,
owner: owner, liveCfg: liveCfg, notify: notify,
}
}
// SetKick wires a low-latency drain trigger (e.g. taskmanager RunTask) invoked
// when a new job is enqueued.
func (m *ArtifactManager) SetKick(kick func()) {
m.mu.Lock()
m.kick = kick
m.mu.Unlock()
}
// Ready returns a ready artifact for serving and bumps its LRU timestamp.
// Returns ErrDownloadNotActive when the artifact is not yet ready.
func (m *ArtifactManager) Ready(ctx context.Context, id string) (*Artifact, error) {
a, err := m.repo.GetByID(ctx, id)
if err != nil {
return nil, err
}
if a.Status != ArtifactReady {
return nil, fmt.Errorf("artifact is %s: %w", a.Status, ErrDownloadNotActive)
}
_ = m.repo.TouchLastUsed(ctx, id)
return a, nil
}
func (m *ArtifactManager) downloadConfig() config.DownloadConfig {
if m.liveCfg != nil {
if c := m.liveCfg(); c != nil {
return c.Download
}
}
return config.DownloadConfig{}
}
// artifactDir resolves the effective output directory for prepared artifacts,
// defaulting under the transcode dir when download.artifact_dir is unset so
// encodes never write relative to the process working directory.
func (m *ArtifactManager) artifactDir() string {
var artifactDir, transcodeDir string
if m.liveCfg != nil {
if c := m.liveCfg(); c != nil {
artifactDir = c.Download.ArtifactDir
transcodeDir = c.Playback.TranscodeDir
}
}
return effectiveArtifactDir(artifactDir, transcodeDir)
}
// Ensure deduplicates and (when new) enqueues an encode job for file in the
// given format, returning the current artifact row. The deterministic
// output_path keeps a reclaimed job idempotent.
func (m *ArtifactManager) Ensure(ctx context.Context, file *models.MediaFile, format string, target playback.PrepareTarget) (*Artifact, error) {
hash := paramsHash(format, target.Container, target.CodecVideo, target.CodecAudio, target.Resolution, target.AudioTrackIndex, target.TargetBitrateKbps, false)
id, err := idgen.NextID()
if err != nil {
return nil, err
}
a := &Artifact{
ID: id,
MediaFileID: file.ID,
Format: format,
ParamsHash: hash,
Container: target.Container,
CodecVideo: target.CodecVideo,
CodecAudio: target.CodecAudio,
Resolution: target.Resolution,
AudioTrackIndex: target.AudioTrackIndex,
TargetBitrateKbps: target.TargetBitrateKbps,
OutputPath: artifactOutputPath(m.artifactDir(), file.ID, format, hash),
MaxAttempts: artifactMaxAttempts,
}
row, created, err := m.repo.EnsureQueued(ctx, a)
if err != nil {
return nil, err
}
if row.Status == ArtifactReady {
_ = m.repo.TouchLastUsed(ctx, row.ID)
return row, nil
}
// A terminally-failed dedup row would otherwise strand every new download
// linked to it in 'preparing' forever (no drain is triggered for an existing
// row). Requeue it for a fresh attempt so the new download can resolve — or
// fail cleanly via reconciliation once the encode is exhausted again.
if row.Status == ArtifactFailed {
switch err := m.repo.Requeue(ctx, row.ID); {
case errors.Is(err, ErrNotFound):
// The failed row was swept between EnsureQueued and Requeue:
// create a fresh job instead of linking to a dead artifact id.
if row, _, err = m.repo.EnsureQueued(ctx, a); err != nil {
return nil, err
}
case err != nil:
return nil, err
default:
row.Status = ArtifactQueued
}
m.triggerDrain()
return row, nil
}
if created {
m.triggerDrain()
}
return row, nil
}
func (m *ArtifactManager) triggerDrain() {
m.mu.Lock()
kick := m.kick
m.mu.Unlock()
if kick != nil {
// Ensure is called on request goroutines and the kick runs the encode
// task to completion (the task manager serializes concurrent runs), so
// it must never execute inline: a POST /downloads would otherwise block
// on the entire queue drain, ffmpeg encodes included.
go kick()
}
}
// RunOnce performs a startup-safe recovery sweep and then drains the queue
// until empty. It is safe to call concurrently across nodes; FOR UPDATE SKIP
// LOCKED prevents double-encoding.
func (m *ArtifactManager) RunOnce(ctx context.Context) error {
m.recover(ctx)
return m.drain(ctx)
}
// recover repairs state a crash or lost lease can leave behind: it reclaims
// expired-lease running rows (back to queued, or failed when attempts are
// exhausted), reconciles linked downloads against terminal artifact states, and
// re-queues ready artifacts whose output file is missing on disk. Safe to run
// repeatedly (each step is idempotent).
func (m *ArtifactManager) recover(ctx context.Context) {
if _, err := m.repo.ReclaimExpiredLeases(ctx); err != nil {
slog.Warn("download artifact lease reclaim failed", "error", err)
}
// Reconcile downloads stranded in 'preparing' against their artifact's
// terminal state: this closes the non-transactional window between an
// artifact's MarkReady and its MarkLinkedDownloadsReady, and fails the links
// of any artifact that reached 'failed' (including the rows just reclaimed to
// failed above) so a download can never sit 'preparing' forever.
readyFlipped, failedFlipped, err := m.downloads.ReconcileLinkedDownloads(ctx)
if err != nil {
slog.Warn("reconciling linked downloads failed", "error", err)
} else {
for _, d := range readyFlipped {
m.publish(ctx, d)
}
for _, d := range failedFlipped {
m.publish(ctx, d)
}
}
// Disk-presence sweep: stats every ready file, so it runs on the startup
// pass and then hourly rather than on every tick.
if !m.maintenanceDue(&m.lastDiskSweep) {
return
}
ready, err := m.repo.ListReady(ctx)
if err != nil {
slog.Warn("download artifact ready scan failed", "error", err)
return
}
for _, a := range ready {
if a.OutputPath == "" {
continue
}
if _, statErr := os.Stat(a.OutputPath); statErr != nil {
slog.Warn("download artifact output missing, re-queuing", "artifact_id", a.ID, "path", a.OutputPath)
if err := m.repo.Requeue(ctx, a.ID); err != nil {
slog.Warn("re-queue artifact failed", "artifact_id", a.ID, "error", err)
continue
}
m.triggerDrain()
}
}
}
// drain claims and encodes jobs through a bounded worker pool until the queue is
// empty or the context is canceled.
func (m *ArtifactManager) drain(ctx context.Context) error {
maxConcurrent := m.downloadConfig().MaxConcurrentPrepares
if maxConcurrent <= 0 {
maxConcurrent = 2
}
sem := make(chan struct{}, maxConcurrent)
var wg sync.WaitGroup
for {
// Acquire a worker slot BEFORE claiming. A claimed job is leased but only
// heartbeated once encodeOne runs; claiming first and then blocking for a
// slot would leave the job leased-but-unattended, so its lease could lapse
// while it waits — letting another node steal it and encode the same
// output path concurrently. Reserving the slot first closes that window.
select {
case sem <- struct{}{}:
case <-ctx.Done():
wg.Wait()
return ctx.Err()
}
job, err := m.repo.ClaimNext(ctx, m.owner, artifactLease)
if err != nil {
<-sem // release the slot we reserved but won't use
if errors.Is(err, ErrNoArtifactJob) {
break
}
wg.Wait()
return err // includes context cancellation (pgx honors ctx)
}
wg.Add(1)
go func(a *Artifact) {
defer wg.Done()
defer func() { <-sem }()
m.encodeOne(ctx, a)
}(job)
}
wg.Wait()
return nil
}
// encodeOne runs one claimed job to completion, extending its lease via a
// heartbeat, and links/notifies the dependent download rows on the outcome.
func (m *ArtifactManager) encodeOne(ctx context.Context, a *Artifact) {
hbCtx, cancelHB := context.WithCancel(ctx)
defer cancelHB()
// heartbeatLoop cancels hbCtx if the lease is lost; PrepareFile runs on hbCtx
// so that cancellation aborts ffmpeg, ensuring we never keep writing the
// output path after another worker has taken the job.
go m.heartbeatLoop(hbCtx, cancelHB, a.ID)
file, err := m.fileRepo.GetByID(ctx, a.MediaFileID)
if err != nil || file == nil {
m.failJob(ctx, a, "source media file unavailable")
return
}
opts := m.buildOpts(file, a)
if err := m.preparer.PrepareFile(hbCtx, opts, a.OutputPath); err != nil {
switch {
case ctx.Err() != nil:
// Parent shutting down: leave the job 'running'; its lease expires and
// recovery (here or on another node) reclaims it.
return
case hbCtx.Err() != nil:
// We lost the lease mid-encode; another worker now owns the job.
slog.Warn("download artifact encode aborted; lease lost", "artifact_id", a.ID)
return
default:
slog.Warn("download artifact encode failed", "artifact_id", a.ID, "error", err)
m.failJob(ctx, a, err.Error())
return
}
}
var size int64
if fi, statErr := os.Stat(a.OutputPath); statErr == nil {
size = fi.Size()
}
// Fenced on lease ownership: if we lost the lease between encode and commit,
// applied is false and the current owner is responsible for flipping links —
// do not flip them here or we would race/duplicate that owner's work.
applied, err := m.repo.MarkReady(ctx, a.ID, m.owner, a.OutputPath, size)
if err != nil {
slog.Error("marking artifact ready failed", "artifact_id", a.ID, "error", err)
return
}
if !applied {
slog.Warn("download artifact ready skipped; lease lost", "artifact_id", a.ID)
return
}
flipped, err := m.downloads.MarkLinkedDownloadsReady(ctx, a.ID, size)
if err != nil {
slog.Error("flipping linked downloads ready failed", "artifact_id", a.ID, "error", err)
return
}
for _, d := range flipped {
m.publish(ctx, d)
}
}
func (m *ArtifactManager) failJob(ctx context.Context, a *Artifact, msg string) {
terminal, applied, err := m.repo.MarkFailedOrRetry(ctx, a.ID, m.owner, msg, backoffFor(a.Attempts))
if err != nil {
slog.Error("marking artifact failed/retry errored", "artifact_id", a.ID, "error", err)
return
}
if !applied {
// Lease lost; the current owner is responsible for the job's outcome.
return
}
if terminal {
m.failLinkedDownloads(ctx, a.ID, msg)
} else {
m.triggerDrain()
}
}
func (m *ArtifactManager) failLinkedDownloads(ctx context.Context, artifactID, msg string) {
flipped, err := m.downloads.MarkLinkedDownloadsFailed(ctx, artifactID, msg)
if err != nil {
slog.Error("flipping linked downloads failed errored", "artifact_id", artifactID, "error", err)
return
}
for _, d := range flipped {
m.publish(ctx, d)
}
}
func (m *ArtifactManager) publish(ctx context.Context, d *Download) {
if m.notify != nil {
m.notify(ctx, d)
}
}
// heartbeatLoop extends the job's lease until ctx is done. If the lease is lost
// (another worker stole it, or the row is gone) it calls cancel to abort the
// encode so two workers never write the same output path. A transient DB error
// is retried on the next tick rather than aborting a healthy encode.
func (m *ArtifactManager) heartbeatLoop(ctx context.Context, cancel context.CancelFunc, id string) {
ticker := time.NewTicker(artifactHeartbeat)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
ok, err := m.repo.Heartbeat(ctx, id, m.owner, artifactLease)
switch {
case err != nil && ctx.Err() != nil:
return // encode finished or shutting down
case err != nil:
slog.Warn("download artifact heartbeat errored", "artifact_id", id, "error", err)
case !ok:
slog.Warn("download artifact lease lost; aborting encode", "artifact_id", id)
cancel()
return
}
}
}
}
func (m *ArtifactManager) buildOpts(file *models.MediaFile, a *Artifact) playback.TranscodeOpts {
cfg := config.Config{}
if m.liveCfg != nil {
if c := m.liveCfg(); c != nil {
cfg = *c
}
}
return playback.TranscodeOpts{
InputPath: file.FilePath,
SourceVideoCodec: file.CodecVideo,
TargetCodecVideo: a.CodecVideo,
TargetCodecAudio: a.CodecAudio,
TargetResolution: a.Resolution,
TargetBitrateKbps: a.TargetBitrateKbps,
AudioTrackIndex: a.AudioTrackIndex,
SubtitleTrackIndex: -1,
FFmpegPath: cfg.Playback.FFmpegPath,
HWAccel: cfg.Playback.HWAccel,
HWDevice: cfg.Playback.HWDevice,
TotalDuration: float64(file.Duration),
}
}
// Hygiene retention windows. These remove only rows nothing can serve again —
// terminally-failed jobs (linked downloads already flipped to failed by
// reconciliation) and ready artifacts whose every referencing download row was
// deleted — plus ephemeral web rows past their convenience-record lifetime.
// The server-disk *quota* is download.artifact_max_bytes (see the download
// limits & restrictions design); this sweep is not a quota.
const (
failedArtifactRetention = 24 * time.Hour
unlinkedArtifactRetention = 30 * 24 * time.Hour
ephemeralDownloadRetention = 7 * 24 * time.Hour
)
// Cleanup runs the hygiene sweep, then evicts ready artifacts (LRU first) once
// the total exceeds the byte budget, never removing one still linked by any
// active download row (managed or ephemeral) — only artifacts whose links are
// all terminal are evictable.
func (m *ArtifactManager) Cleanup(ctx context.Context) error {
m.sweepStale(ctx)
budget := m.downloadConfig().ArtifactMaxBytes
if budget <= 0 {
return nil // unlimited
}
total, err := m.repo.TotalReadyBytes(ctx)
if err != nil {
return err
}
if total <= budget {
return nil
}
candidates, err := m.repo.ListReady(ctx) // least-recently-used first
if err != nil {
return err
}
for _, a := range candidates {
if total <= budget {
break
}
active, err := m.repo.HasActiveLink(ctx, a.ID)
if err != nil {
slog.Warn("artifact link check failed", "artifact_id", a.ID, "error", err)
continue
}
if active {
continue
}
if a.OutputPath != "" {
if err := os.Remove(a.OutputPath); err != nil && !os.IsNotExist(err) {
slog.Warn("removing evicted artifact file failed", "artifact_id", a.ID, "error", err)
}
}
if err := m.repo.DeleteArtifact(ctx, a.ID); err != nil {
slog.Warn("deleting evicted artifact row failed", "artifact_id", a.ID, "error", err)
continue
}
slog.Info("evicted download artifact (LRU)", "artifact_id", a.ID, "bytes", a.FileSize)
total -= a.FileSize
}
return nil
}
// sweepStale is the age-based hygiene pass: cold terminally-failed artifacts
// (with their leftover .part files), orphaned ready artifacts no download row
// references, and expired ephemeral download rows. Best-effort; every step
// logs and continues.
func (m *ArtifactManager) sweepStale(ctx context.Context) {
if !m.maintenanceDue(&m.lastStaleSweep) {
return
}
now := time.Now()
if failed, err := m.repo.ListFailedBefore(ctx, now.Add(-failedArtifactRetention)); err != nil {
slog.Warn("failed-artifact sweep list failed", "error", err)
} else {
for _, a := range failed {
m.removeArtifact(ctx, a, "failed")
}
}
if orphans, err := m.repo.ListUnlinkedReadyBefore(ctx, now.Add(-unlinkedArtifactRetention)); err != nil {
slog.Warn("unlinked-artifact sweep list failed", "error", err)
} else {
for _, a := range orphans {
m.removeArtifact(ctx, a, "unlinked")
}
}
if m.downloads != nil {
if n, err := m.downloads.PruneEphemeralOlderThan(ctx, now.Add(-ephemeralDownloadRetention)); err != nil {
slog.Warn("ephemeral download prune failed", "error", err)
} else if n > 0 {
slog.Info("pruned expired ephemeral downloads", "rows", n)
}
}
}
// removeArtifact deletes an artifact's output file, its .part leftover, and
// its row. Used by the hygiene sweep for rows nothing can serve again.
func (m *ArtifactManager) removeArtifact(ctx context.Context, a *Artifact, reason string) {
if a.OutputPath != "" {
if err := os.Remove(a.OutputPath); err != nil && !os.IsNotExist(err) {
slog.Warn("removing swept artifact file failed", "artifact_id", a.ID, "error", err)
}
if err := os.Remove(a.OutputPath + ".part"); err != nil && !os.IsNotExist(err) {
slog.Warn("removing swept artifact partial failed", "artifact_id", a.ID, "error", err)
}
}
if err := m.repo.DeleteArtifact(ctx, a.ID); err != nil {
slog.Warn("deleting swept artifact row failed", "artifact_id", a.ID, "error", err)
return
}
slog.Info("swept stale download artifact", "artifact_id", a.ID, "reason", reason, "bytes", a.FileSize)
}
// backoffFor returns the retry delay for the next attempt after a failure.
func backoffFor(attempts int) time.Duration {
if attempts < 1 {
attempts = 1
}
d := time.Duration(attempts) * 30 * time.Second
if d > 5*time.Minute {
d = 5 * time.Minute
}
return d
}
@@ -1,4 +1,4 @@
package download
package downloads
import (
"context"
@@ -65,12 +65,14 @@ func (bm *BandwidthManager) getUserLimiter(userID int) *rate.Limiter {
return nil
}
if v, ok := bm.userLimiters.Load(userID); ok {
return v.(*rate.Limiter)
lim, _ := v.(*rate.Limiter)
return lim
}
burst := max(int(ubps/4), 32*1024)
limiter := rate.NewLimiter(rate.Limit(ubps), burst)
actual, _ := bm.userLimiters.LoadOrStore(userID, limiter)
return actual.(*rate.Limiter)
lim, _ := actual.(*rate.Limiter)
return lim
}
// ThrottledReader wraps an io.ReadSeeker with bandwidth limiting for the given user.
@@ -1,4 +1,4 @@
package download
package downloads
import (
"context"
+490
View File
@@ -0,0 +1,490 @@
package downloads
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"os"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/subtitles"
)
// manifestVersion is bumped whenever the OfflineManifest DTO shape changes.
const manifestVersion = 2
const apiDownloadsPrefix = "/api/v1/downloads/"
// ManifestSource assembles catalog detail for a content id. GetItemDetail
// enforces per-profile content/library access via its filter, which doubles as
// the manifest/artwork access re-check.
type ManifestSource interface {
GetItemDetail(ctx context.Context, contentID string, filter catalog.AccessFilter) (*catalog.ItemDetail, error)
}
// SubtitleSource enumerates and fetches downloaded (S3) subtitle assets.
type SubtitleSource interface {
ListDownloadedSubtitles(ctx context.Context, mediaFileID int) ([]subtitles.DownloadedSubtitle, error)
GetSubtitleContent(ctx context.Context, id int) (*subtitles.DownloadedSubtitle, []byte, error)
}
// Marker is a time range (intro/credits/recap/preview) in seconds.
type Marker struct {
Start float64 `json:"start"`
End float64 `json:"end"`
}
// OfflineChapter is a chapter with only stable references (no presigned URL).
type OfflineChapter struct {
Index int `json:"index"`
Title string `json:"title,omitempty"`
StartSeconds float64 `json:"start_seconds"`
EndSeconds float64 `json:"end_seconds"`
ThumbnailThumbhash string `json:"thumbnail_thumbhash,omitempty"`
}
// OfflineSubtitle is one downloadable subtitle asset. FetchURL is an
// authenticated proxy endpoint, never a presigned URL.
type OfflineSubtitle struct {
Language string `json:"language"`
Format string `json:"format"`
Forced bool `json:"forced"`
HearingImpaired bool `json:"hearing_impaired"`
External bool `json:"external"`
FetchURL string `json:"fetch_url"`
FileSize int64 `json:"file_size,omitempty"`
}
// OfflineAudioTrack describes audio streams the client may expose offline.
type OfflineAudioTrack struct {
Index int `json:"index"`
Title string `json:"title,omitempty"`
Language string `json:"language,omitempty"`
Codec string `json:"codec,omitempty"`
Layout string `json:"layout,omitempty"`
Channels int `json:"channels,omitempty"`
Bitrate int `json:"bitrate,omitempty"`
SampleRate int `json:"sample_rate,omitempty"`
Default bool `json:"default"`
}
// OfflineIdentity mirrors userstore.WatchIdentity so a client can re-resolve
// content_id after a server-side rescan.
type OfflineIdentity struct {
StableType string `json:"stable_type,omitempty"`
ProviderIDs map[string]string `json:"provider_ids,omitempty"`
SeriesProviderIDs map[string]string `json:"series_provider_ids,omitempty"`
Season *int `json:"season,omitempty"`
Episode *int `json:"episode,omitempty"`
}
// OfflineIntegrity gives clients stable metadata for local file validation.
type OfflineIntegrity struct {
ExpectedBytes int64 `json:"expected_bytes"`
MediaFileHash string `json:"media_file_hash,omitempty"`
MetadataETag string `json:"metadata_etag"`
}
// OfflineManifest is the stable, presigned-URL-free bundle a client stores to
// play a managed download fully offline.
type OfflineManifest struct {
DownloadID string `json:"download_id"`
ContentID string `json:"content_id"`
EpisodeID string `json:"episode_id,omitempty"`
Type string `json:"type"`
Revision int `json:"revision"`
Quality string `json:"quality"`
EffectiveQuality string `json:"effective_quality"`
DeliveryFormat string `json:"delivery_format"`
TargetBitrateKbps int `json:"target_bitrate_kbps"`
MediaFileID int `json:"media_file_id"`
FileSize int64 `json:"file_size"`
Title string `json:"title"`
Year int `json:"year,omitempty"`
Overview string `json:"overview,omitempty"`
Runtime int `json:"runtime,omitempty"`
ContentRating string `json:"content_rating,omitempty"`
Genres []string `json:"genres,omitempty"`
SeriesID string `json:"series_id,omitempty"`
SeriesTitle string `json:"series_title,omitempty"`
SeasonNumber *int `json:"season_number,omitempty"`
EpisodeNumber *int `json:"episode_number,omitempty"`
// Artwork: stable thumbhashes inline + authenticated proxy URLs (never
// presigned S3 URLs). The client downloads the proxy URLs once.
PosterThumbhash string `json:"poster_thumbhash,omitempty"`
BackdropThumbhash string `json:"backdrop_thumbhash,omitempty"`
ArtworkURLs struct {
Poster string `json:"poster,omitempty"`
Backdrop string `json:"backdrop,omitempty"`
Logo string `json:"logo,omitempty"`
} `json:"artwork_urls"`
Container string `json:"container"`
CodecVideo string `json:"codec_video"`
CodecAudio string `json:"codec_audio"`
Resolution string `json:"resolution"`
HDR bool `json:"hdr"`
Duration int `json:"duration_seconds"`
SelectedAudioTrackIndex *int `json:"selected_audio_track_index,omitempty"`
AudioTracks []OfflineAudioTrack `json:"audio_tracks,omitempty"`
Chapters []OfflineChapter `json:"chapters,omitempty"`
Intro *Marker `json:"intro,omitempty"`
Credits *Marker `json:"credits,omitempty"`
Recap *Marker `json:"recap,omitempty"`
Preview *Marker `json:"preview,omitempty"`
Subtitles []OfflineSubtitle `json:"subtitles"`
StableIdentity OfflineIdentity `json:"stable_identity"`
Integrity OfflineIntegrity `json:"integrity"`
ManifestVersion int `json:"manifest_version"`
GeneratedAt string `json:"generated_at"`
}
// ManifestBuilder assembles an OfflineManifest from the catalog detail path and
// the download's subtitle assets, stripping every presigned URL.
type ManifestBuilder struct {
detail ManifestSource
subs SubtitleSource
fileRepo FileResolver
// artifact resolves a download's linked prepared artifact so artifact-backed
// manifests can describe the delivered file instead of the catalog source.
artifact func(ctx context.Context, id string) (*Artifact, error)
}
// NewManifestBuilder constructs a ManifestBuilder. artifact may be nil when no
// prepare-to-file pipeline exists (only original downloads are servable then).
func NewManifestBuilder(detail ManifestSource, subs SubtitleSource, fileRepo FileResolver, artifact func(ctx context.Context, id string) (*Artifact, error)) *ManifestBuilder {
return &ManifestBuilder{detail: detail, subs: subs, fileRepo: fileRepo, artifact: artifact}
}
// Build assembles the manifest for a managed entry. The filter enforces the
// requesting profile's content access (GetItemDetail returns
// catalog.ErrItemNotFound when denied).
func (b *ManifestBuilder) Build(ctx context.Context, dl *Download, filter catalog.AccessFilter) (*OfflineManifest, error) {
return b.build(ctx, dl, filter, nil)
}
// build is Build with an optional per-batch series-detail cache: a season
// batch shares one series, so the batch endpoint resolves its detail once
// instead of once per episode.
func (b *ManifestBuilder) build(ctx context.Context, dl *Download, filter catalog.AccessFilter, seriesCache map[string]*catalog.ItemDetail) (*OfflineManifest, error) {
detail, err := b.detail.GetItemDetail(ctx, manifestContentID(dl), filter)
if err != nil {
return nil, err
}
file := b.lookupFile(ctx, dl.MediaFileID)
var seriesDetail *catalog.ItemDetail
if dl.EpisodeID != "" && detail.SeriesID != "" {
if cached, ok := seriesCache[detail.SeriesID]; ok {
seriesDetail = cached
} else if sd, err := b.detail.GetItemDetail(ctx, detail.SeriesID, filter); err == nil {
seriesDetail = sd
if seriesCache != nil {
seriesCache[detail.SeriesID] = sd
}
}
}
m := &OfflineManifest{
DownloadID: dl.ID,
ContentID: dl.ContentID,
EpisodeID: dl.EpisodeID,
Type: detail.Type,
Revision: dl.Revision,
Quality: dl.Quality,
EffectiveQuality: dl.EffectiveQuality,
DeliveryFormat: dl.Format,
TargetBitrateKbps: dl.TargetBitrateKbps,
MediaFileID: dl.MediaFileID,
FileSize: dl.FileSize,
Title: detail.Title,
Year: detail.Year,
Overview: detail.Overview,
Runtime: detail.Runtime,
ContentRating: detail.ContentRating,
Genres: detail.Genres,
SeriesID: detail.SeriesID,
SeriesTitle: detail.SeriesTitle,
SeasonNumber: detail.SeasonNumber,
EpisodeNumber: detail.EpisodeNumber,
PosterThumbhash: detail.PosterThumbhash,
BackdropThumbhash: detail.BackdropThumbhash,
Intro: toMarker(detail.Intro),
Credits: toMarker(detail.Credits),
Recap: toMarker(detail.Recap),
Preview: toMarker(detail.Preview),
StableIdentity: stableIdentity(dl, detail, seriesDetail),
Integrity: buildIntegrity(dl, file),
ManifestVersion: manifestVersion,
GeneratedAt: time.Now().UTC().Format(time.RFC3339),
}
// Artwork: emit a proxy URL only when the source actually has the image.
if detail.PosterURL != "" {
m.ArtworkURLs.Poster = artworkProxyURL(dl.ID, "poster")
}
if detail.BackdropURL != "" {
m.ArtworkURLs.Backdrop = artworkProxyURL(dl.ID, "backdrop")
}
if detail.LogoURL != "" {
m.ArtworkURLs.Logo = artworkProxyURL(dl.ID, "logo")
}
if v := pickVersion(detail, dl.MediaFileID); v != nil {
m.Container = v.Container
m.CodecVideo = v.CodecVideo
m.CodecAudio = v.CodecAudio
m.Resolution = v.Resolution
m.HDR = v.HDR
m.Duration = v.Duration
m.SelectedAudioTrackIndex = v.EffectiveAudioTrackIndex
m.AudioTracks = toOfflineAudioTracks(v.AudioTracks)
m.Chapters = toOfflineChapters(v.Chapters)
}
// Remux/transcode entries deliver the prepared artifact, not the catalog
// source: describe that file so the client picks the right decoder/tracks.
if dl.Format != FormatOriginal && dl.ArtifactID != "" && b.artifact != nil {
if a, err := b.artifact(ctx, dl.ArtifactID); err == nil && a != nil {
applyArtifactParams(m, a)
}
}
m.Subtitles = b.buildSubtitles(ctx, dl, file)
return m, nil
}
// applyArtifactParams overwrites the source file's media parameters with the
// prepared artifact's target parameters. "copy" targets keep the source value.
func applyArtifactParams(m *OfflineManifest, a *Artifact) {
if a.Container != "" {
m.Container = a.Container
}
if a.CodecVideo != "" && a.CodecVideo != "copy" {
m.CodecVideo = a.CodecVideo
}
if a.CodecAudio != "" && a.CodecAudio != "copy" {
m.CodecAudio = a.CodecAudio
}
if a.Resolution != "" {
m.Resolution = a.Resolution
}
// A prepared file contains exactly one audio stream — the track the encode
// selected (playback.PrepareFile maps a single audio track).
if len(m.AudioTracks) > 0 {
idx := a.AudioTrackIndex
if idx < 0 || idx >= len(m.AudioTracks) {
idx = 0
}
track := m.AudioTracks[idx]
track.Index = 0
track.Default = true
if a.CodecAudio != "" && a.CodecAudio != "copy" {
track.Codec = a.CodecAudio
}
m.AudioTracks = []OfflineAudioTrack{track}
selected := 0
m.SelectedAudioTrackIndex = &selected
}
}
// buildSubtitles enumerates external (sidecar) + downloaded (S3) subtitle assets
// for the download's media file (already loaded by build — no re-fetch).
// Embedded tracks live inside the downloaded video file and need no separate
// fetch.
func (b *ManifestBuilder) buildSubtitles(ctx context.Context, dl *Download, file *models.MediaFile) []OfflineSubtitle {
out := []OfflineSubtitle{}
if file != nil {
for i, ext := range file.ExternalSubtitles {
var size int64
if info, statErr := os.Stat(ext.Path); statErr == nil {
size = info.Size()
}
out = append(out, OfflineSubtitle{
Language: ext.Language,
Format: ext.Format,
Forced: ext.Forced,
HearingImpaired: ext.HearingImpaired,
External: true,
FetchURL: subtitleProxyURL(dl.ID, fmt.Sprintf("external:%d", i)),
FileSize: size,
})
}
}
if b.subs != nil {
if downloaded, err := b.subs.ListDownloadedSubtitles(ctx, dl.MediaFileID); err == nil {
for _, sub := range downloaded {
out = append(out, OfflineSubtitle{
Language: sub.Language,
Format: string(sub.Format),
HearingImpaired: sub.HearingImpaired,
External: false,
FetchURL: subtitleProxyURL(dl.ID, fmt.Sprintf("downloaded:%d", sub.ID)),
})
}
}
}
return out
}
func (b *ManifestBuilder) lookupFile(ctx context.Context, mediaFileID int) *models.MediaFile {
if b.fileRepo == nil || mediaFileID <= 0 {
return nil
}
file, err := b.fileRepo.GetByID(ctx, mediaFileID)
if err != nil {
return nil
}
return file
}
// manifestContentID resolves the item the manifest describes: the episode's own
// content id for episode entries, otherwise the movie's content id.
func manifestContentID(dl *Download) string {
if dl.EpisodeID != "" {
return dl.EpisodeID
}
return dl.ContentID
}
func artworkProxyURL(downloadID, kind string) string {
return apiDownloadsPrefix + downloadID + "/artwork/" + kind
}
func subtitleProxyURL(downloadID, ref string) string {
return apiDownloadsPrefix + downloadID + "/subtitles/" + ref
}
func pickVersion(detail *catalog.ItemDetail, mediaFileID int) *catalog.FileVersion {
for i := range detail.Versions {
if detail.Versions[i].FileID == mediaFileID {
return &detail.Versions[i]
}
}
if len(detail.Versions) > 0 {
return &detail.Versions[0]
}
return nil
}
func toMarker(m *catalog.Marker) *Marker {
if m == nil {
return nil
}
return &Marker{Start: m.Start, End: m.End}
}
func toOfflineChapters(chapters []catalog.VersionChapter) []OfflineChapter {
if len(chapters) == 0 {
return nil
}
out := make([]OfflineChapter, 0, len(chapters))
for _, c := range chapters {
out = append(out, OfflineChapter{
Index: c.Index,
Title: c.Title,
StartSeconds: c.StartSeconds,
EndSeconds: c.EndSeconds,
ThumbnailThumbhash: c.ThumbnailThumbhash,
})
}
return out
}
func toOfflineAudioTracks(tracks []models.AudioTrack) []OfflineAudioTrack {
if len(tracks) == 0 {
return nil
}
out := make([]OfflineAudioTrack, 0, len(tracks))
for i, t := range tracks {
out = append(out, OfflineAudioTrack{
Index: i,
Title: firstNonEmpty(t.Title, t.EmbeddedTitle),
Language: t.Language,
Codec: t.Codec,
Layout: t.Layout,
Channels: t.Channels,
Bitrate: t.Bitrate,
SampleRate: t.SampleRate,
Default: t.Default,
})
}
return out
}
func stableIdentity(dl *Download, detail, seriesDetail *catalog.ItemDetail) OfflineIdentity {
providerIDs := map[string]string{}
addProviderIDs(providerIDs, detail)
id := OfflineIdentity{
StableType: detail.Type,
ProviderIDs: providerIDs,
Season: detail.SeasonNumber,
Episode: detail.EpisodeNumber,
}
if dl.EpisodeID != "" {
id.StableType = "episode"
seriesProviderIDs := map[string]string{}
addProviderIDs(seriesProviderIDs, seriesDetail)
if len(seriesProviderIDs) > 0 {
id.SeriesProviderIDs = seriesProviderIDs
}
}
if len(providerIDs) == 0 {
id.ProviderIDs = nil
}
return id
}
func addProviderIDs(out map[string]string, detail *catalog.ItemDetail) {
if detail == nil {
return
}
if detail.ImdbID != "" {
out["imdb"] = detail.ImdbID
}
if detail.TmdbID != "" {
out["tmdb"] = detail.TmdbID
}
if detail.TvdbID != "" {
out["tvdb"] = detail.TvdbID
}
}
func buildIntegrity(dl *Download, file *models.MediaFile) OfflineIntegrity {
hash := ""
modified := ""
if file != nil {
hash = file.FileHash
if file.FileModifiedAt != nil {
modified = file.FileModifiedAt.UTC().Format(time.RFC3339Nano)
}
}
sum := sha256.Sum256([]byte(fmt.Sprintf("%s|%d|%s|%s|%s|%d|%d|%s",
dl.ID, dl.Revision, dl.Format, dl.Quality,
dl.EffectiveQuality, dl.TargetBitrateKbps, dl.FileSize, modified)))
return OfflineIntegrity{
ExpectedBytes: dl.FileSize,
MediaFileHash: hash,
MetadataETag: hex.EncodeToString(sum[:]),
}
}
func firstNonEmpty(values ...string) string {
for _, v := range values {
if v != "" {
return v
}
}
return ""
}
+192
View File
@@ -0,0 +1,192 @@
package downloads
import (
"bytes"
"context"
"encoding/json"
"errors"
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/subtitles"
)
type fakeManifestSource struct {
detail *catalog.ItemDetail
err error
}
func (f fakeManifestSource) GetItemDetail(context.Context, string, catalog.AccessFilter) (*catalog.ItemDetail, error) {
if f.err != nil {
return nil, f.err
}
return f.detail, nil
}
type fakeSubtitleSource struct {
downloaded []subtitles.DownloadedSubtitle
}
func (f fakeSubtitleSource) ListDownloadedSubtitles(context.Context, int) ([]subtitles.DownloadedSubtitle, error) {
return f.downloaded, nil
}
func (f fakeSubtitleSource) GetSubtitleContent(context.Context, int) (*subtitles.DownloadedSubtitle, []byte, error) {
return nil, nil, ErrAssetNotFound
}
type fakeFileResolver struct {
file *models.MediaFile
}
func (f fakeFileResolver) GetByID(context.Context, int) (*models.MediaFile, error) {
return f.file, nil
}
func (f fakeFileResolver) GetByContentID(context.Context, string) ([]*models.MediaFile, error) {
return nil, nil
}
func (f fakeFileResolver) GetByEpisodeID(context.Context, string) ([]*models.MediaFile, error) {
return nil, nil
}
func (f fakeFileResolver) ListByEpisodeIDs(context.Context, []string) (map[string][]*models.MediaFile, error) {
return nil, nil
}
// TestManifestBuilderDeniesRestrictedProfile is the Phase 2 acceptance criterion
// at the source: when the requesting profile is denied content access,
// GetItemDetail returns ErrItemNotFound and Build propagates it.
func TestManifestBuilderDeniesRestrictedProfile(t *testing.T) {
b := NewManifestBuilder(fakeManifestSource{err: catalog.ErrItemNotFound}, nil, nil, nil)
_, err := b.Build(context.Background(), &Download{ID: "dl1", ContentID: "c1"}, catalog.AccessFilter{})
if !errors.Is(err, catalog.ErrItemNotFound) {
t.Fatalf("Build err = %v, want catalog.ErrItemNotFound", err)
}
}
func TestManifestBuilderAssembles(t *testing.T) {
detail := &catalog.ItemDetail{
Type: "movie",
Title: "The Movie",
Year: 2021,
Overview: "A film.",
Runtime: 120,
ContentRating: "PG-13",
Genres: []string{"Drama"},
PosterURL: "https://s3.example.com/poster.jpg?sig=SECRET",
PosterThumbhash: "PHASH",
BackdropURL: "https://s3.example.com/backdrop.jpg?sig=SECRET",
BackdropThumbhash: "BHASH",
ImdbID: "tt123",
TmdbID: "456",
Intro: &catalog.Marker{Start: 0, End: 60},
Versions: []catalog.FileVersion{{
FileID: 99,
Container: "mkv",
CodecVideo: "h264",
CodecAudio: "aac",
Resolution: "1080p",
Duration: 7200,
Chapters: []catalog.VersionChapter{{
Index: 1, Title: "Opening", StartSeconds: 0, EndSeconds: 600,
ThumbnailURL: "https://s3.example.com/chap.jpg?sig=SECRET", ThumbnailThumbhash: "CHASH",
}},
}},
}
file := &models.MediaFile{ExternalSubtitles: []models.ExternalSubtitle{
{Path: "/media/sub.en.srt", Language: "en", Format: "srt", Forced: true},
}}
subs := fakeSubtitleSource{downloaded: []subtitles.DownloadedSubtitle{
{ID: 7, MediaFileID: 99, Language: "fr", Format: subtitles.SubtitleFormat("vtt")},
}}
b := NewManifestBuilder(fakeManifestSource{detail: detail}, subs, fakeFileResolver{file: file}, nil)
dl := &Download{ID: "dl1", ContentID: "c1", MediaFileID: 99, FileSize: 1024, Format: FormatOriginal}
m, err := b.Build(context.Background(), dl, catalog.AccessFilter{})
if err != nil {
t.Fatalf("Build: %v", err)
}
if m.Title != "The Movie" || m.Year != 2021 || m.Runtime != 120 {
t.Fatalf("metadata mismatch: %+v", m)
}
if m.PosterThumbhash != "PHASH" || m.BackdropThumbhash != "BHASH" {
t.Fatalf("thumbhashes not inlined: %+v", m)
}
if m.ArtworkURLs.Poster != "/api/v1/downloads/dl1/artwork/poster" {
t.Fatalf("poster url = %q, want proxy path", m.ArtworkURLs.Poster)
}
if m.ArtworkURLs.Backdrop != "/api/v1/downloads/dl1/artwork/backdrop" {
t.Fatalf("backdrop url = %q, want proxy path", m.ArtworkURLs.Backdrop)
}
if m.ArtworkURLs.Logo != "" {
t.Fatalf("logo url = %q, want empty (no LogoURL)", m.ArtworkURLs.Logo)
}
if m.Container != "mkv" || m.CodecVideo != "h264" || m.Resolution != "1080p" || m.Duration != 7200 {
t.Fatalf("playback metadata mismatch: %+v", m)
}
if m.Intro == nil || m.Intro.End != 60 {
t.Fatalf("intro marker = %+v", m.Intro)
}
if len(m.Chapters) != 1 || m.Chapters[0].ThumbnailThumbhash != "CHASH" {
t.Fatalf("chapters = %+v", m.Chapters)
}
if len(m.Subtitles) != 2 {
t.Fatalf("subtitles = %+v, want 2", m.Subtitles)
}
if m.Subtitles[0].FetchURL != "/api/v1/downloads/dl1/subtitles/external:0" || !m.Subtitles[0].External {
t.Fatalf("external subtitle = %+v", m.Subtitles[0])
}
if m.Subtitles[1].FetchURL != "/api/v1/downloads/dl1/subtitles/downloaded:7" || m.Subtitles[1].External {
t.Fatalf("downloaded subtitle = %+v", m.Subtitles[1])
}
if m.StableIdentity.ProviderIDs["imdb"] != "tt123" || m.StableIdentity.ProviderIDs["tmdb"] != "456" {
t.Fatalf("stable identity = %+v", m.StableIdentity)
}
if m.ManifestVersion != manifestVersion {
t.Fatalf("manifest version = %d, want %d", m.ManifestVersion, manifestVersion)
}
// The whole point of the manifest: a stored copy must contain NO presigned
// URL. Serialize and assert the upstream signature/host never leaks.
encoded, err := json.Marshal(m)
if err != nil {
t.Fatalf("marshal: %v", err)
}
for _, leak := range [][]byte{[]byte("sig=SECRET"), []byte("s3.example.com")} {
if bytes.Contains(encoded, leak) {
t.Fatalf("manifest leaks presigned URL fragment %q: %s", leak, encoded)
}
}
}
func TestParseSubtitleRef(t *testing.T) {
cases := []struct {
ref string
wantKind string
wantValue int
wantErr bool
}{
{"external:0", "external", 0, false},
{"external:12", "external", 12, false},
{"downloaded:7", "downloaded", 7, false},
{"bogus", "", 0, true},
{"external:x", "", 0, true},
{"weird:1", "", 0, true},
{"", "", 0, true},
}
for _, tc := range cases {
t.Run(tc.ref, func(t *testing.T) {
kind, value, err := parseSubtitleRef(tc.ref)
if tc.wantErr {
if !errors.Is(err, ErrInvalidSubtitleRef) {
t.Fatalf("parseSubtitleRef(%q) err = %v, want ErrInvalidSubtitleRef", tc.ref, err)
}
return
}
if err != nil || kind != tc.wantKind || value != tc.wantValue {
t.Fatalf("parseSubtitleRef(%q) = (%q, %d, %v)", tc.ref, kind, value, err)
}
})
}
}
+141
View File
@@ -0,0 +1,141 @@
// Package downloads owns the downloads domain: a single device-aware,
// quality-aware registry shared by the web app and mobile clients for offline
// playback. It replaces the former internal/download package, absorbing its
// bandwidth/quota/serving logic and reshaping the table and /downloads contract. See
// docs/superpowers/specs/2026-06-18-offline-sync-mobile-design.md.
package downloads
import (
"errors"
"time"
)
// Format constants are internal delivery formats recorded on a download row.
// They are not user-facing quality presets: clients choose a Quality*, then the
// service records the concrete delivery format it can fulfill.
const (
FormatOriginal = "original"
FormatRemux = "remux"
FormatTranscode = "transcode"
)
// Quality constants are the public download choices clients may request.
const (
QualityOriginal = "original"
Quality20Mbps = "20mbps"
Quality10Mbps = "10mbps"
Quality5Mbps = "5mbps"
Quality2Mbps = "2mbps"
Quality1Mbps = "1mbps"
)
// QualityPresets is the ordered UI/API ladder for download quality.
var QualityPresets = []string{
QualityOriginal,
Quality20Mbps,
Quality10Mbps,
Quality5Mbps,
Quality2Mbps,
Quality1Mbps,
}
// Download status constants.
const (
// Ephemeral / web lifecycle (unchanged from downloads v1):
// queued -> downloading -> completed | failed | canceled.
StatusQueued = "queued"
StatusDownloading = "downloading"
StatusCompleted = "completed"
StatusFailed = "failed"
StatusCancelled = "cancelled" //nolint:misspell // persisted DB enum value (migration 042)
// Managed device-entry lifecycle (Phase 1+):
// [preparing ->] ready -> completed, plus revoked (reserved for the
// admin revoke flow; see the download limits & restrictions design).
StatusPreparing = "preparing"
StatusReady = "ready"
StatusRevoked = "revoked"
)
// Download kind constants.
const (
KindQueued = "queued"
)
// Sentinel errors.
var (
ErrNotFound = errors.New("download not found")
ErrDownloadNotAllowed = errors.New("user is not allowed to download")
ErrFeatureDisabled = errors.New("downloads are disabled")
ErrConcurrentLimitReached = errors.New("concurrent download limit reached")
ErrPeriodLimitReached = errors.New("download period limit reached")
ErrDownloadNotActive = errors.New("download is not in an active state")
ErrStatusConflict = errors.New("download status transition conflict")
ErrTranscodeDisabled = errors.New("download transcode is disabled")
ErrInvalidQuality = errors.New("invalid download quality")
ErrProfileRequired = errors.New("managed download requires a profile")
ErrInvalidStatus = errors.New("invalid download status transition")
ErrManifestUnavailable = errors.New("offline manifest is not available")
ErrInvalidSubtitleRef = errors.New("invalid subtitle reference")
ErrAssetNotFound = errors.New("download asset not found")
ErrFormatUnavailable = errors.New("requested download format is not available")
// ErrQualityUnavailable means the requested quality is valid and permitted in
// principle but cannot be fulfilled by the current server wiring.
ErrQualityUnavailable = errors.New("requested download quality is not available")
// ErrBulkQualityUnavailable keeps season/series batches original-only until
// batch artifact UX and storage reporting are explicit.
ErrBulkQualityUnavailable = errors.New("bulk quality downloads are not available")
// ErrNoDownloadableEpisodes means a series/season download matched a valid
// series but none of its episodes have a downloadable file.
ErrNoDownloadableEpisodes = errors.New("no downloadable episodes found")
// Series-monitoring (download subscription) errors.
ErrSubscriptionNotFound = errors.New("download subscription not found")
ErrSubscriptionsUnavailable = errors.New("download subscriptions are not available")
ErrInvalidSubscriptionMode = errors.New("invalid subscription mode")
ErrSeasonsRequired = errors.New("season_numbers is required for specific_seasons mode")
ErrInvalidSeasonNumbers = errors.New("season_numbers contains an out-of-range season number")
ErrNotSeries = errors.New("content_id is not a series")
)
// Download represents a row in the downloads table. It carries both the
// ephemeral/web lifecycle (DeviceID == "") and the managed device-entry
// lifecycle (DeviceID set); ProfileID/DeviceID/ArtifactID map to nullable
// columns and are empty strings when unset.
type Download struct {
ID string
UserID int
ProfileID string // "" for ephemeral/web rows
DeviceID string // "" = ephemeral; set = managed device entry
MediaFileID int
ContentID string
EpisodeID string
BatchID string
Kind string // direct or queued
Status string // see Status* constants
Format string // internal delivery format; see Format* constants
Quality string // user-requested quality; see Quality* constants
EffectiveQuality string // actual quality after original compatibility fallback
TargetBitrateKbps int // 0 for original/remux; bitrate cap for transcode
Revision int // increments when a managed row is replaced in place
ArtifactID string // "" until a remux/transcode artifact is linked (Phase 3)
FileSize int64
BytesSent int64
ErrorMessage string
CreatedAt time.Time
UpdatedAt time.Time
CompletedAt *time.Time
}
// SkippedDownload explains why a bulk series/season request did not create a
// row for an episode.
type SkippedDownload struct {
EpisodeID string `json:"episode_id"`
Reason string `json:"reason"`
}
// IsManaged reports whether this is a managed device-library entry (DeviceID set)
// rather than an ephemeral/account-level row.
func (d *Download) IsManaged() bool {
return d.DeviceID != ""
}
+246
View File
@@ -0,0 +1,246 @@
package downloads
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"strconv"
"strings"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/subtitles"
)
// authorizeManagedAsset applies invariant-2 row authorization (user, profile,
// device) and the revoked guard to a managed entry before any asset is served.
// The per-profile content-access re-check is performed by each caller (via
// GetItemDetail or EnsureAccessible) before bytes leave the server.
func (s *Service) authorizeManagedAsset(ctx context.Context, userID int, profileID, deviceID, downloadID string) (*Download, error) {
if _, err := s.enabledConfig(ctx); err != nil {
return nil, err
}
if profileID == "" || deviceID == "" {
return nil, ErrProfileRequired
}
dl, err := s.repo.GetManagedByID(ctx, downloadID, userID, profileID, deviceID)
if err != nil {
return nil, err
}
if dl.Status == StatusRevoked {
return nil, fmt.Errorf("download is revoked: %w", ErrDownloadNotActive)
}
return dl, nil
}
// BuildManifest returns the offline manifest for a managed entry, authorized on
// (user, profile, device) with a per-profile content-access re-check inside the
// builder's GetItemDetail call.
func (s *Service) BuildManifest(ctx context.Context, userID int, profileID, deviceID, downloadID string, filter catalog.AccessFilter) (*OfflineManifest, error) {
dl, err := s.authorizeManagedAsset(ctx, userID, profileID, deviceID, downloadID)
if err != nil {
return nil, err
}
if s.manifest == nil {
return nil, ErrManifestUnavailable
}
return s.manifest.Build(ctx, dl, filter)
}
// SkippedManifest reports a batch entry whose manifest could not be built —
// one bad episode (revoked, deleted from the catalog, access-filtered) must
// not make the rest of a season's manifests unfetchable.
type SkippedManifest struct {
DownloadID string `json:"download_id"`
Reason string `json:"reason"` // revoked | not_found | error
}
// BuildBatchManifests returns the manifests for every managed entry in a batch
// owned by the calling profile/device, plus the entries it had to skip.
// Entries in one batch share a series, so the series detail is resolved once.
func (s *Service) BuildBatchManifests(ctx context.Context, userID int, profileID, deviceID, batchID string, filter catalog.AccessFilter) ([]*OfflineManifest, []SkippedManifest, error) {
if _, err := s.enabledConfig(ctx); err != nil {
return nil, nil, err
}
if profileID == "" || deviceID == "" {
return nil, nil, ErrProfileRequired
}
if s.manifest == nil {
return nil, nil, ErrManifestUnavailable
}
rows, err := s.repo.ListManagedByBatch(ctx, userID, profileID, deviceID, batchID)
if err != nil {
return nil, nil, err
}
if len(rows) == 0 {
return nil, nil, ErrNotFound
}
out := make([]*OfflineManifest, 0, len(rows))
skipped := make([]SkippedManifest, 0)
seriesCache := make(map[string]*catalog.ItemDetail, 1)
for _, dl := range rows {
if dl.Status == StatusRevoked {
skipped = append(skipped, SkippedManifest{DownloadID: dl.ID, Reason: "revoked"})
continue
}
m, err := s.manifest.build(ctx, dl, filter, seriesCache)
if err != nil {
reason := "error"
if errors.Is(err, catalog.ErrItemNotFound) {
reason = "not_found"
} else {
slog.Warn("batch manifest build failed", "download_id", dl.ID, "batch_id", batchID, "error", err)
}
skipped = append(skipped, SkippedManifest{DownloadID: dl.ID, Reason: reason})
continue
}
out = append(out, m)
}
return out, skipped, nil
}
// ServeArtwork streams poster/backdrop/logo bytes for a managed entry through
// the image resolver (never a presigned redirect), re-checking per-profile
// access via GetItemDetail before serving.
func (s *Service) ServeArtwork(ctx context.Context, w http.ResponseWriter, r *http.Request, userID int, profileID, deviceID, downloadID, kind string, filter catalog.AccessFilter) error {
dl, err := s.authorizeManagedAsset(ctx, userID, profileID, deviceID, downloadID)
if err != nil {
return err
}
if s.artworkSource == nil {
return ErrManifestUnavailable
}
detail, err := s.artworkSource.GetItemDetail(ctx, manifestContentID(dl), filter)
if err != nil {
return err
}
var url string
switch kind {
case "poster":
url = detail.PosterURL
case "backdrop":
url = detail.BackdropURL
case "logo":
url = detail.LogoURL
default:
return ErrAssetNotFound
}
if url == "" {
return ErrAssetNotFound
}
return s.streamArtwork(ctx, w, r, url)
}
func (s *Service) streamArtwork(ctx context.Context, w http.ResponseWriter, _ *http.Request, url string) error {
client := s.httpClient
if client == nil {
client = http.DefaultClient
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return fmt.Errorf("building artwork request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("fetching artwork: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("artwork upstream status %d: %w", resp.StatusCode, ErrAssetNotFound)
}
if ct := resp.Header.Get("Content-Type"); ct != "" {
w.Header().Set("Content-Type", ct)
}
if cl := resp.Header.Get("Content-Length"); cl != "" {
w.Header().Set("Content-Length", cl)
}
// Artwork is immutable for a stored manifest; let the client cache it once.
w.Header().Set("Cache-Control", "private, max-age=31536000, immutable")
if _, err := io.Copy(w, resp.Body); err != nil {
return fmt.Errorf("streaming artwork: %w", err)
}
return nil
}
// ServeSubtitle streams a subtitle asset (external sidecar or downloaded S3 file)
// for a managed entry, authorized on (user, profile, device) with a per-profile
// content-access re-check. ref encodes "external:{index}" or "downloaded:{id}".
func (s *Service) ServeSubtitle(ctx context.Context, w http.ResponseWriter, _ *http.Request, userID int, profileID, deviceID, downloadID, ref string, filter catalog.AccessFilter) error {
dl, err := s.authorizeManagedAsset(ctx, userID, profileID, deviceID, downloadID)
if err != nil {
return err
}
if err := s.itemAccess.EnsureAccessible(ctx, dl.ContentID, filter); err != nil {
return err
}
kind, value, err := parseSubtitleRef(ref)
if err != nil {
return err
}
switch kind {
case "external":
idx := value
file, err := s.fileRepo.GetByID(ctx, dl.MediaFileID)
if err != nil {
return fmt.Errorf("loading media file: %w", err)
}
if file == nil || idx < 0 || idx >= len(file.ExternalSubtitles) {
return ErrAssetNotFound
}
ext := file.ExternalSubtitles[idx]
data, err := playback.LoadExternalSubtitleRaw(ext.Path)
if err != nil {
return fmt.Errorf("reading external subtitle: %w", ErrAssetNotFound)
}
writeSubtitle(w, ext.Format, data)
return nil
case "downloaded":
if s.subtitleSource == nil {
return ErrManifestUnavailable
}
sub, data, err := s.subtitleSource.GetSubtitleContent(ctx, value)
if err != nil {
return fmt.Errorf("loading downloaded subtitle: %w", ErrAssetNotFound)
}
// The subtitle must belong to this download's media file; a download id
// never grants access to an arbitrary subtitle id.
if sub == nil || sub.MediaFileID != dl.MediaFileID {
return ErrAssetNotFound
}
writeSubtitle(w, string(sub.Format), data)
return nil
default:
return ErrInvalidSubtitleRef
}
}
// parseSubtitleRef parses a subtitle reference of the form "external:{index}"
// or "downloaded:{id}" into its kind and integer value.
func parseSubtitleRef(ref string) (kind string, value int, err error) {
k, v, ok := strings.Cut(ref, ":")
if !ok {
return "", 0, ErrInvalidSubtitleRef
}
switch k {
case "external", "downloaded":
n, perr := strconv.Atoi(v)
if perr != nil {
return "", 0, ErrInvalidSubtitleRef
}
return k, n, nil
default:
return "", 0, ErrInvalidSubtitleRef
}
}
// writeSubtitle writes subtitle bytes with a format-appropriate content type
// (the shared subtitles mapping — no local copy to drift).
func writeSubtitle(w http.ResponseWriter, format string, data []byte) {
w.Header().Set("Content-Type", subtitles.SubtitleContentType(subtitles.SubtitleFormat(strings.ToLower(format))))
w.Header().Set("Cache-Control", "private, max-age=31536000, immutable")
_, _ = w.Write(data)
}
+82
View File
@@ -0,0 +1,82 @@
package downloads
import (
"context"
"fmt"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/config"
)
// mapManifestSource serves per-content-id details and counts lookups, so tests
// can assert both access-denial handling and the per-batch series cache.
type mapManifestSource struct {
details map[string]*catalog.ItemDetail
calls map[string]int
}
func (m *mapManifestSource) GetItemDetail(_ context.Context, contentID string, _ catalog.AccessFilter) (*catalog.ItemDetail, error) {
m.calls[contentID]++
if d, ok := m.details[contentID]; ok {
return d, nil
}
return nil, catalog.ErrItemNotFound
}
// TestBuildBatchManifestsSkipsAndCachesSeries pins two batch behaviors: one
// bad entry (deleted/access-filtered item) is reported in skipped[] instead of
// failing the whole batch, and the shared series detail is resolved once for
// the batch rather than once per episode.
func TestBuildBatchManifestsSkipsAndCachesSeries(t *testing.T) {
ctx := context.Background()
f := seedManagedFixture(t)
batchID := fmt.Sprintf("batch-%d", time.Now().UnixNano())
now := time.Now()
mkRow := func(n int, episodeID string) string {
id := fmt.Sprintf("dl-batch-%d-%d", n, now.UnixNano())
if err := f.repo.Create(ctx, &Download{
ID: id, UserID: f.userID, ProfileID: f.profileA, DeviceID: f.deviceA,
MediaFileID: f.fileID, ContentID: f.contentID, EpisodeID: episodeID,
BatchID: batchID, Kind: KindQueued, Status: StatusReady,
Format: FormatOriginal, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create row %d: %v", n, err)
}
return id
}
ep1 := mkRow(1, "ep-c1")
ep2 := mkRow(2, "ep-c2")
gone := mkRow(3, "ep-gone")
src := &mapManifestSource{
details: map[string]*catalog.ItemDetail{
"ep-c1": {Type: "episode", Title: "E1", SeriesID: "series-1"},
"ep-c2": {Type: "episode", Title: "E2", SeriesID: "series-1"},
"series-1": {Type: "series", Title: "The Show"},
},
calls: map[string]int{},
}
svc := NewService(f.repo, nil, nil, nil, nil, nil, nil, nil, nil, &config.DownloadConfig{Enabled: true})
svc.SetOfflineDeps(src, nil, nil)
manifests, skipped, err := svc.BuildBatchManifests(ctx, f.userID, f.profileA, f.deviceA, batchID, catalog.AccessFilter{})
if err != nil {
t.Fatalf("BuildBatchManifests: %v", err)
}
if len(manifests) != 2 {
t.Fatalf("manifests = %d, want 2", len(manifests))
}
got := map[string]bool{manifests[0].DownloadID: true, manifests[1].DownloadID: true}
if !got[ep1] || !got[ep2] {
t.Fatalf("manifest ids = %v, want %s and %s", got, ep1, ep2)
}
if len(skipped) != 1 || skipped[0].DownloadID != gone || skipped[0].Reason != "not_found" {
t.Fatalf("skipped = %+v, want [{%s not_found}]", skipped, gone)
}
if calls := src.calls["series-1"]; calls != 1 {
t.Fatalf("series detail resolved %d times for one batch, want 1", calls)
}
}
+180
View File
@@ -0,0 +1,180 @@
package downloads
import (
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
// QualityDecision is the resolved server-side target for a public download
// quality request.
type QualityDecision struct {
RequestedQuality string
EffectiveQuality string
DeliveryFormat string
TargetBitrateKbps int
PrepareTarget playback.PrepareTarget
RequiresArtifact bool
}
// DownloadQualityResolver validates a client-facing quality request and maps it
// to the concrete delivery format and encode target the server should record.
type DownloadQualityResolver struct{}
// Resolve returns a concrete delivery decision for file. Empty quality defaults
// to "original"; legacy user-facing delivery formats are intentionally rejected.
func (DownloadQualityResolver) Resolve(
requested string,
user *models.User,
cfg config.DownloadConfig,
file *models.MediaFile,
caps playback.ClientCapabilities,
artifactsAvailable bool,
) (QualityDecision, error) {
quality := normalizeQuality(requested)
if !ValidQuality(quality) {
return QualityDecision{}, ErrInvalidQuality
}
if quality != QualityOriginal {
if err := ensureTranscodeAllowed(user, cfg); err != nil {
return QualityDecision{}, err
}
if !artifactsAvailable {
return QualityDecision{}, ErrQualityUnavailable
}
bitrate := QualityBitrateKbps(quality)
target := playback.ResolvePrepareTarget(file, FormatTranscode, caps, playback.AdminSettings{
TranscodeEnabled: true,
Allow4KTranscode: true,
})
target.TargetBitrateKbps = bitrate
return QualityDecision{
RequestedQuality: quality,
EffectiveQuality: quality,
DeliveryFormat: FormatTranscode,
TargetBitrateKbps: bitrate,
PrepareTarget: target,
RequiresArtifact: true,
}, nil
}
decision := playback.PlayDirect
if hasCapabilities(caps) {
playDecision := playback.Resolve(file, caps, playback.AdminSettings{
TranscodeEnabled: cfg.TranscodeEnabled && user.DownloadTranscodeAllowed,
Allow4KTranscode: true,
})
decision = playDecision.Method
}
switch decision {
case playback.PlayDirect:
return QualityDecision{
RequestedQuality: QualityOriginal,
EffectiveQuality: QualityOriginal,
DeliveryFormat: FormatOriginal,
}, nil
case playback.PlayRemux:
if !artifactsAvailable {
return QualityDecision{}, ErrQualityUnavailable
}
target := playback.ResolvePrepareTarget(file, FormatRemux, caps, playback.AdminSettings{
TranscodeEnabled: cfg.TranscodeEnabled && user.DownloadTranscodeAllowed,
Allow4KTranscode: true,
})
return QualityDecision{
RequestedQuality: QualityOriginal,
EffectiveQuality: QualityOriginal,
DeliveryFormat: FormatRemux,
PrepareTarget: target,
RequiresArtifact: true,
}, nil
default:
if err := ensureTranscodeAllowed(user, cfg); err != nil {
return QualityDecision{}, err
}
if !artifactsAvailable {
return QualityDecision{}, ErrQualityUnavailable
}
target := playback.ResolvePrepareTarget(file, FormatTranscode, caps, playback.AdminSettings{
TranscodeEnabled: true,
Allow4KTranscode: true,
})
target.TargetBitrateKbps = QualityBitrateKbps(Quality20Mbps)
return QualityDecision{
RequestedQuality: QualityOriginal,
EffectiveQuality: Quality20Mbps,
DeliveryFormat: FormatTranscode,
TargetBitrateKbps: QualityBitrateKbps(Quality20Mbps),
PrepareTarget: target,
RequiresArtifact: true,
}, nil
}
}
// PresetsFor returns the ordered quality list currently fulfillable for a
// user. Always non-nil: the capability contract documents quality_presets as
// an array, and a nil slice would serialize as JSON null.
func (DownloadQualityResolver) PresetsFor(user *models.User, cfg config.DownloadConfig, artifactsAvailable bool) []string {
if !cfg.Enabled || user == nil || !user.DownloadAllowed {
return []string{}
}
presets := []string{QualityOriginal}
if artifactsAvailable && cfg.TranscodeEnabled && user.DownloadTranscodeAllowed {
presets = append(presets, Quality20Mbps, Quality10Mbps, Quality5Mbps, Quality2Mbps, Quality1Mbps)
}
return presets
}
// ValidQuality reports whether q is a public download quality value.
func ValidQuality(q string) bool {
switch q {
case QualityOriginal, Quality20Mbps, Quality10Mbps, Quality5Mbps, Quality2Mbps, Quality1Mbps:
return true
default:
return false
}
}
// QualityBitrateKbps returns the video bitrate cap for a bitrate preset. It is
// zero for original and invalid inputs.
func QualityBitrateKbps(q string) int {
switch q {
case Quality20Mbps:
return 20000
case Quality10Mbps:
return 10000
case Quality5Mbps:
return 5000
case Quality2Mbps:
return 2000
case Quality1Mbps:
return 1000
default:
return 0
}
}
func normalizeQuality(q string) string {
if q == "" {
return QualityOriginal
}
return q
}
func ensureTranscodeAllowed(user *models.User, cfg config.DownloadConfig) error {
if !cfg.TranscodeEnabled {
return ErrTranscodeDisabled
}
if user == nil || !user.DownloadTranscodeAllowed {
return ErrDownloadNotAllowed
}
return nil
}
func hasCapabilities(caps playback.ClientCapabilities) bool {
return len(caps.CodecsVideo) > 0 || len(caps.CodecsAudio) > 0 ||
len(caps.AudioPassthroughCodecs) > 0 || len(caps.Containers) > 0 ||
caps.MaxResolution != "" || caps.HDR
}
+152
View File
@@ -0,0 +1,152 @@
package downloads
import (
"errors"
"testing"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
func TestDownloadQualityResolverResolve(t *testing.T) {
file := &models.MediaFile{
ID: 1,
CodecVideo: "h264",
CodecAudio: "aac",
Container: "mp4",
Resolution: "1080p",
}
remuxFile := &models.MediaFile{
ID: 2,
CodecVideo: "h264",
CodecAudio: "aac",
Container: "mkv",
Resolution: "1080p",
}
transcodeFile := &models.MediaFile{
ID: 3,
CodecVideo: "hevc",
CodecAudio: "aac",
Container: "mp4",
Resolution: "1080p",
}
caps := playback.ClientCapabilities{
CodecsVideo: []string{"h264"},
CodecsAudio: []string{"aac"},
Containers: []string{"mp4"},
}
cases := []struct {
name string
requested string
file *models.MediaFile
caps playback.ClientCapabilities
transcodeEnabled bool
userTranscode bool
artifactsAvailable bool
wantFormat string
wantQuality string
wantEffective string
wantBitrate int
wantErr error
}{
{
name: "empty defaults to direct original without caps",
file: file,
wantFormat: FormatOriginal,
wantQuality: QualityOriginal,
wantEffective: QualityOriginal,
},
{
name: "bitrate quality resolves to transcode target",
requested: Quality5Mbps,
file: file,
transcodeEnabled: true,
userTranscode: true,
artifactsAvailable: true,
wantFormat: FormatTranscode,
wantQuality: Quality5Mbps,
wantEffective: Quality5Mbps,
wantBitrate: 5000,
},
{
name: "original falls back to remux when container is incompatible",
requested: QualityOriginal,
file: remuxFile,
caps: caps,
transcodeEnabled: true,
userTranscode: true,
artifactsAvailable: true,
wantFormat: FormatRemux,
wantQuality: QualityOriginal,
wantEffective: QualityOriginal,
},
{
name: "original falls back to 20mbps when video transcode is required",
requested: QualityOriginal,
file: transcodeFile,
caps: caps,
transcodeEnabled: true,
userTranscode: true,
artifactsAvailable: true,
wantFormat: FormatTranscode,
wantQuality: QualityOriginal,
wantEffective: Quality20Mbps,
wantBitrate: 20000,
},
{
name: "remux is not a public quality",
requested: FormatRemux,
file: file,
wantErr: ErrInvalidQuality,
},
{
name: "bitrate blocked by server gate",
requested: Quality10Mbps,
file: file,
userTranscode: true,
artifactsAvailable: true,
wantErr: ErrTranscodeDisabled,
},
{
name: "bitrate blocked by user flag",
requested: Quality10Mbps,
file: file,
transcodeEnabled: true,
artifactsAvailable: true,
wantErr: ErrDownloadNotAllowed,
},
{
name: "bitrate blocked without artifact pipeline",
requested: Quality10Mbps,
file: file,
transcodeEnabled: true,
userTranscode: true,
wantErr: ErrQualityUnavailable,
},
}
var resolver DownloadQualityResolver
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
user := &models.User{DownloadAllowed: true, DownloadTranscodeAllowed: tc.userTranscode}
cfg := config.DownloadConfig{Enabled: true, TranscodeEnabled: tc.transcodeEnabled}
got, err := resolver.Resolve(tc.requested, user, cfg, tc.file, tc.caps, tc.artifactsAvailable)
if tc.wantErr != nil {
if !errors.Is(err, tc.wantErr) {
t.Fatalf("Resolve(%q) err = %v, want %v", tc.requested, err, tc.wantErr)
}
return
}
if err != nil {
t.Fatalf("Resolve(%q) unexpected err: %v", tc.requested, err)
}
if got.DeliveryFormat != tc.wantFormat || got.RequestedQuality != tc.wantQuality ||
got.EffectiveQuality != tc.wantEffective || got.TargetBitrateKbps != tc.wantBitrate {
t.Fatalf("Resolve(%q) = %+v", tc.requested, got)
}
})
}
}
+663
View File
@@ -0,0 +1,663 @@
package downloads
import (
"context"
"errors"
"fmt"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
const downloadColumns = `id, user_id, profile_id, device_id, media_file_id, content_id, episode_id, batch_id,
kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at`
const insertDownloadSQL = `INSERT INTO downloads (id, user_id, profile_id, device_id, media_file_id, content_id,
episode_id, batch_id, kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22)`
// Repository provides CRUD operations for the downloads table.
type Repository struct {
pool *pgxpool.Pool
}
// NewRepository creates a new Repository backed by the given pool.
func NewRepository(pool *pgxpool.Pool) *Repository {
return &Repository{pool: pool}
}
// scanInto scans a single download row's columns (in downloadColumns order)
// into d, mapping nullable text columns to empty strings.
func scanInto(row pgx.Row, d *Download) error {
var profileID, deviceID, episodeID, batchID, artifactID *string
err := row.Scan(
&d.ID, &d.UserID, &profileID, &deviceID, &d.MediaFileID, &d.ContentID, &episodeID, &batchID,
&d.Kind, &d.Status, &d.Format, &d.Quality, &d.EffectiveQuality, &d.TargetBitrateKbps, &d.Revision, &artifactID, &d.FileSize, &d.BytesSent, &d.ErrorMessage,
&d.CreatedAt, &d.UpdatedAt, &d.CompletedAt,
)
if err != nil {
return err
}
d.ProfileID = deref(profileID)
d.DeviceID = deref(deviceID)
d.EpisodeID = deref(episodeID)
d.BatchID = deref(batchID)
d.ArtifactID = deref(artifactID)
return nil
}
func scanDownload(row pgx.Row) (*Download, error) {
var d Download
if err := scanInto(row, &d); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("scanning download: %w", err)
}
return &d, nil
}
func scanDownloads(rows pgx.Rows) ([]*Download, error) {
var downloads []*Download
for rows.Next() {
var d Download
if err := scanInto(rows, &d); err != nil {
return nil, fmt.Errorf("scanning download row: %w", err)
}
downloads = append(downloads, &d)
}
return downloads, rows.Err()
}
func (r *Repository) insertArgs(d *Download) []any {
format := d.Format
if format == "" {
format = FormatOriginal
}
quality := d.Quality
if quality == "" {
quality = QualityOriginal
}
effectiveQuality := d.EffectiveQuality
if effectiveQuality == "" {
effectiveQuality = quality
}
revision := d.Revision
if revision <= 0 {
revision = 1
}
return []any{
d.ID, d.UserID, nilIfEmpty(d.ProfileID), nilIfEmpty(d.DeviceID), d.MediaFileID, d.ContentID,
nilIfEmpty(d.EpisodeID), nilIfEmpty(d.BatchID), d.Kind, d.Status, format, quality, effectiveQuality,
d.TargetBitrateKbps, revision, nilIfEmpty(d.ArtifactID), d.FileSize, d.BytesSent, d.ErrorMessage,
d.CreatedAt, d.UpdatedAt, d.CompletedAt,
}
}
// Create inserts a new download record.
func (r *Repository) Create(ctx context.Context, d *Download) error {
if _, err := r.pool.Exec(ctx, insertDownloadSQL, r.insertArgs(d)...); err != nil {
return fmt.Errorf("inserting download: %w", err)
}
return nil
}
// CreateBatch inserts multiple download records in a single transaction.
func (r *Repository) CreateBatch(ctx context.Context, downloads []*Download) error {
tx, err := r.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("beginning batch insert: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
for _, d := range downloads {
if _, err := tx.Exec(ctx, insertDownloadSQL, r.insertArgs(d)...); err != nil {
return fmt.Errorf("inserting batch download: %w", err)
}
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("committing batch insert: %w", err)
}
return nil
}
// GetByID retrieves a download by its ID.
func (r *Repository) GetByID(ctx context.Context, id string) (*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads WHERE id = $1`
return scanDownload(r.pool.QueryRow(ctx, query, id))
}
// PruneEphemeralOlderThan deletes ephemeral (device-less) rows not touched
// since cutoff, regardless of status — they are convenience records for
// one-shot web downloads, not managed library entries. Pruning bounds
// GET /downloads growth for long-lived accounts and unpins artifacts
// (HasActiveLink counts ephemeral links). Returns the number of rows removed.
func (r *Repository) PruneEphemeralOlderThan(ctx context.Context, cutoff time.Time) (int64, error) {
tag, err := r.pool.Exec(ctx,
`DELETE FROM downloads WHERE device_id IS NULL AND updated_at < $1`, cutoff)
if err != nil {
return 0, fmt.Errorf("pruning ephemeral downloads: %w", err)
}
return tag.RowsAffected(), nil
}
// CountActiveByUser returns the number of active downloads for a user.
// 'preparing' counts: an artifact-backed row holds encode work in flight, so
// it must consume the concurrent quota like a live transfer does.
func (r *Repository) CountActiveByUser(ctx context.Context, userID int) (int, error) {
var count int
err := r.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM downloads WHERE user_id = $1 AND status IN ('queued', 'downloading', 'preparing')`,
userID,
).Scan(&count)
if err != nil {
return 0, fmt.Errorf("counting active downloads: %w", err)
}
return count, nil
}
// CountByUserSince returns the number of successful downloads created since the given time.
// Canceled and failed downloads are excluded so transient failures don't consume quota.
func (r *Repository) CountByUserSince(ctx context.Context, userID int, since time.Time) (int, error) {
var count int
err := r.pool.QueryRow(ctx,
`SELECT COUNT(*) FROM downloads WHERE user_id = $1 AND created_at >= $2 AND status NOT IN ('cancelled', 'failed')`,
userID, since,
).Scan(&count)
if err != nil {
return 0, fmt.Errorf("counting downloads in period: %w", err)
}
return count, nil
}
// UpdateStatus sets the status and optionally the bytes_sent and completed_at fields.
func (r *Repository) UpdateStatus(ctx context.Context, id, status string, bytesSent int64, completedAt *time.Time) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = $1, bytes_sent = $2, completed_at = $3, updated_at = now() WHERE id = $4`,
status, bytesSent, completedAt, id,
)
if err != nil {
return fmt.Errorf("updating download status: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// TransitionStatus atomically transitions a download from expectedStatus to newStatus.
// Returns ErrStatusConflict if the row is not in expectedStatus (another request
// already transitioned it).
func (r *Repository) TransitionStatus(ctx context.Context, id, expectedStatus, newStatus string, bytesSent int64, completedAt *time.Time) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = $1, bytes_sent = $2, completed_at = $3, updated_at = now()
WHERE id = $4 AND status = $5`,
newStatus, bytesSent, completedAt, id, expectedStatus,
)
if err != nil {
return fmt.Errorf("transitioning download status: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrStatusConflict
}
return nil
}
// Delete removes a download record. Returns ErrNotFound if the row doesn't exist
// or doesn't belong to the given user.
func (r *Repository) Delete(ctx context.Context, id string, userID int) error {
tag, err := r.pool.Exec(ctx,
`DELETE FROM downloads WHERE id = $1 AND user_id = $2`, id, userID,
)
if err != nil {
return fmt.Errorf("deleting download: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// CancelByID sets a download to canceled if it is still queued or downloading.
func (r *Repository) CancelByID(ctx context.Context, id string, userID int) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = 'cancelled', updated_at = now()
WHERE id = $1 AND user_id = $2 AND status IN ('queued', 'downloading')`,
id, userID,
)
if err != nil {
return fmt.Errorf("canceling download: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// EnsureDevice upserts a row in public.user_devices so a managed download's
// composite FK (user_id, profile_id, device_id) is satisfied even when the
// device's first request is the download itself. Mirrors the per-user store's
// RegisterDevice upsert; a no-op when profile or device is empty.
func (r *Repository) EnsureDevice(ctx context.Context, userID int, profileID, deviceID, deviceName, devicePlatform string) error {
if profileID == "" || deviceID == "" {
return nil
}
_, err := r.pool.Exec(ctx,
`INSERT INTO user_devices (user_id, profile_id, device_id, device_name, device_platform, last_seen_at)
VALUES ($1, $2, $3, $4, $5, now())
ON CONFLICT (user_id, profile_id, device_id) DO UPDATE SET
device_name = CASE WHEN excluded.device_name <> '' THEN excluded.device_name ELSE user_devices.device_name END,
device_platform = CASE WHEN excluded.device_platform <> '' THEN excluded.device_platform ELSE user_devices.device_platform END,
last_seen_at = now()`,
userID, profileID, deviceID, deviceName, devicePlatform,
)
if err != nil {
return fmt.Errorf("ensuring device %q: %w", deviceID, err)
}
return nil
}
// PurgeProfileDevices removes a profile's device rows (cascading its managed
// downloads and subscriptions via their composite FKs). Profiles may live
// outside Postgres, so no FK cascade covers user_devices on profile deletion;
// the profile-deletion handler calls this instead.
func (r *Repository) PurgeProfileDevices(ctx context.Context, userID int, profileID string) error {
if profileID == "" {
return nil
}
_, err := r.pool.Exec(ctx,
`DELETE FROM user_devices WHERE user_id = $1 AND profile_id = $2`,
userID, profileID,
)
if err != nil {
return fmt.Errorf("purging devices for profile %q: %w", profileID, err)
}
return nil
}
// GetManagedEntry returns the managed download uniquely identified by
// (user, profile, device, content, episode), or ErrNotFound. episodeID "" maps
// to the movie/no-episode slot via COALESCE(episode_id,”).
func (r *Repository) GetManagedEntry(ctx context.Context, userID int, profileID, deviceID, contentID, episodeID string) (*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3
AND content_id = $4 AND COALESCE(episode_id, '') = $5
LIMIT 1`
return scanDownload(r.pool.QueryRow(ctx, query, userID, profileID, deviceID, contentID, episodeID))
}
// CreateManagedEntry inserts d and returns it, or — if a concurrent insert won
// the race to the same managed identity — returns the existing winning row.
// Callers have already classified the item as new.
func (r *Repository) CreateManagedEntry(ctx context.Context, d *Download) (*Download, error) {
if err := r.Create(ctx, d); err != nil {
if existing, gerr := r.GetManagedEntry(ctx, d.UserID, d.ProfileID, d.DeviceID, d.ContentID, d.EpisodeID); gerr == nil {
return existing, nil
}
return nil, err
}
return d, nil
}
// ManagedEntryKey is the (content, episode) identity of a managed entry within
// one device library. EpisodeID "" is the movie/no-episode slot.
type ManagedEntryKey struct {
ContentID string
EpisodeID string
}
// GetManagedEntriesByKeys returns the device's existing managed rows for the
// given identities in one query, keyed by ManagedEntryKey. The bulk
// registration paths use this instead of a per-item GetManagedEntry loop (a
// 300-episode series would otherwise issue 300 sequential lookups).
func (r *Repository) GetManagedEntriesByKeys(ctx context.Context, userID int, profileID, deviceID string, keys []ManagedEntryKey) (map[ManagedEntryKey]*Download, error) {
out := make(map[ManagedEntryKey]*Download, len(keys))
if len(keys) == 0 {
return out, nil
}
contentIDs := make([]string, 0, len(keys))
episodeIDs := make([]string, 0, len(keys))
want := make(map[ManagedEntryKey]bool, len(keys))
for _, k := range keys {
contentIDs = append(contentIDs, k.ContentID)
episodeIDs = append(episodeIDs, k.EpisodeID)
want[k] = true
}
// ANY() on each column over-selects the cross product within this device's
// rows; the exact-pair filter below trims it.
rows, err := r.pool.Query(ctx,
`SELECT `+downloadColumns+` FROM downloads
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3
AND content_id = ANY($4) AND COALESCE(episode_id, '') = ANY($5)`,
userID, profileID, deviceID, contentIDs, episodeIDs,
)
if err != nil {
return nil, fmt.Errorf("listing managed entries by key: %w", err)
}
defer rows.Close()
list, err := scanDownloads(rows)
if err != nil {
return nil, err
}
for _, d := range list {
k := ManagedEntryKey{ContentID: d.ContentID, EpisodeID: d.EpisodeID}
if want[k] {
out[k] = d
}
}
return out, nil
}
// CreateManagedEntriesBatch inserts managed rows in one statement, skipping
// identities that already exist (including concurrent-insert races) via ON
// CONFLICT DO NOTHING on the managed-entry unique index. Returns only the rows
// actually inserted, so callers can report an honest newly-registered count.
func (r *Repository) CreateManagedEntriesBatch(ctx context.Context, ds []*Download) ([]*Download, error) {
if len(ds) == 0 {
return nil, nil
}
const insertCols = 22
var sb strings.Builder
sb.WriteString(`INSERT INTO downloads (id, user_id, profile_id, device_id, media_file_id, content_id,
episode_id, batch_id, kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at) VALUES `)
args := make([]any, 0, len(ds)*insertCols)
for i, d := range ds {
if i > 0 {
sb.WriteString(", ")
}
sb.WriteByte('(')
for j := 0; j < insertCols; j++ {
if j > 0 {
sb.WriteByte(',')
}
sb.WriteByte('$')
sb.WriteString(strconv.Itoa(i*insertCols + j + 1))
}
sb.WriteByte(')')
args = append(args, r.insertArgs(d)...)
}
sb.WriteString(` ON CONFLICT (user_id, profile_id, device_id, content_id, (COALESCE(episode_id, ''))) WHERE device_id IS NOT NULL DO NOTHING RETURNING ` + downloadColumns)
rows, err := r.pool.Query(ctx, sb.String(), args...)
if err != nil {
return nil, fmt.Errorf("batch inserting managed entries: %w", err)
}
defer rows.Close()
return scanDownloads(rows)
}
// ReplaceManagedEntry updates an existing managed row to a new file/quality
// target, incrementing its revision so clients can replace stale local bytes.
func (r *Repository) ReplaceManagedEntry(ctx context.Context, existing *Download, replacement *Download) (*Download, error) {
query := `UPDATE downloads SET
media_file_id = $6,
batch_id = $7,
kind = $8,
status = $9,
format = $10,
quality = $11,
effective_quality = $12,
target_bitrate_kbps = $13,
artifact_id = $14,
file_size = $15,
bytes_sent = 0,
error_message = '',
completed_at = NULL,
revision = revision + 1,
updated_at = now()
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4 AND revision = $5
RETURNING ` + downloadColumns
row := r.pool.QueryRow(ctx, query,
existing.ID, existing.UserID, existing.ProfileID, existing.DeviceID, existing.Revision,
replacement.MediaFileID, nilIfEmpty(replacement.BatchID), replacement.Kind, replacement.Status,
replacement.Format, replacement.Quality, replacement.EffectiveQuality, replacement.TargetBitrateKbps,
nilIfEmpty(replacement.ArtifactID), replacement.FileSize,
)
d, err := scanDownload(row)
if err != nil {
if errors.Is(err, ErrNotFound) {
return r.GetManagedEntry(ctx, existing.UserID, existing.ProfileID, existing.DeviceID, existing.ContentID, existing.EpisodeID)
}
return nil, fmt.Errorf("replacing managed download: %w", err)
}
return d, nil
}
// UpdateManagedBatch moves an otherwise unchanged managed row into the latest
// batch without incrementing revision or resetting local completion state.
func (r *Repository) UpdateManagedBatch(ctx context.Context, existing *Download, batchID string) (*Download, error) {
query := `UPDATE downloads SET
batch_id = $6,
updated_at = now()
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4 AND revision = $5
RETURNING ` + downloadColumns
row := r.pool.QueryRow(ctx, query,
existing.ID, existing.UserID, existing.ProfileID, existing.DeviceID, existing.Revision,
nilIfEmpty(batchID),
)
d, err := scanDownload(row)
if err != nil {
if errors.Is(err, ErrNotFound) {
return r.GetManagedEntry(ctx, existing.UserID, existing.ProfileID, existing.DeviceID, existing.ContentID, existing.EpisodeID)
}
return nil, fmt.Errorf("updating managed download batch: %w", err)
}
return d, nil
}
// SumManagedFileSize returns the total file_size of a device's managed entries
// that still count toward on-device storage (excluding revoked, failed, and
// canceled rows). It backs the auto-download storage soft-gate; it is the server's
// best-effort view, since the client is the source of truth for what is actually
// on disk.
func (r *Repository) SumManagedFileSize(ctx context.Context, userID int, profileID, deviceID string) (int64, error) {
var total int64
err := r.pool.QueryRow(ctx,
`SELECT COALESCE(SUM(file_size), 0) FROM downloads
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3
AND status NOT IN ('revoked', 'failed', 'cancelled')`,
userID, profileID, deviceID,
).Scan(&total)
if err != nil {
return 0, fmt.Errorf("summing managed download size: %w", err)
}
return total, nil
}
// GetManagedByID returns a managed download by id, authorized on
// (user_id, profile_id, device_id). A mismatch yields ErrNotFound so the
// endpoint never reveals the existence of another profile's/device's row.
func (r *Repository) GetManagedByID(ctx context.Context, id string, userID int, profileID, deviceID string) (*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4`
return scanDownload(r.pool.QueryRow(ctx, query, id, userID, profileID, deviceID))
}
// ListManaged returns the managed entries for one device, most recent first.
func (r *Repository) ListManaged(ctx context.Context, userID int, profileID, deviceID string) ([]*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3
ORDER BY created_at DESC`
rows, err := r.pool.Query(ctx, query, userID, profileID, deviceID)
if err != nil {
return nil, fmt.Errorf("listing managed downloads: %w", err)
}
defer rows.Close()
result, err := scanDownloads(rows)
if err != nil {
return nil, fmt.Errorf("scanning managed download rows: %w", err)
}
return result, nil
}
// ListManagedByBatch returns managed entries in a batch for one authorized
// device. It powers batch manifest fetches without revealing other devices' rows.
func (r *Repository) ListManagedByBatch(ctx context.Context, userID int, profileID, deviceID, batchID string) ([]*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3 AND batch_id = $4
ORDER BY created_at ASC`
rows, err := r.pool.Query(ctx, query, userID, profileID, deviceID, batchID)
if err != nil {
return nil, fmt.Errorf("listing managed batch downloads: %w", err)
}
defer rows.Close()
result, err := scanDownloads(rows)
if err != nil {
return nil, fmt.Errorf("scanning managed batch downloads: %w", err)
}
return result, nil
}
// ListEphemeral returns the account-level (device_id IS NULL) rows for a user.
func (r *Repository) ListEphemeral(ctx context.Context, userID int) ([]*Download, error) {
query := `SELECT ` + downloadColumns + ` FROM downloads
WHERE user_id = $1 AND device_id IS NULL
ORDER BY created_at DESC`
rows, err := r.pool.Query(ctx, query, userID)
if err != nil {
return nil, fmt.Errorf("listing ephemeral downloads: %w", err)
}
defer rows.Close()
result, err := scanDownloads(rows)
if err != nil {
return nil, fmt.Errorf("scanning ephemeral download rows: %w", err)
}
return result, nil
}
// UpdateManagedStatus sets a managed entry's status (client confirming local
// state), authorized on (user, profile, device). Only an artifact-ready entry
// may be moved into the client-driven serve lifecycle, so the transition is
// gated to source states ('ready','downloading','completed'); a 'preparing'
// (artifact not yet encoded), 'failed', or 'revoked' row is never patchable to
// downloading/completed. Returns ErrNotFound when nothing matches the gate.
func (r *Repository) UpdateManagedStatus(ctx context.Context, id string, userID int, profileID, deviceID, status string, completedAt *time.Time) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = $5, completed_at = $6, updated_at = now()
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4
AND status IN ('ready', 'downloading', 'completed')`,
id, userID, profileID, deviceID, status, completedAt,
)
if err != nil {
return fmt.Errorf("updating managed download status: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// DeleteManaged removes a managed entry, authorized on (user, profile, device).
// Returns ErrNotFound when nothing matches.
func (r *Repository) DeleteManaged(ctx context.Context, id string, userID int, profileID, deviceID string) error {
tag, err := r.pool.Exec(ctx,
`DELETE FROM downloads WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4`,
id, userID, profileID, deviceID,
)
if err != nil {
return fmt.Errorf("deleting managed download: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// MarkLinkedDownloadsReady flips every preparing download linked to a now-ready
// artifact to ready (recording the real size), returning the affected rows for
// client notification.
func (r *Repository) MarkLinkedDownloadsReady(ctx context.Context, artifactID string, fileSize int64) ([]*Download, error) {
rows, err := r.pool.Query(ctx,
`UPDATE downloads SET status = 'ready', file_size = $2, updated_at = now()
WHERE artifact_id = $1 AND status = 'preparing'
RETURNING `+downloadColumns,
artifactID, fileSize,
)
if err != nil {
return nil, fmt.Errorf("flipping linked downloads ready: %w", err)
}
defer rows.Close()
return scanDownloads(rows)
}
// MarkLinkedDownloadsFailed flips every preparing download linked to a failed
// artifact to failed, returning the affected rows for client notification.
func (r *Repository) MarkLinkedDownloadsFailed(ctx context.Context, artifactID, errMsg string) ([]*Download, error) {
rows, err := r.pool.Query(ctx,
`UPDATE downloads SET status = 'failed', error_message = $2, updated_at = now()
WHERE artifact_id = $1 AND status = 'preparing'
RETURNING `+downloadColumns,
artifactID, errMsg,
)
if err != nil {
return nil, fmt.Errorf("flipping linked downloads failed: %w", err)
}
defer rows.Close()
return scanDownloads(rows)
}
// ReconcileLinkedDownloads repairs downloads stranded in 'preparing' against a
// terminal artifact state. It covers the crash window between an artifact's
// MarkReady and its MarkLinkedDownloadsReady (the two are not one transaction),
// and any preparing row whose artifact reached 'failed' without its links being
// flipped. Two set-based updates (no per-artifact N+1): preparing→ready for rows
// linked to a ready artifact (recording the artifact size), and preparing→failed
// for rows linked to a failed artifact. Returns the rows it changed so the
// caller can publish state events. Idempotent — only 'preparing' rows are
// touched, so re-running it is a no-op.
func (r *Repository) ReconcileLinkedDownloads(ctx context.Context) (ready []*Download, failed []*Download, err error) {
readyRows, err := r.pool.Query(ctx,
`UPDATE downloads SET status = 'ready',
file_size = COALESCE((SELECT a.file_size FROM download_artifacts a WHERE a.id = downloads.artifact_id), file_size),
updated_at = now()
WHERE status = 'preparing' AND artifact_id IS NOT NULL
AND artifact_id IN (SELECT id FROM download_artifacts WHERE status = 'ready')
RETURNING `+downloadColumns,
)
if err != nil {
return nil, nil, fmt.Errorf("reconciling ready downloads: %w", err)
}
ready, err = scanDownloads(readyRows)
readyRows.Close()
if err != nil {
return nil, nil, err
}
failedRows, err := r.pool.Query(ctx,
`UPDATE downloads SET status = 'failed',
error_message = COALESCE((SELECT NULLIF(a.error_message, '') FROM download_artifacts a WHERE a.id = downloads.artifact_id), 'artifact encode failed'),
updated_at = now()
WHERE status = 'preparing' AND artifact_id IS NOT NULL
AND artifact_id IN (SELECT id FROM download_artifacts WHERE status = 'failed')
RETURNING `+downloadColumns,
)
if err != nil {
return nil, nil, fmt.Errorf("reconciling failed downloads: %w", err)
}
failed, err = scanDownloads(failedRows)
failedRows.Close()
if err != nil {
return nil, nil, err
}
return ready, failed, nil
}
func nilIfEmpty(s string) *string {
if s == "" {
return nil
}
return &s
}
func deref(s *string) string {
if s == nil {
return ""
}
return *s
}
+503
View File
@@ -0,0 +1,503 @@
package downloads
import (
"context"
"errors"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
)
// managedFixture is the seeded data a managed-entry authorization test needs:
// one user, two profiles on that same user_id, and a device per profile.
type managedFixture struct {
pool *pgxpool.Pool
repo *Repository
userID int
profileA string
profileB string
deviceA string
deviceB string
contentID string
fileID int
}
func newDownloadsTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
// Skip when the managed downloads reshape migration has not been applied.
var col *string
err = pool.QueryRow(ctx, `SELECT column_name FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'downloads' AND column_name = 'device_id'`).Scan(&col)
if errors.Is(err, pgx.ErrNoRows) || col == nil {
t.Skip("downloads managed reshape migration has not been applied")
}
if err != nil {
t.Fatalf("check downloads reshape: %v", err)
}
return pool
}
func seedManagedFixture(t *testing.T) managedFixture {
t.Helper()
ctx := context.Background()
pool := newDownloadsTestPool(t)
suffix := time.Now().UnixNano()
var folderID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_folders (type, name) VALUES ('movies', $1) RETURNING id`,
fmt.Sprintf("Downloads Test %d", suffix),
).Scan(&folderID); err != nil {
t.Fatalf("seed media folder: %v", err)
}
contentID := fmt.Sprintf("dl-content-%d", suffix)
var fileID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_files (content_id, media_folder_id, file_path, file_size)
VALUES ($1, $2, $3, 1024) RETURNING id`,
contentID, folderID, fmt.Sprintf("/tmp/downloads-test-%d.mp4", suffix),
).Scan(&fileID); err != nil {
t.Fatalf("seed media file: %v", err)
}
var userID int
if err := pool.QueryRow(ctx,
`INSERT INTO users (username, role, download_allowed) VALUES ($1, 'user', true) RETURNING id`,
fmt.Sprintf("dluser-%d", suffix),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
profileA := fmt.Sprintf("dlp-a-%d", suffix)
profileB := fmt.Sprintf("dlp-b-%d", suffix)
for _, p := range []struct{ id, name string }{{profileA, "A"}, {profileB, "B"}} {
if _, err := pool.Exec(ctx,
`INSERT INTO user_profiles (id, user_id, name) VALUES ($1, $2, $3)`,
p.id, userID, p.name,
); err != nil {
t.Fatalf("seed profile %s: %v", p.id, err)
}
}
repo := NewRepository(pool)
deviceA := fmt.Sprintf("dev-a-%d", suffix)
deviceB := fmt.Sprintf("dev-b-%d", suffix)
if err := repo.EnsureDevice(ctx, userID, profileA, deviceA, "Phone A", "android"); err != nil {
t.Fatalf("ensure device A: %v", err)
}
if err := repo.EnsureDevice(ctx, userID, profileB, deviceB, "Phone B", "android"); err != nil {
t.Fatalf("ensure device B: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM downloads WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_devices WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_profiles WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
return managedFixture{
pool: pool, repo: repo, userID: userID,
profileA: profileA, profileB: profileB, deviceA: deviceA, deviceB: deviceB,
contentID: contentID, fileID: fileID,
}
}
func (f managedFixture) createManagedEntry(t *testing.T) string {
t.Helper()
id := fmt.Sprintf("dl-%d", time.Now().UnixNano())
now := time.Now()
if err := f.repo.Create(context.Background(), &Download{
ID: id, UserID: f.userID, ProfileID: f.profileA, DeviceID: f.deviceA,
MediaFileID: f.fileID, ContentID: f.contentID, Kind: KindQueued,
Status: StatusReady, Format: FormatOriginal, FileSize: 1024,
CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create managed entry: %v", err)
}
return id
}
// TestManagedEntryCrossProfileDeviceDenied is the Phase 1 / invariant-2
// acceptance test: a second profile on the SAME user_id (and a different
// device) is denied a managed row's /file (GetManagedByID), PATCH
// (UpdateManagedStatus), and DELETE (DeleteManaged).
func TestManagedEntryCrossProfileDeviceDenied(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
id := f.createManagedEntry(t)
// Owner can read its own row.
if _, err := f.repo.GetManagedByID(ctx, id, f.userID, f.profileA, f.deviceA); err != nil {
t.Fatalf("owner GetManagedByID: %v", err)
}
// Cross-profile / cross-device reads are denied as not-found (no leak).
denials := []struct {
name string
profile, device string
}{
{"other profile + other device", f.profileB, f.deviceB},
{"same profile + other device", f.profileA, f.deviceB},
{"other profile + same device", f.profileB, f.deviceA},
}
for _, d := range denials {
t.Run("file/"+d.name, func(t *testing.T) {
if _, err := f.repo.GetManagedByID(ctx, id, f.userID, d.profile, d.device); !errors.Is(err, ErrNotFound) {
t.Fatalf("GetManagedByID(%s) err = %v, want ErrNotFound", d.name, err)
}
})
}
// PATCH by the second profile/device is denied; the owner succeeds.
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileB, f.deviceB, StatusCompleted, nil); !errors.Is(err, ErrNotFound) {
t.Fatalf("cross-profile PATCH err = %v, want ErrNotFound", err)
}
now := time.Now()
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileA, f.deviceA, StatusCompleted, &now); err != nil {
t.Fatalf("owner PATCH: %v", err)
}
// DELETE by the second profile/device is denied; the owner succeeds.
if err := f.repo.DeleteManaged(ctx, id, f.userID, f.profileB, f.deviceB); !errors.Is(err, ErrNotFound) {
t.Fatalf("cross-profile DELETE err = %v, want ErrNotFound", err)
}
if err := f.repo.DeleteManaged(ctx, id, f.userID, f.profileA, f.deviceA); err != nil {
t.Fatalf("owner DELETE: %v", err)
}
if _, err := f.repo.GetManagedByID(ctx, id, f.userID, f.profileA, f.deviceA); !errors.Is(err, ErrNotFound) {
t.Fatalf("after delete GetManagedByID err = %v, want ErrNotFound", err)
}
}
// TestManagedEntryUniqueAndRevoke covers the per-device unique constraint and
// the revoked-entry guard on status transitions.
func TestManagedEntryUniqueAndRevoke(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
id := f.createManagedEntry(t)
// A second managed entry for the same (user, profile, device, content,
// episode) violates the partial unique index.
dupNow := time.Now()
dupErr := f.repo.Create(ctx, &Download{
ID: id + "-dup", UserID: f.userID, ProfileID: f.profileA, DeviceID: f.deviceA,
MediaFileID: f.fileID, ContentID: f.contentID, Kind: KindQueued,
Status: StatusReady, Format: FormatOriginal, FileSize: 1024,
CreatedAt: dupNow, UpdatedAt: dupNow,
})
if dupErr == nil {
t.Fatal("expected unique-violation creating a duplicate managed entry, got nil")
}
// GetManagedEntry resolves the row by its unique key.
got, err := f.repo.GetManagedEntry(ctx, f.userID, f.profileA, f.deviceA, f.contentID, "")
if err != nil {
t.Fatalf("GetManagedEntry: %v", err)
}
if got.ID != id {
t.Fatalf("GetManagedEntry id = %q, want %q", got.ID, id)
}
// A revoked entry cannot be transitioned back out of revoked.
if _, err := f.pool.Exec(ctx, `UPDATE downloads SET status = 'revoked' WHERE id = $1`, id); err != nil {
t.Fatalf("revoke: %v", err)
}
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileA, f.deviceA, StatusCompleted, nil); !errors.Is(err, ErrNotFound) {
t.Fatalf("PATCH of revoked entry err = %v, want ErrNotFound", err)
}
}
// TestManagedStatusTransitionGate verifies UpdateManagedStatus only advances an
// artifact-ready entry into the client serve lifecycle: a 'preparing' (artifact
// still encoding) or 'failed' entry can never be marked downloading/completed.
func TestManagedStatusTransitionGate(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
id := f.createManagedEntry(t) // created StatusReady
// ready -> completed is allowed.
now := time.Now()
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileA, f.deviceA, StatusCompleted, &now); err != nil {
t.Fatalf("ready->completed: %v", err)
}
// A preparing entry (artifact not yet encoded) is not patchable.
if _, err := f.pool.Exec(ctx, `UPDATE downloads SET status = 'preparing' WHERE id = $1`, id); err != nil {
t.Fatalf("set preparing: %v", err)
}
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileA, f.deviceA, StatusCompleted, &now); !errors.Is(err, ErrNotFound) {
t.Fatalf("preparing->completed err = %v, want ErrNotFound", err)
}
// A failed entry is not patchable either.
if _, err := f.pool.Exec(ctx, `UPDATE downloads SET status = 'failed' WHERE id = $1`, id); err != nil {
t.Fatalf("set failed: %v", err)
}
if err := f.repo.UpdateManagedStatus(ctx, id, f.userID, f.profileA, f.deviceA, StatusDownloading, nil); !errors.Is(err, ErrNotFound) {
t.Fatalf("failed->downloading err = %v, want ErrNotFound", err)
}
}
// TestReconcileLinkedDownloads verifies recovery repairs downloads stranded in
// 'preparing' against a terminal artifact state: ready→ready (the crash window
// between MarkReady and MarkLinkedDownloadsReady) and failed→failed, and that
// re-running it is a no-op.
func TestReconcileLinkedDownloads(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
var present *string
if err := f.pool.QueryRow(ctx, `SELECT to_regclass('public.download_artifacts')::text`).Scan(&present); err != nil {
t.Fatalf("check download_artifacts: %v", err)
}
if present == nil {
t.Skip("download_artifacts migration has not been applied")
}
arepo := NewArtifactRepository(f.pool)
t.Cleanup(func() {
_, _ = f.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id = $1`, f.fileID)
})
suffix := time.Now().UnixNano()
// A ready artifact whose linked (ephemeral) download is stuck 'preparing'.
readyArt := newArtifact(t, f.fileID, fmt.Sprintf("hash-recon-ready-%d", suffix))
if _, _, err := arepo.EnsureQueued(ctx, readyArt); err != nil {
t.Fatalf("ensure ready artifact: %v", err)
}
if _, err := arepo.ClaimNext(ctx, "w", time.Minute); err != nil {
t.Fatalf("claim ready artifact: %v", err)
}
if ok, err := arepo.MarkReady(ctx, readyArt.ID, "w", "/tmp/ready.mp4", 4242); err != nil || !ok {
t.Fatalf("MarkReady = (%v, %v)", ok, err)
}
// A failed artifact whose linked download is stuck 'preparing'.
failedArt := newArtifact(t, f.fileID, fmt.Sprintf("hash-recon-failed-%d", suffix))
if _, _, err := arepo.EnsureQueued(ctx, failedArt); err != nil {
t.Fatalf("ensure failed artifact: %v", err)
}
for i := 0; i < failedArt.MaxAttempts; i++ {
if _, err := arepo.ClaimNext(ctx, "w", time.Minute); err != nil {
t.Fatalf("claim failed artifact: %v", err)
}
if _, _, err := arepo.MarkFailedOrRetry(ctx, failedArt.ID, "w", "boom", time.Millisecond); err != nil {
t.Fatalf("MarkFailedOrRetry: %v", err)
}
_, _ = f.pool.Exec(ctx, `UPDATE download_artifacts SET next_retry_at = now() - interval '1 second' WHERE id = $1`, failedArt.ID)
}
now := time.Now()
mkPreparing := func(artID string) string {
id := fmt.Sprintf("dl-%s-%d", artID, now.UnixNano())
if err := f.repo.Create(ctx, &Download{
ID: id, UserID: f.userID, MediaFileID: f.fileID, ContentID: f.contentID,
Kind: KindQueued, Status: StatusPreparing, Format: FormatTranscode, ArtifactID: artID,
CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create preparing download: %v", err)
}
return id
}
readyDL := mkPreparing(readyArt.ID)
failedDL := mkPreparing(failedArt.ID)
ready, failed, err := f.repo.ReconcileLinkedDownloads(ctx)
if err != nil {
t.Fatalf("ReconcileLinkedDownloads: %v", err)
}
if len(ready) != 1 || ready[0].ID != readyDL || ready[0].FileSize != 4242 {
t.Fatalf("ready flipped = %+v, want one %s with size 4242", ready, readyDL)
}
if len(failed) != 1 || failed[0].ID != failedDL {
t.Fatalf("failed flipped = %+v, want one %s", failed, failedDL)
}
gotReady, err := f.repo.GetByID(ctx, readyDL)
if err != nil || gotReady.Status != StatusReady {
t.Fatalf("ready download status = %v (%v), want ready", gotReady.Status, err)
}
gotFailed, err := f.repo.GetByID(ctx, failedDL)
if err != nil || gotFailed.Status != StatusFailed {
t.Fatalf("failed download status = %v (%v), want failed", gotFailed.Status, err)
}
// Idempotent: nothing left in 'preparing', so a second pass flips nothing.
ready2, failed2, err := f.repo.ReconcileLinkedDownloads(ctx)
if err != nil {
t.Fatalf("second ReconcileLinkedDownloads: %v", err)
}
if len(ready2) != 0 || len(failed2) != 0 {
t.Fatalf("second reconcile flipped ready=%d failed=%d, want 0/0", len(ready2), len(failed2))
}
}
// TestManagedEntryWithoutPostgresProfileRow is the dual-backend regression
// test: with the sqlite userdb backend, profiles exist only in per-user SQLite
// stores and public.user_profiles stays empty, so device registration and
// managed creates must not depend on a Postgres profile row (the
// user_devices_profile_fkey drop). Simulated here by never seeding
// user_profiles for the user.
func TestManagedEntryWithoutPostgresProfileRow(t *testing.T) {
ctx := context.Background()
pool := newDownloadsTestPool(t)
var fkExists bool
if err := pool.QueryRow(ctx,
`SELECT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'user_devices_profile_fkey')`,
).Scan(&fkExists); err != nil {
t.Fatalf("check profile fkey: %v", err)
}
if fkExists {
t.Skip("drop_user_devices_profile_fkey migration has not been applied")
}
suffix := time.Now().UnixNano()
var folderID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_folders (type, name) VALUES ('movies', $1) RETURNING id`,
fmt.Sprintf("Downloads SQLite Test %d", suffix),
).Scan(&folderID); err != nil {
t.Fatalf("seed media folder: %v", err)
}
contentID := fmt.Sprintf("dl-sqlite-content-%d", suffix)
var fileID int
if err := pool.QueryRow(ctx,
`INSERT INTO media_files (content_id, media_folder_id, file_path, file_size)
VALUES ($1, $2, $3, 1024) RETURNING id`,
contentID, folderID, fmt.Sprintf("/tmp/downloads-sqlite-test-%d.mp4", suffix),
).Scan(&fileID); err != nil {
t.Fatalf("seed media file: %v", err)
}
var userID int
if err := pool.QueryRow(ctx,
`INSERT INTO users (username, role, download_allowed) VALUES ($1, 'user', true) RETURNING id`,
fmt.Sprintf("dlsqlite-%d", suffix),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM downloads WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_devices WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
repo := NewRepository(pool)
profileID := fmt.Sprintf("sqlite-only-profile-%d", suffix)
deviceID := fmt.Sprintf("sqlite-dev-%d", suffix)
// No user_profiles row exists for profileID; this must still succeed.
if err := repo.EnsureDevice(ctx, userID, profileID, deviceID, "Phone", "android"); err != nil {
t.Fatalf("EnsureDevice without Postgres profile row: %v", err)
}
now := time.Now()
if _, err := repo.CreateManagedEntry(ctx, &Download{
ID: fmt.Sprintf("dl-sqlite-%d", suffix), UserID: userID, ProfileID: profileID,
DeviceID: deviceID, MediaFileID: fileID, ContentID: contentID, Kind: KindQueued,
Status: StatusReady, Format: FormatOriginal, FileSize: 1024,
CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("CreateManagedEntry without Postgres profile row: %v", err)
}
}
// TestPurgeProfileDevices verifies the app-level replacement for the dropped
// FK cascade: purging a profile removes its device rows and (via the
// downloads composite FK) its managed downloads, leaving other profiles'
// libraries untouched.
func TestPurgeProfileDevices(t *testing.T) {
ctx := context.Background()
f := seedManagedFixture(t)
id := f.createManagedEntry(t)
if err := f.repo.PurgeProfileDevices(ctx, f.userID, f.profileA); err != nil {
t.Fatalf("PurgeProfileDevices: %v", err)
}
if _, err := f.repo.GetManagedByID(ctx, id, f.userID, f.profileA, f.deviceA); !errors.Is(err, ErrNotFound) {
t.Fatalf("managed entry survived profile purge: err=%v", err)
}
var deviceRows int
if err := f.pool.QueryRow(ctx,
`SELECT count(*) FROM user_devices WHERE user_id = $1 AND profile_id = $2`,
f.userID, f.profileA,
).Scan(&deviceRows); err != nil || deviceRows != 0 {
t.Fatalf("profileA device rows = %d (%v), want 0", deviceRows, err)
}
var otherRows int
if err := f.pool.QueryRow(ctx,
`SELECT count(*) FROM user_devices WHERE user_id = $1 AND profile_id = $2`,
f.userID, f.profileB,
).Scan(&otherRows); err != nil || otherRows != 1 {
t.Fatalf("profileB device rows = %d (%v), want 1", otherRows, err)
}
}
// TestRegisterManagedItemsBatchAndCount pins the bulk-registration contract:
// registration is one batched fetch + one batched insert (not a per-episode
// loop), and the returned rows are ONLY the newly created ones, so the sync
// response's "registered" count reports 0 in the steady state.
func TestRegisterManagedItemsBatchAndCount(t *testing.T) {
ctx := context.Background()
f := seedManagedFixture(t)
mkItems := func(n int) []managedItem {
items := make([]managedItem, 0, n)
for i := 0; i < n; i++ {
items = append(items, managedItem{
file: &models.MediaFile{ID: f.fileID, FileSize: 1024},
contentID: f.contentID,
episodeID: fmt.Sprintf("reg-ep-%d", i),
})
}
return items
}
first, err := registerManagedItems(ctx, f.repo, f.userID, f.profileA, f.deviceA, mkItems(3), "batch-reg")
if err != nil {
t.Fatalf("first register: %v", err)
}
if len(first) != 3 {
t.Fatalf("first register = %d rows, want 3", len(first))
}
// Steady state: nothing new → zero rows returned.
again, err := registerManagedItems(ctx, f.repo, f.userID, f.profileA, f.deviceA, mkItems(3), "batch-reg")
if err != nil {
t.Fatalf("second register: %v", err)
}
if len(again) != 0 {
t.Fatalf("steady-state register = %d rows, want 0", len(again))
}
// A grown scope registers only the delta.
grown, err := registerManagedItems(ctx, f.repo, f.userID, f.profileA, f.deviceA, mkItems(5), "batch-reg")
if err != nil {
t.Fatalf("grown register: %v", err)
}
if len(grown) != 2 {
t.Fatalf("grown register = %d rows, want 2", len(grown))
}
}
File diff suppressed because it is too large Load Diff
+100
View File
@@ -0,0 +1,100 @@
package downloads
import (
"context"
"errors"
"fmt"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/playback"
)
// stubQuotaPreparer satisfies EncodePreparer without running ffmpeg; the quota
// test never drains the queue, so it is never invoked.
type stubQuotaPreparer struct{}
func (stubQuotaPreparer) PrepareFile(_ context.Context, _ playback.TranscodeOpts, _ string) error {
return nil
}
// TestArtifactQuotaCheckedBeforeEnqueue pins two quota invariants for
// artifact-backed downloads: 'preparing' rows consume the concurrent quota,
// and a quota-rejected request must not leave an encode job behind.
func TestArtifactQuotaCheckedBeforeEnqueue(t *testing.T) {
ctx := context.Background()
f := seedManagedFixture(t)
var present *string
if err := f.pool.QueryRow(ctx, `SELECT to_regclass('public.download_artifacts')::text`).Scan(&present); err != nil {
t.Fatalf("check download_artifacts: %v", err)
}
if present == nil {
t.Skip("download_artifacts migration has not been applied")
}
// Second media file so the rejected request would need a distinct artifact.
suffix := time.Now().UnixNano()
var folderID int
if err := f.pool.QueryRow(ctx, `SELECT media_folder_id FROM media_files WHERE id = $1`, f.fileID).Scan(&folderID); err != nil {
t.Fatalf("resolve folder: %v", err)
}
contentID2 := fmt.Sprintf("dl-quota-content-%d", suffix)
var fileID2 int
if err := f.pool.QueryRow(ctx,
`INSERT INTO media_files (content_id, media_folder_id, file_path, file_size)
VALUES ($1, $2, $3, 2048) RETURNING id`,
contentID2, folderID, fmt.Sprintf("/tmp/downloads-quota-test-%d.mp4", suffix),
).Scan(&fileID2); err != nil {
t.Fatalf("seed second media file: %v", err)
}
t.Cleanup(func() {
_, _ = f.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id IN ($1, $2)`, f.fileID, fileID2)
_, _ = f.pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID2)
})
limiter := NewQuantityLimiter(f.repo, 1, 0, 0)
svc := NewService(f.repo, nil, limiter, nil, nil, nil, nil, nil, nil, &config.DownloadConfig{Enabled: true})
svc.SetArtifactManager(NewArtifactManager(
NewArtifactRepository(f.pool), f.repo, nil, stubQuotaPreparer{}, "quota-test",
func() *config.Config { return nil }, nil,
))
decision := QualityDecision{
RequestedQuality: Quality5Mbps,
EffectiveQuality: Quality5Mbps,
DeliveryFormat: FormatTranscode,
TargetBitrateKbps: 5000,
RequiresArtifact: true,
PrepareTarget: playback.PrepareTarget{Container: "mp4", CodecVideo: "h264", CodecAudio: "aac", TargetBitrateKbps: 5000},
}
first, err := svc.createArtifactDownload(ctx, f.userID, CreateRequest{Quality: Quality5Mbps},
&models.MediaFile{ID: f.fileID, ContentID: f.contentID, FileSize: 1024}, decision)
if err != nil {
t.Fatalf("first artifact download: %v", err)
}
if first.Status != StatusPreparing {
t.Fatalf("first download status = %q, want %q", first.Status, StatusPreparing)
}
// Over the concurrent cap of 1: the 'preparing' row above must count.
_, err = svc.createArtifactDownload(ctx, f.userID, CreateRequest{Quality: Quality5Mbps},
&models.MediaFile{ID: fileID2, ContentID: contentID2, FileSize: 2048}, decision)
if !errors.Is(err, ErrConcurrentLimitReached) {
t.Fatalf("second artifact download error = %v, want ErrConcurrentLimitReached", err)
}
// The rejected request must not have enqueued encode work.
var enqueued int
if err := f.pool.QueryRow(ctx,
`SELECT count(*) FROM download_artifacts WHERE media_file_id = $1`, fileID2,
).Scan(&enqueued); err != nil {
t.Fatalf("count artifacts: %v", err)
}
if enqueued != 0 {
t.Fatalf("rejected request enqueued %d artifact job(s), want 0", enqueued)
}
}
+39
View File
@@ -0,0 +1,39 @@
package downloads
import "testing"
func TestReusableManagedStatus(t *testing.T) {
for _, status := range []string{StatusReady, StatusPreparing, StatusDownloading, StatusCompleted} {
if !reusableManagedStatus(status) {
t.Fatalf("status %q should be reusable", status)
}
}
for _, status := range []string{StatusCancelled, StatusFailed, StatusRevoked} {
if reusableManagedStatus(status) {
t.Fatalf("status %q should force replacement", status)
}
}
}
func TestSameManagedTargetIgnoresBatch(t *testing.T) {
a := &Download{
MediaFileID: 1,
BatchID: "old",
Format: FormatTranscode,
Quality: Quality5Mbps,
EffectiveQuality: Quality5Mbps,
TargetBitrateKbps: 5000,
ArtifactID: "artifact",
FileSize: 123,
}
b := *a
b.BatchID = "new"
if !sameManagedTarget(a, &b) {
t.Fatal("batch changes should not force media replacement")
}
b.TargetBitrateKbps = 2000
if sameManagedTarget(a, &b) {
t.Fatal("bitrate changes should force media replacement")
}
}
+135
View File
@@ -0,0 +1,135 @@
package downloads
import (
"time"
"github.com/Silo-Server/silo-server/internal/models"
)
// Subscription monitor modes — which episodes of a series a download
// subscription keeps on the device.
const (
// SubModeAll keeps every episode of the series (existing + future).
SubModeAll = "all"
// SubModeFuture keeps only episodes that become available after subscribing
// (no backfill of the existing catalog).
SubModeFuture = "future"
// SubModeLatestSeason keeps the latest season at subscribe time and any newer
// season going forward (episodes whose season_number >= TargetSeason).
SubModeLatestSeason = "latest_season"
// SubModeSpecificSeasons keeps only the seasons listed in SeasonNumbers.
SubModeSpecificSeasons = "specific_seasons"
)
// ValidSubMode reports whether mode is a recognized subscription mode.
func ValidSubMode(mode string) bool {
switch mode {
case SubModeAll, SubModeFuture, SubModeLatestSeason, SubModeSpecificSeasons:
return true
default:
return false
}
}
// Subscription is a device-scoped opt-in to keep a series downloaded on one
// device ("monitor a series"). The client triggers a sync (on open / background
// refresh); the server then registers the in-scope, not-yet-downloaded episodes.
// It is keyed on (UserID, ProfileID, DeviceID, SeriesID) and is deliberately
// separate from the device-less, derived profile_series_interest the
// notifications system uses.
type Subscription struct {
ID string
UserID int
ProfileID string
DeviceID string
SeriesID string
Mode string // see SubMode* constants
SeasonNumbers []int // monitored seasons when Mode == SubModeSpecificSeasons
TargetSeason *int // latest season at subscribe time when Mode == SubModeLatestSeason
DeleteWatched bool // client-enforced: delete episodes once watched
MaxStorageBytes int64 // 0 = unlimited; client-enforced, server soft-gates auto-registration
Active bool
CreatedAt time.Time
UpdatedAt time.Time
}
// CoversSeason reports whether an episode in the given season number falls
// within this subscription's monitored scope. SubModeAll and SubModeFuture cover
// every season — they differ only in SubModeFuture's air-date cutoff (see
// coversEpisode). SubModeLatestSeason covers the subscribe-time latest season and
// every newer one (season_number >= TargetSeason), so a new season is picked up
// on the next sync.
func (s *Subscription) CoversSeason(seasonNumber int) bool {
switch s.Mode {
case SubModeAll, SubModeFuture:
return true
case SubModeLatestSeason:
return s.TargetSeason != nil && seasonNumber >= *s.TargetSeason
case SubModeSpecificSeasons:
for _, n := range s.SeasonNumbers {
if n == seasonNumber {
return true
}
}
return false
default:
return false
}
}
// coversEpisode reports whether a specific available episode is in scope for a
// sync. It layers SubModeFuture's "only episodes that aired after I subscribed"
// cutoff onto the season scope, so future-only never registers the back
// catalog. An episode with no air date falls back to its ingest time.
func (s *Subscription) coversEpisode(ep *models.Episode) bool {
if !s.CoversSeason(ep.SeasonNumber) {
return false
}
if s.Mode == SubModeFuture {
// air_date is a date-only column (midnight), so a strict instant
// comparison against the subscribe timestamp would permanently exclude
// an episode airing the same day the user subscribed. Compare calendar
// days (UTC) instead; with no air date, a genuinely new episode still
// counts as future via its ingest time.
sc := s.CreatedAt.UTC()
cutoff := time.Date(sc.Year(), sc.Month(), sc.Day(), 0, 0, 0, 0, time.UTC)
if ep.AirDate != nil {
return !ep.AirDate.Before(cutoff)
}
return !ep.CreatedAt.Before(s.CreatedAt)
}
return true
}
// Admits reports whether adding an item of addBytes keeps the device within this
// subscription's storage budget. MaxStorageBytes <= 0 means unlimited. It is the
// single home of the storage-cap rule shared by backfill and the live worker;
// the caller owns reading the current usage (best-effort, client-enforced).
func (s *Subscription) Admits(usedBytes, addBytes int64) bool {
return s.MaxStorageBytes <= 0 || usedBytes+addBytes <= s.MaxStorageBytes
}
// SubscriptionRequest is the input to create (or re-create) a subscription. The
// identity fields are taken from the X-Silo-Device-Id header and the
// viewer-access profile, never the body.
type SubscriptionRequest struct {
SeriesID string
Mode string
SeasonNumbers []int
DeleteWatched bool
MaxStorageBytes int64
ProfileID string
DeviceID string
DeviceName string
DevicePlatform string
}
// SubscriptionPatch is a partial update to a subscription; nil fields are left
// unchanged.
type SubscriptionPatch struct {
Mode *string
SeasonNumbers *[]int
DeleteWatched *bool
MaxStorageBytes *int64
Active *bool
}
+178
View File
@@ -0,0 +1,178 @@
package downloads
import (
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
const subscriptionColumns = `id, user_id, profile_id, device_id, series_id, mode,
season_numbers, target_season, delete_watched, max_storage_bytes, active, created_at, updated_at`
// SubscriptionRepository provides CRUD for download_subscriptions. Monitoring
// is client-pull only: devices sync on app open / background refresh; there is
// no server-side auto-register worker.
type SubscriptionRepository struct {
pool *pgxpool.Pool
}
// NewSubscriptionRepository creates a SubscriptionRepository backed by pool.
func NewSubscriptionRepository(pool *pgxpool.Pool) *SubscriptionRepository {
return &SubscriptionRepository{pool: pool}
}
func scanSubscriptionInto(row pgx.Row, s *Subscription) error {
var seasons []int32
var target *int32
if err := row.Scan(
&s.ID, &s.UserID, &s.ProfileID, &s.DeviceID, &s.SeriesID, &s.Mode,
&seasons, &target, &s.DeleteWatched, &s.MaxStorageBytes, &s.Active,
&s.CreatedAt, &s.UpdatedAt,
); err != nil {
return err
}
s.SeasonNumbers = int32sToInts(seasons)
if target != nil {
t := int(*target)
s.TargetSeason = &t
}
return nil
}
func scanSubscription(row pgx.Row) (*Subscription, error) {
var s Subscription
if err := scanSubscriptionInto(row, &s); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrSubscriptionNotFound
}
return nil, fmt.Errorf("scanning subscription: %w", err)
}
return &s, nil
}
func scanSubscriptions(rows pgx.Rows) ([]*Subscription, error) {
var subs []*Subscription
for rows.Next() {
var s Subscription
if err := scanSubscriptionInto(rows, &s); err != nil {
return nil, fmt.Errorf("scanning subscription row: %w", err)
}
subs = append(subs, &s)
}
return subs, rows.Err()
}
// Upsert inserts the subscription, or updates the existing one for the same
// (user, profile, device, series) — re-monitoring a series is idempotent and
// rewrites the mode/options in place. created_at is preserved on update so the
// future-only cutoff stays anchored to the first subscribe. Returns the stored row.
func (r *SubscriptionRepository) Upsert(ctx context.Context, s *Subscription) (*Subscription, error) {
const q = `INSERT INTO download_subscriptions
(id, user_id, profile_id, device_id, series_id, mode, season_numbers, target_season,
delete_watched, max_storage_bytes, active, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, true, now(), now())
ON CONFLICT (user_id, profile_id, device_id, series_id) DO UPDATE SET
mode = excluded.mode,
season_numbers = excluded.season_numbers,
target_season = excluded.target_season,
delete_watched = excluded.delete_watched,
max_storage_bytes = excluded.max_storage_bytes,
active = true,
updated_at = now()
RETURNING ` + subscriptionColumns
return scanSubscription(r.pool.QueryRow(ctx, q,
s.ID, s.UserID, s.ProfileID, s.DeviceID, s.SeriesID, s.Mode,
intsToInt32s(s.SeasonNumbers), int32Ptr(s.TargetSeason),
s.DeleteWatched, s.MaxStorageBytes))
}
// GetByID returns a subscription by id, authorized on (user, profile, device).
// A mismatch yields ErrSubscriptionNotFound so the endpoint never reveals the
// existence of another profile's/device's subscription.
func (r *SubscriptionRepository) GetByID(ctx context.Context, id string, userID int, profileID, deviceID string) (*Subscription, error) {
q := `SELECT ` + subscriptionColumns + ` FROM download_subscriptions
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4`
return scanSubscription(r.pool.QueryRow(ctx, q, id, userID, profileID, deviceID))
}
// ListByDevice returns the calling device's subscriptions, most recent first.
func (r *SubscriptionRepository) ListByDevice(ctx context.Context, userID int, profileID, deviceID string) ([]*Subscription, error) {
q := `SELECT ` + subscriptionColumns + ` FROM download_subscriptions
WHERE user_id = $1 AND profile_id = $2 AND device_id = $3
ORDER BY created_at DESC`
rows, err := r.pool.Query(ctx, q, userID, profileID, deviceID)
if err != nil {
return nil, fmt.Errorf("listing subscriptions: %w", err)
}
defer rows.Close()
return scanSubscriptions(rows)
}
// Update writes the mutable fields of s (mode/seasons/target/delete_watched/
// max_storage_bytes/active), authorized on (user, profile, device). Returns
// ErrSubscriptionNotFound when nothing matches.
func (r *SubscriptionRepository) Update(ctx context.Context, s *Subscription) error {
tag, err := r.pool.Exec(ctx,
`UPDATE download_subscriptions
SET mode = $5, season_numbers = $6, target_season = $7, delete_watched = $8,
max_storage_bytes = $9, active = $10, updated_at = now()
WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4`,
s.ID, s.UserID, s.ProfileID, s.DeviceID,
s.Mode, intsToInt32s(s.SeasonNumbers), int32Ptr(s.TargetSeason),
s.DeleteWatched, s.MaxStorageBytes, s.Active,
)
if err != nil {
return fmt.Errorf("updating subscription: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrSubscriptionNotFound
}
return nil
}
// Delete removes a subscription, authorized on (user, profile, device). It does
// NOT delete already-downloaded episodes (the client owns on-device deletion).
// Returns ErrSubscriptionNotFound when nothing matches.
func (r *SubscriptionRepository) Delete(ctx context.Context, id string, userID int, profileID, deviceID string) error {
tag, err := r.pool.Exec(ctx,
`DELETE FROM download_subscriptions WHERE id = $1 AND user_id = $2 AND profile_id = $3 AND device_id = $4`,
id, userID, profileID, deviceID,
)
if err != nil {
return fmt.Errorf("deleting subscription: %w", err)
}
if tag.RowsAffected() == 0 {
return ErrSubscriptionNotFound
}
return nil
}
func intsToInt32s(in []int) []int32 {
out := make([]int32, len(in))
for i, v := range in {
out[i] = int32(v)
}
return out
}
func int32sToInts(in []int32) []int {
if len(in) == 0 {
return nil
}
out := make([]int, len(in))
for i, v := range in {
out[i] = int(v)
}
return out
}
func int32Ptr(p *int) *int32 {
if p == nil {
return nil
}
v := int32(*p)
return &v
}
+371
View File
@@ -0,0 +1,371 @@
package downloads
import (
"context"
"fmt"
"log/slog"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/idgen"
"github.com/Silo-Server/silo-server/internal/models"
)
// SubscriptionResult is the outcome of creating/updating a subscription: the
// stored subscription plus how many in-scope episodes were registered as managed
// downloads.
type SubscriptionResult struct {
Subscription *Subscription
Registered int
}
// CreateSubscription creates (or idempotently re-creates) a device-scoped series
// monitor and registers the in-scope episodes that already exist as managed
// downloads. New episodes are picked up by later client-triggered syncs (see
// SyncSubscriptions). Series monitoring is original-only, like the rest of the
// series flow.
func (s *Service) CreateSubscription(ctx context.Context, userID int, req SubscriptionRequest, filter catalog.AccessFilter) (*SubscriptionResult, error) {
if s.subRepo == nil {
return nil, ErrSubscriptionsUnavailable
}
if _, err := s.enabledConfig(ctx); err != nil {
return nil, err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return nil, fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return nil, ErrDownloadNotAllowed
}
if req.ProfileID == "" || req.DeviceID == "" {
return nil, ErrProfileRequired
}
if !ValidSubMode(req.Mode) {
return nil, ErrInvalidSubscriptionMode
}
if req.Mode == SubModeSpecificSeasons && len(req.SeasonNumbers) == 0 {
return nil, ErrSeasonsRequired
}
if !validSeasonNumbers(req.SeasonNumbers) {
return nil, ErrInvalidSeasonNumbers
}
item, err := s.itemRepo.GetByID(ctx, req.SeriesID)
if err != nil {
return nil, fmt.Errorf("loading series: %w", err)
}
if item.Type != "series" {
return nil, ErrNotSeries
}
if err := s.itemAccess.EnsureAccessible(ctx, req.SeriesID, filter); err != nil {
return nil, err
}
// The device row must exist for the subscription's composite FK.
if err := s.repo.EnsureDevice(ctx, userID, req.ProfileID, req.DeviceID, req.DeviceName, req.DevicePlatform); err != nil {
return nil, err
}
id, err := idgen.NextID()
if err != nil {
return nil, fmt.Errorf("generating subscription ID: %w", err)
}
sub := &Subscription{
ID: id,
UserID: userID,
ProfileID: req.ProfileID,
DeviceID: req.DeviceID,
SeriesID: req.SeriesID,
Mode: req.Mode,
SeasonNumbers: normalizeSeasons(req.Mode, req.SeasonNumbers),
DeleteWatched: req.DeleteWatched,
MaxStorageBytes: req.MaxStorageBytes,
Active: true,
}
if req.Mode == SubModeLatestSeason {
target, err := s.latestSeason(ctx, req.SeriesID)
if err != nil {
return nil, err
}
sub.TargetSeason = target
}
stored, err := s.subRepo.Upsert(ctx, sub)
if err != nil {
return nil, err
}
// The initial registration is best-effort: the subscription is durably stored
// and later client syncs still pick up episodes; a transient failure here can
// be retried by syncing or re-monitoring. So log it rather than failing.
registered, err := s.syncSubscription(ctx, stored)
if err != nil {
slog.Warn("download subscription initial sync failed", "subscription_id", stored.ID, "error", err)
}
return &SubscriptionResult{Subscription: stored, Registered: registered}, nil
}
// ListSubscriptions returns the calling device's subscriptions.
func (s *Service) ListSubscriptions(ctx context.Context, userID int, profileID, deviceID string) ([]*Subscription, error) {
if s.subRepo == nil {
return nil, ErrSubscriptionsUnavailable
}
if profileID == "" || deviceID == "" {
return nil, ErrProfileRequired
}
return s.subRepo.ListByDevice(ctx, userID, profileID, deviceID)
}
// GetSubscription returns one subscription, authorized on (user, profile, device).
func (s *Service) GetSubscription(ctx context.Context, userID int, profileID, deviceID, id string) (*Subscription, error) {
if s.subRepo == nil {
return nil, ErrSubscriptionsUnavailable
}
if profileID == "" || deviceID == "" {
return nil, ErrProfileRequired
}
return s.subRepo.GetByID(ctx, id, userID, profileID, deviceID)
}
// UpdateSubscription applies a partial update, re-anchoring the latest-season
// target when the mode changes, then registers any newly in-scope episodes.
func (s *Service) UpdateSubscription(ctx context.Context, userID int, profileID, deviceID, id string, patch SubscriptionPatch, filter catalog.AccessFilter) (*SubscriptionResult, error) {
if s.subRepo == nil {
return nil, ErrSubscriptionsUnavailable
}
if profileID == "" || deviceID == "" {
return nil, ErrProfileRequired
}
// Same feature/permission gate as CreateSubscription and SyncSubscriptions:
// a patch can re-activate or widen a monitor and backfill managed rows, so
// it must not bypass an admin disabling downloads or revoking the user.
if _, err := s.enabledConfig(ctx); err != nil {
return nil, err
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return nil, fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return nil, ErrDownloadNotAllowed
}
sub, err := s.subRepo.GetByID(ctx, id, userID, profileID, deviceID)
if err != nil {
return nil, err
}
// Re-check the requesting profile's access to the series before mutating or
// backfilling — access may have been revoked since the subscription was made.
if err := s.itemAccess.EnsureAccessible(ctx, sub.SeriesID, filter); err != nil {
return nil, err
}
scopeChanged := patch.Mode != nil || patch.SeasonNumbers != nil
wasActive := sub.Active
oldMaxStorageBytes := sub.MaxStorageBytes
modeChanged := false
if patch.Mode != nil {
if !ValidSubMode(*patch.Mode) {
return nil, ErrInvalidSubscriptionMode
}
modeChanged = *patch.Mode != sub.Mode
sub.Mode = *patch.Mode
}
if patch.SeasonNumbers != nil {
sub.SeasonNumbers = *patch.SeasonNumbers
}
if patch.DeleteWatched != nil {
sub.DeleteWatched = *patch.DeleteWatched
}
if patch.MaxStorageBytes != nil {
sub.MaxStorageBytes = *patch.MaxStorageBytes
}
if patch.Active != nil {
sub.Active = *patch.Active
}
if sub.Mode == SubModeSpecificSeasons && len(sub.SeasonNumbers) == 0 {
return nil, ErrSeasonsRequired
}
if !validSeasonNumbers(sub.SeasonNumbers) {
return nil, ErrInvalidSeasonNumbers
}
sub.SeasonNumbers = normalizeSeasons(sub.Mode, sub.SeasonNumbers)
switch {
case sub.Mode == SubModeLatestSeason && (modeChanged || sub.TargetSeason == nil):
target, err := s.latestSeason(ctx, sub.SeriesID)
if err != nil {
return nil, err
}
sub.TargetSeason = target
case sub.Mode != SubModeLatestSeason:
sub.TargetSeason = nil
}
if err := s.subRepo.Update(ctx, sub); err != nil {
return nil, err
}
reactivated := patch.Active != nil && !wasActive && sub.Active
storageIncreased := patch.MaxStorageBytes != nil &&
oldMaxStorageBytes > 0 &&
(sub.MaxStorageBytes <= 0 || sub.MaxStorageBytes > oldMaxStorageBytes)
// A paused subscription never syncs — including when this very patch
// paused it while also changing scope. SyncSubscriptions applies the same
// guard; registering episodes the user just stopped monitoring would make
// the device pull them anyway.
if !sub.Active || (!scopeChanged && !reactivated && !storageIncreased) {
return &SubscriptionResult{Subscription: sub}, nil
}
registered, err := s.syncSubscription(ctx, sub)
if err != nil {
slog.Warn("download subscription update sync failed", "subscription_id", sub.ID, "error", err)
}
return &SubscriptionResult{Subscription: sub, Registered: registered}, nil
}
// DeleteSubscription stops monitoring a series for the device. It never deletes
// already-downloaded episodes — the client owns on-device deletion.
func (s *Service) DeleteSubscription(ctx context.Context, userID int, profileID, deviceID, id string) error {
if s.subRepo == nil {
return ErrSubscriptionsUnavailable
}
if profileID == "" || deviceID == "" {
return ErrProfileRequired
}
return s.subRepo.Delete(ctx, id, userID, profileID, deviceID)
}
// SyncSubscriptions registers newly in-scope episodes for all of the calling
// device's active monitors. The client calls this on open / background refresh
// and then pulls the registered files on its own schedule — so monitoring needs
// no server-side worker and no notifications subsystem. Per-series access is
// re-checked, and one series failing does not fail the whole sync.
func (s *Service) SyncSubscriptions(ctx context.Context, userID int, profileID, deviceID string, filter catalog.AccessFilter) (int, error) {
if s.subRepo == nil {
return 0, ErrSubscriptionsUnavailable
}
if _, err := s.enabledConfig(ctx); err != nil {
return 0, err
}
if profileID == "" || deviceID == "" {
return 0, ErrProfileRequired
}
user, err := s.userRepo.GetByID(ctx, userID)
if err != nil {
return 0, fmt.Errorf("loading user: %w", err)
}
if !user.DownloadAllowed {
return 0, ErrDownloadNotAllowed
}
subs, err := s.subRepo.ListByDevice(ctx, userID, profileID, deviceID)
if err != nil {
return 0, err
}
total := 0
for _, sub := range subs {
if !sub.Active {
continue
}
if err := s.itemAccess.EnsureAccessible(ctx, sub.SeriesID, filter); err != nil {
continue // series no longer accessible to this profile; skip it silently
}
n, err := s.syncSubscription(ctx, sub)
if err != nil {
slog.Warn("download subscription sync failed", "subscription_id", sub.ID, "error", err)
continue
}
total += n
}
return total, nil
}
// syncSubscription registers the in-scope, available episodes a subscription
// covers as managed downloads (idempotent — already-registered episodes are
// skipped) and returns how many were NEWLY registered, so a steady-state sync
// reports 0. Run at create/update time and on each client-triggered sync. One
// ListBySeries + coversEpisode filter handles every mode, including latest_season
// following new seasons (>= TargetSeason) and future-only excluding the back
// catalog (aired on/after the subscribe day).
func (s *Service) syncSubscription(ctx context.Context, sub *Subscription) (int, error) {
episodes, err := s.episodeRepo.ListBySeries(ctx, sub.SeriesID)
if err != nil {
return 0, fmt.Errorf("listing episodes: %w", err)
}
inScope := make([]*models.Episode, 0, len(episodes))
for _, ep := range episodes {
if sub.coversEpisode(ep) {
inScope = append(inScope, ep)
}
}
items, err := s.episodeItems(ctx, sub.SeriesID, inScope)
if err != nil {
return 0, err
}
items = s.capItemsToStorage(ctx, sub, items)
rows, err := registerManagedItems(ctx, s.repo, sub.UserID, sub.ProfileID, sub.DeviceID, items, sub.ID)
return len(rows), err
}
// capItemsToStorage trims items so registering them keeps the device under the
// subscription's max_storage_bytes (0 = unlimited). The server sum is a
// best-effort view; the client enforces the hard cap and owns deletion.
func (s *Service) capItemsToStorage(ctx context.Context, sub *Subscription, items []managedItem) []managedItem {
if sub.MaxStorageBytes <= 0 || len(items) == 0 {
return items
}
used, err := s.repo.SumManagedFileSize(ctx, sub.UserID, sub.ProfileID, sub.DeviceID)
if err != nil {
slog.Warn("download subscription storage gate: sum failed; skipping cap", "subscription_id", sub.ID, "error", err)
return items
}
kept := make([]managedItem, 0, len(items))
for _, it := range items {
if !sub.Admits(used, it.file.FileSize) {
break
}
used += it.file.FileSize
kept = append(kept, it)
}
return kept
}
// latestSeason returns the highest season number that has available episodes,
// used to anchor a latest_season subscription. Returns nil when the series has
// no available seasons.
func (s *Service) latestSeason(ctx context.Context, seriesID string) (*int, error) {
seasons, err := s.episodeRepo.ListSeasons(ctx, seriesID)
if err != nil {
return nil, fmt.Errorf("listing seasons: %w", err)
}
if len(seasons) == 0 {
return nil, nil
}
max := seasons[0].SeasonNumber
for _, season := range seasons[1:] {
if season.SeasonNumber > max {
max = season.SeasonNumber
}
}
return &max, nil
}
// maxSeasonNumber bounds client-supplied season numbers (Specials are season
// 0). Values are persisted as int32, so an unchecked int could silently wrap
// to an unrelated — possibly negative — season.
const maxSeasonNumber = 9999
// validSeasonNumbers reports whether every season number is within
// [0, maxSeasonNumber].
func validSeasonNumbers(nums []int) bool {
for _, n := range nums {
if n < 0 || n > maxSeasonNumber {
return false
}
}
return true
}
// normalizeSeasons keeps explicit season numbers only for specific-seasons mode;
// other modes derive their scope from the mode itself.
func normalizeSeasons(mode string, seasons []int) []int {
if mode != SubModeSpecificSeasons {
return nil
}
return seasons
}
+94
View File
@@ -0,0 +1,94 @@
package downloads
import (
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestValidSubMode(t *testing.T) {
for _, m := range []string{SubModeAll, SubModeFuture, SubModeLatestSeason, SubModeSpecificSeasons} {
if !ValidSubMode(m) {
t.Errorf("ValidSubMode(%q) = false, want true", m)
}
}
if ValidSubMode("bogus") {
t.Errorf("ValidSubMode(bogus) = true, want false")
}
}
func TestSubscriptionCoversSeason(t *testing.T) {
three := 3
cases := []struct {
name string
sub Subscription
season int
want bool
}{
{"all covers any", Subscription{Mode: SubModeAll}, 1, true},
{"future covers any season", Subscription{Mode: SubModeFuture}, 7, true},
{"latest covers target", Subscription{Mode: SubModeLatestSeason, TargetSeason: &three}, 3, true},
{"latest covers newer season", Subscription{Mode: SubModeLatestSeason, TargetSeason: &three}, 4, true},
{"latest excludes older season", Subscription{Mode: SubModeLatestSeason, TargetSeason: &three}, 2, false},
{"latest without target excludes", Subscription{Mode: SubModeLatestSeason}, 3, false},
{"specific includes listed", Subscription{Mode: SubModeSpecificSeasons, SeasonNumbers: []int{1, 3}}, 3, true},
{"specific excludes unlisted", Subscription{Mode: SubModeSpecificSeasons, SeasonNumbers: []int{1, 3}}, 2, false},
{"unknown mode excludes", Subscription{Mode: "bogus"}, 1, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := tc.sub.CoversSeason(tc.season); got != tc.want {
t.Fatalf("CoversSeason(%d) = %v, want %v", tc.season, got, tc.want)
}
})
}
}
func TestSubscriptionCoversEpisodeFutureCutoff(t *testing.T) {
subscribedAt := time.Date(2026, 6, 19, 12, 0, 0, 0, time.UTC)
before := time.Date(2026, 6, 18, 0, 0, 0, 0, time.UTC)
after := time.Date(2026, 6, 20, 0, 0, 0, 0, time.UTC)
future := &Subscription{Mode: SubModeFuture, CreatedAt: subscribedAt}
if future.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: &before}) {
t.Errorf("future-only must skip an episode that aired before the subscription")
}
if !future.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: &after}) {
t.Errorf("future-only must cover an episode that aired after the subscription")
}
// Same-day boundary: air_date is date-only (midnight), so an episode
// airing the day the user subscribed must still count as future.
sameDay := time.Date(2026, 6, 19, 0, 0, 0, 0, time.UTC)
if !future.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: &sameDay}) {
t.Errorf("future-only must cover an episode airing the same day as the subscription")
}
// No air date: fall back to the episode's ingest time.
if future.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: nil, CreatedAt: subscribedAt.Add(-time.Hour)}) {
t.Errorf("future-only must skip a no-air-date episode ingested before the subscription")
}
if !future.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: nil, CreatedAt: subscribedAt.Add(time.Hour)}) {
t.Errorf("future-only must cover a no-air-date episode ingested after the subscription")
}
// 'all' has no air-date cutoff: it covers in-season episodes regardless.
all := &Subscription{Mode: SubModeAll, CreatedAt: subscribedAt}
if !all.coversEpisode(&models.Episode{SeasonNumber: 1, AirDate: &before}) {
t.Errorf("all-mode must cover episodes regardless of air date")
}
}
func TestSubscriptionAdmits(t *testing.T) {
unlimited := &Subscription{MaxStorageBytes: 0}
if !unlimited.Admits(1<<40, 1<<30) {
t.Errorf("MaxStorageBytes<=0 must admit anything (unlimited)")
}
capped := &Subscription{MaxStorageBytes: 100}
if !capped.Admits(60, 40) { // 100 <= 100
t.Errorf("must admit when used+add equals the cap")
}
if capped.Admits(61, 40) { // 101 > 100
t.Errorf("must reject when used+add exceeds the cap")
}
}
+84 -47
View File
@@ -19,7 +19,7 @@ import (
)
const (
maxDownloadBytes = 10 * 1024 * 1024 // 10 MB
maxDownloadBytes = 25 * 1024 * 1024 // allow oversized provider originals; cached variants are dimension-capped
downloadTimeout = 30 * time.Second
)
@@ -29,6 +29,10 @@ type ObjectPutter interface {
Bucket() string
}
type objectExister interface {
ObjectExists(ctx context.Context, bucket, key string) (bool, error)
}
// ImageURLResolver resolves plugin:// paths to HTTP URLs.
type ImageURLResolver interface {
ResolveImageURL(ctx context.Context, path string, variant string) string
@@ -52,9 +56,11 @@ type CacheRequest struct {
// CacheResult is returned by Cache on success.
type CacheResult struct {
BasePath string // S3 key prefix, e.g. "tmdb/movies/550/poster"
Thumbhash string // base64-encoded
Ext string // file extension including dot (e.g. ".jpg", ".png")
BasePath string // S3 key prefix, e.g. "tmdb/movies/550/poster"
Thumbhash string // base64-encoded
Ext string // file extension including dot (e.g. ".jpg", ".png")
UploadedVariants int
ExistingVariants int
}
// Cacher downloads and stores image variants to S3.
@@ -92,9 +98,11 @@ func (c *Cacher) CacheImage(ctx context.Context, req metadata.CacheImageRequest)
return nil, err
}
return &metadata.CacheImageResult{
BasePath: result.BasePath,
Thumbhash: result.Thumbhash,
Ext: result.Ext,
BasePath: result.BasePath,
Thumbhash: result.Thumbhash,
Ext: result.Ext,
UploadedVariants: result.UploadedVariants,
ExistingVariants: result.ExistingVariants,
}, nil
}
@@ -160,25 +168,18 @@ func (c *Cacher) CacheBytes(ctx context.Context, data []byte, req CacheRequest)
}
basePath := buildBasePath(req.ProviderID, req.ContentType, req.ContentID, req.ImageType, req.Language, req.SeasonNumber, req.EpisodeNumber)
bucket := c.s3.Bucket()
var wg sync.WaitGroup
uploadErrs := make([]error, len(result.Variants))
for i, v := range result.Variants {
wg.Add(1)
go func(idx int, variant imageutil.Variant) {
defer wg.Done()
key := basePath + "/" + variant.Key + result.Ext
if err := putObjectWithRetry(ctx, c.s3, bucket, key, variant.Data); err != nil {
uploadErrs[idx] = fmt.Errorf("imagecache: upload %s: %w", key, err)
}
}(i, v)
uploadStats, err := c.uploadVariants(ctx, bucket, basePath, result)
if err != nil {
return nil, err
}
wg.Wait()
for _, err := range uploadErrs {
if err != nil {
return nil, err
}
}
return &CacheResult{BasePath: basePath, Thumbhash: thumbhash, Ext: result.Ext}, nil
return &CacheResult{
BasePath: basePath,
Thumbhash: thumbhash,
Ext: result.Ext,
UploadedVariants: uploadStats.uploaded,
ExistingVariants: uploadStats.existing,
}, nil
}
// Cache downloads the image at req.SourceURL, generates variants, computes a
@@ -232,31 +233,17 @@ func (c *Cacher) Cache(ctx context.Context, req CacheRequest) (*CacheResult, err
basePath := buildBasePath(req.ProviderID, req.ContentType, req.ContentID, req.ImageType, req.Language, req.SeasonNumber, req.EpisodeNumber)
bucket := c.s3.Bucket()
// Upload all variants concurrently.
var wg sync.WaitGroup
uploadErrs := make([]error, len(result.Variants))
for i, v := range result.Variants {
wg.Add(1)
go func(idx int, variant imageutil.Variant) {
defer wg.Done()
key := basePath + "/" + variant.Key + result.Ext
if err := putObjectWithRetry(ctx, c.s3, bucket, key, variant.Data); err != nil {
uploadErrs[idx] = fmt.Errorf("imagecache: upload %s: %w", key, err)
}
}(i, v)
}
wg.Wait()
for _, err := range uploadErrs {
if err != nil {
return nil, err
}
uploadStats, err := c.uploadVariants(ctx, bucket, basePath, result)
if err != nil {
return nil, err
}
return &CacheResult{
BasePath: basePath,
Thumbhash: thumbhash,
Ext: result.Ext,
BasePath: basePath,
Thumbhash: thumbhash,
Ext: result.Ext,
UploadedVariants: uploadStats.uploaded,
ExistingVariants: uploadStats.existing,
}, nil
}
@@ -278,6 +265,56 @@ func variantWidths(t metadata.ImageType) []int {
}
}
type uploadVariantStats struct {
uploaded int
existing int
}
func (c *Cacher) uploadVariants(ctx context.Context, bucket, basePath string, result *imageutil.VariantResult) (uploadVariantStats, error) {
var wg sync.WaitGroup
uploadErrs := make([]error, len(result.Variants))
stats := make([]uploadVariantStats, len(result.Variants))
for i, v := range result.Variants {
wg.Add(1)
go func(idx int, variant imageutil.Variant) {
defer wg.Done()
key := basePath + "/" + variant.Key + result.Ext
if exists, err := objectExists(ctx, c.s3, bucket, key); err != nil {
uploadErrs[idx] = fmt.Errorf("imagecache: check existing %s: %w", key, err)
return
} else if exists {
stats[idx].existing = 1
return
}
if err := putObjectWithRetry(ctx, c.s3, bucket, key, variant.Data); err != nil {
uploadErrs[idx] = fmt.Errorf("imagecache: upload %s: %w", key, err)
return
}
stats[idx].uploaded = 1
}(i, v)
}
wg.Wait()
var total uploadVariantStats
for _, err := range uploadErrs {
if err != nil {
return total, err
}
}
for _, s := range stats {
total.uploaded += s.uploaded
total.existing += s.existing
}
return total, nil
}
func objectExists(ctx context.Context, putter ObjectPutter, bucket, key string) (bool, error) {
exister, ok := putter.(objectExister)
if !ok {
return false, nil
}
return exister.ObjectExists(ctx, bucket, key)
}
func putObjectWithRetry(ctx context.Context, putter ObjectPutter, bucket, key string, data []byte) error {
const maxAttempts = 3
var lastErr error
+196 -1
View File
@@ -8,11 +8,14 @@ import (
"image"
"image/color"
"image/jpeg"
"image/png"
"net/http"
"net/http/httptest"
"sync"
"testing"
"github.com/h2non/bimg"
"github.com/Silo-Server/silo-server/internal/metadata"
)
@@ -32,6 +35,21 @@ func makeTestJPEG(t *testing.T) []byte {
return buf.Bytes()
}
func makeTestPNG(t *testing.T, width, height int) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := range height {
for x := range width {
img.SetRGBA(x, y, color.RGBA{R: uint8(x % 255), G: uint8(y % 255), B: 180, A: 255})
}
}
var buf bytes.Buffer
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("makeTestPNG: encode: %v", err)
}
return buf.Bytes()
}
// mockS3 records all PutObject calls for test assertions.
type mockS3 struct {
mu sync.Mutex
@@ -39,12 +57,16 @@ type mockS3 struct {
bucket string
putErr error // if non-nil, returned for every PutObject call
failuresBeforeSuccess int
existing map[string]bool
existsErr error
existsCalls []string
}
type putCall struct {
bucket string
key string
size int
data []byte
}
func (m *mockS3) PutObject(_ context.Context, bucket, key string, data []byte) error {
@@ -57,12 +79,23 @@ func (m *mockS3) PutObject(_ context.Context, bucket, key string, data []byte) e
if m.putErr != nil {
return m.putErr
}
m.calls = append(m.calls, putCall{bucket: bucket, key: key, size: len(data)})
copied := append([]byte(nil), data...)
m.calls = append(m.calls, putCall{bucket: bucket, key: key, size: len(data), data: copied})
return nil
}
func (m *mockS3) Bucket() string { return m.bucket }
func (m *mockS3) ObjectExists(_ context.Context, _ string, key string) (bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
m.existsCalls = append(m.existsCalls, key)
if m.existsErr != nil {
return false, m.existsErr
}
return m.existing[key], nil
}
func (m *mockS3) keys() []string {
m.mu.Lock()
defer m.mu.Unlock()
@@ -73,6 +106,25 @@ func (m *mockS3) keys() []string {
return keys
}
func (m *mockS3) objectData(key string) []byte {
m.mu.Lock()
defer m.mu.Unlock()
for _, c := range m.calls {
if c.key == key {
return c.data
}
}
return nil
}
func (m *mockS3) checkedKeys() []string {
m.mu.Lock()
defer m.mu.Unlock()
keys := make([]string, len(m.existsCalls))
copy(keys, m.existsCalls)
return keys
}
func containsKey(keys []string, suffix string) bool {
for _, k := range keys {
if len(k) >= len(suffix) && k[len(k)-len(suffix):] == suffix {
@@ -150,6 +202,79 @@ func TestCache_Poster(t *testing.T) {
}
}
func TestCacheSkipsUploadingVariantsThatAlreadyExist(t *testing.T) {
jpeg := makeTestJPEG(t)
srv := startImageServer(t, jpeg, http.StatusOK)
wantBase := "tmdb/movies/550/poster"
s3 := &mockS3{
bucket: "media",
existing: map[string]bool{
wantBase + "/original.webp": true,
wantBase + "/w500.webp": true,
wantBase + "/w300.webp": true,
},
}
c := newWithHTTPClient(s3, srv.Client())
result, err := c.Cache(context.Background(), CacheRequest{
SourceURL: srv.URL + "/poster.jpg",
ProviderID: "tmdb",
ContentType: "movies",
ContentID: "550",
ImageType: metadata.ImagePoster,
})
if err != nil {
t.Fatalf("Cache poster with existing variants: %v", err)
}
if result.BasePath != wantBase {
t.Fatalf("BasePath = %q, want %q", result.BasePath, wantBase)
}
if got := s3.keys(); len(got) != 0 {
t.Fatalf("uploaded keys = %v, want none when variants already exist", got)
}
if result.UploadedVariants != 0 || result.ExistingVariants != 3 {
t.Fatalf("upload stats = uploaded %d existing %d, want uploaded 0 existing 3", result.UploadedVariants, result.ExistingVariants)
}
for _, key := range []string{wantBase + "/original.webp", wantBase + "/w500.webp", wantBase + "/w300.webp"} {
if !hasKey(s3.checkedKeys(), key) {
t.Fatalf("ObjectExists was not checked for %q; checked %v", key, s3.checkedKeys())
}
}
}
func TestCacheUploadsOnlyMissingVariants(t *testing.T) {
jpeg := makeTestJPEG(t)
srv := startImageServer(t, jpeg, http.StatusOK)
wantBase := "tmdb/movies/550/poster"
s3 := &mockS3{
bucket: "media",
existing: map[string]bool{
wantBase + "/original.webp": true,
wantBase + "/w500.webp": true,
},
}
c := newWithHTTPClient(s3, srv.Client())
result, err := c.Cache(context.Background(), CacheRequest{
SourceURL: srv.URL + "/poster.jpg",
ProviderID: "tmdb",
ContentType: "movies",
ContentID: "550",
ImageType: metadata.ImagePoster,
})
if err != nil {
t.Fatalf("Cache poster with partial existing variants: %v", err)
}
if got := s3.keys(); len(got) != 1 || got[0] != wantBase+"/w300.webp" {
t.Fatalf("uploaded keys = %v, want only missing w300 variant", got)
}
if result.UploadedVariants != 1 || result.ExistingVariants != 2 {
t.Fatalf("upload stats = uploaded %d existing %d, want uploaded 1 existing 2", result.UploadedVariants, result.ExistingVariants)
}
}
func TestCache_Backdrop(t *testing.T) {
jpeg := makeTestJPEG(t)
srv := startImageServer(t, jpeg, http.StatusOK)
@@ -231,6 +356,76 @@ func TestCache_Logo(t *testing.T) {
}
}
func TestCache_ConvertsSVGLogo(t *testing.T) {
svg := []byte(`<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="400" viewBox="0 0 1200 400"><rect width="1200" height="400" fill="#111"/><text x="80" y="255" fill="#fff" font-family="Arial" font-size="180">SILO</text></svg>`)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/svg+xml")
_, _ = w.Write(svg)
}))
t.Cleanup(srv.Close)
s3 := &mockS3{bucket: "media"}
c := newWithHTTPClient(s3, srv.Client())
result, err := c.Cache(context.Background(), CacheRequest{
SourceURL: srv.URL + "/logo.svg",
ProviderID: "tmdb",
ContentType: "series",
ContentID: "1396",
ImageType: metadata.ImageLogo,
})
if err != nil {
t.Fatalf("Cache SVG logo: %v", err)
}
if result.Thumbhash == "" {
t.Fatal("Thumbhash is empty")
}
wantBase := "tmdb/series/1396/logo"
for _, variant := range []string{"original", "w500"} {
want := wantBase + "/" + variant + ".webp"
if !hasKey(s3.keys(), want) {
t.Errorf("missing S3 key %q in %v", want, s3.keys())
}
}
}
func TestCache_CapsLargeOriginalVariant(t *testing.T) {
pngData := makeTestPNG(t, 2600, 900)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/png")
_, _ = w.Write(pngData)
}))
t.Cleanup(srv.Close)
s3 := &mockS3{bucket: "media"}
c := newWithHTTPClient(s3, srv.Client())
_, err := c.Cache(context.Background(), CacheRequest{
SourceURL: srv.URL + "/logo.png",
ProviderID: "tmdb",
ContentType: "series",
ContentID: "1396",
ImageType: metadata.ImageLogo,
})
if err != nil {
t.Fatalf("Cache large logo: %v", err)
}
original := s3.objectData("tmdb/series/1396/logo/original.webp")
if len(original) == 0 {
t.Fatal("missing original.webp upload")
}
size, err := bimg.NewImage(original).Size()
if err != nil {
t.Fatalf("reading original.webp size: %v", err)
}
if size.Width > 1920 {
t.Fatalf("original.webp width = %d, want <= 1920", size.Width)
}
if len(original) >= 10*1024*1024 {
t.Fatalf("original.webp size = %d bytes, want < 10 MiB", len(original))
}
}
func TestCache_LocalizedPosterUsesLanguageScopedPath(t *testing.T) {
jpeg := makeTestJPEG(t)
srv := startImageServer(t, jpeg, http.StatusOK)
+53 -6
View File
@@ -16,7 +16,11 @@ import (
"go.n16f.net/thumbhash"
)
const webpQuality = 90
const (
webpQuality = 90
maxCachedOriginalDimension = 1920
thumbhashSourceDimension = 100
)
// Variant holds a named image variant (e.g. "original", "w500").
type Variant struct {
@@ -39,18 +43,22 @@ func GenerateVariants(data []byte, widths []int) (*VariantResult, error) {
img := bimg.NewImage(data)
// Validate input by reading size.
if _, err := img.Size(); err != nil {
size, err := img.Size()
if err != nil {
return nil, fmt.Errorf("imageutil: invalid image: %w", err)
}
variants := make([]Variant, 0, len(widths)+1)
// Original — re-encode as WebP, strip metadata, no resize.
original, err := bimg.NewImage(data).Process(bimg.Options{
// Original — re-encode as WebP, strip metadata, and cap very large provider
// artwork so cached originals do not preserve oversized source dimensions.
originalOptions := bimg.Options{
Type: bimg.WEBP,
Quality: webpQuality,
StripMetadata: true,
})
}
fitWithin(&originalOptions, size, maxCachedOriginalDimension)
original, err := bimg.NewImage(data).Process(originalOptions)
if err != nil {
return nil, fmt.Errorf("imageutil: encode original: %w", err)
}
@@ -148,7 +156,14 @@ func GenerateSquareVariants(data []byte, sizes []int) (*VariantResult, error) {
func Thumbhash(data []byte) (string, error) {
img, _, err := image.Decode(bytes.NewReader(data))
if err != nil {
return "", fmt.Errorf("imageutil: decode for thumbhash: %w", err)
normalized, normalizeErr := normalizeThumbhashSource(data)
if normalizeErr != nil {
return "", fmt.Errorf("imageutil: decode for thumbhash: %w", err)
}
img, _, err = image.Decode(bytes.NewReader(normalized))
if err != nil {
return "", fmt.Errorf("imageutil: decode normalized thumbhash source: %w", err)
}
}
scaled := scaleImage(img, 100)
@@ -156,6 +171,38 @@ func Thumbhash(data []byte) (string, error) {
return base64.StdEncoding.EncodeToString(hashBytes), nil
}
func normalizeThumbhashSource(data []byte) ([]byte, error) {
img := bimg.NewImage(data)
size, err := img.Size()
if err != nil {
return nil, fmt.Errorf("imageutil: invalid image: %w", err)
}
opts := bimg.Options{
Type: bimg.PNG,
StripMetadata: true,
}
fitWithin(&opts, size, thumbhashSourceDimension)
out, err := img.Process(opts)
if err != nil {
return nil, fmt.Errorf("imageutil: normalize thumbhash source: %w", err)
}
return out, nil
}
func fitWithin(opts *bimg.Options, size bimg.ImageSize, maxDim int) {
if opts == nil || maxDim <= 0 {
return
}
if size.Width <= maxDim && size.Height <= maxDim {
return
}
if size.Width >= size.Height {
opts.Width = maxDim
return
}
opts.Height = maxDim
}
// scaleImage scales src so its longest dimension does not exceed maxDim,
// preserving aspect ratio. Uses nearest-neighbour interpolation.
func scaleImage(src image.Image, maxDim int) *image.NRGBA {
+55 -9
View File
@@ -18,11 +18,14 @@ import (
)
type testCompatSessionManager struct {
sessions map[string]*playback.Session
audioTrackCalls []compatAudioTrackCall
progressCalls int
stopCalls []string
startCalls int
sessions map[string]*playback.Session
audioTrackCalls []compatAudioTrackCall
progressCalls int
progressUpdates []compatProgressCall
stopCalls []string
startCalls int
beginTransportCalls []string
endTransportCalls []string
}
type compatAudioTrackCall struct {
@@ -31,6 +34,12 @@ type compatAudioTrackCall struct {
method playback.PlayMethod
}
type compatProgressCall struct {
sessionID string
position float64
isPaused bool
}
func (m *testCompatSessionManager) StartSession(userID int, profileID string, fileID int, method playback.PlayMethod, transcodeAudio bool) (*playback.Session, error) {
m.startCalls++
session := &playback.Session{
@@ -49,8 +58,38 @@ func (m *testCompatSessionManager) StartSession(userID int, profileID string, fi
return session, nil
}
func (m *testCompatSessionManager) UpdateProgress(string, float64, bool) error {
func (m *testCompatSessionManager) UpdateProgress(sessionID string, position float64, isPaused bool) error {
m.progressCalls++
m.progressUpdates = append(m.progressUpdates, compatProgressCall{
sessionID: sessionID,
position: position,
isPaused: isPaused,
})
if m.sessions != nil {
if _, ok := m.sessions[sessionID]; !ok {
return playback.ErrSessionNotFound
}
}
return nil
}
func (m *testCompatSessionManager) BeginTransport(sessionID string) error {
m.beginTransportCalls = append(m.beginTransportCalls, sessionID)
if m.sessions != nil {
if _, ok := m.sessions[sessionID]; !ok {
return playback.ErrSessionNotFound
}
}
return nil
}
func (m *testCompatSessionManager) EndTransport(sessionID string) error {
m.endTransportCalls = append(m.endTransportCalls, sessionID)
if m.sessions != nil {
if _, ok := m.sessions[sessionID]; !ok {
return playback.ErrSessionNotFound
}
}
return nil
}
@@ -200,7 +239,7 @@ func TestHandlePlaybackReport_UpdatesSelectedAudioStreamAndUpstreamTrack(t *test
handler := &PlaybackHandler{
playbackStore: playbackStore,
sessionMgr: sessionMgr,
transcodes: make(map[string]*playback.TranscodeSession),
tm: playback.NewTranscodeManager(),
}
req := httptest.NewRequest(http.MethodPost, "/Sessions/Playing/Progress", strings.NewReader(`{"PlaySessionId":"play-1","MediaSourceId":"`+source.ID+`","AudioStreamIndex":2,"PositionTicks":30000000}`))
@@ -254,7 +293,7 @@ func TestEnsureTranscodeSession_UsesSelectedAudioTrack(t *testing.T) {
fileResolver: testCompatFileResolver{file: &models.MediaFile{ID: version.FileID, FilePath: filePath}},
TranscodeDir: t.TempDir(),
FFmpegPath: writeCompatTestFFmpeg(t),
transcodes: make(map[string]*playback.TranscodeSession),
tm: playback.NewTranscodeManager(),
}
transcodeSession, err := handler.ensureTranscodeSession(context.Background(), "play-1", "upstream-1", source)
@@ -288,9 +327,16 @@ func TestStartRemoteTranscode_IncludesSelectedAudioTrack(t *testing.T) {
}))
defer server.Close()
handler := &PlaybackHandler{JWTSecret: "secret"}
playbackStore := NewPlaybackSessionStore(time.Hour, nil)
playbackStore.Put(PlaybackSession{ID: "play-1", UpstreamSessionID: "upstream-1"})
handler := &PlaybackHandler{
JWTSecret: "secret",
playbackStore: playbackStore,
tm: playback.NewTranscodeManager(),
}
if err := handler.startRemoteTranscode(
context.Background(),
"play-1",
"upstream-1",
source,
&models.MediaFile{ID: version.FileID, FilePath: filePath},
@@ -160,6 +160,11 @@ func (f *scopeFakeBrowseSource) BrowsePage(_ context.Context, filters catalog.Br
return &catalog.BrowseResult{Items: []*models.MediaItem{}, Total: 0, HasMore: false}, nil
}
func (f *scopeFakeBrowseSource) BrowseRecentlyAddedAcrossLibraries(_ context.Context, base catalog.BrowseFilters, _ []int) (*catalog.BrowseResult, error) {
f.lastFilters = base
return &catalog.BrowseResult{Items: []*models.MediaItem{}, Total: 0, HasMore: false}, nil
}
func (f *scopeFakeBrowseSource) ListGenres(_ context.Context, filters catalog.BrowseFilters) ([]string, error) {
f.lastFilters = filters
return []string{}, nil
+1 -1
View File
@@ -219,7 +219,7 @@ func resolveCompatToken(ctx context.Context, sessions *SessionStore, keyAuth *Ad
// PlaybackSessionAuth creates middleware that falls back to playback session
// authentication for media stream endpoints where external players (e.g. libmpv)
// don't forward auth headers or query parameters.
func PlaybackSessionAuth(sessions *SessionStore, playbackStore *PlaybackSessionStore, keyAuth *AdminAPIKeyAuthenticator) func(next http.Handler) http.Handler {
func PlaybackSessionAuth(sessions *SessionStore, playbackStore CompatPlaybackStore, keyAuth *AdminAPIKeyAuthenticator) func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Try standard token auth first — a compat session token or an sa_
+1
View File
@@ -43,6 +43,7 @@ type episodeRepoForBatchLoader interface {
HasFilesByIDs(ctx context.Context, contentIDs []string) (map[string]bool, error)
ListBySeason(ctx context.Context, seriesID string, seasonNum int) ([]*models.Episode, error)
ListBySeries(ctx context.Context, seriesID string) ([]*models.Episode, error)
ListAdjacentInSeries(ctx context.Context, seriesID string, seasonNumber, episodeNumber int) ([]*models.Episode, error)
}
func (h *ItemsHandler) fetchCompatItemsByContentIDs(ctx context.Context, session *Session, contentIDs []string, libraryID *int) (map[string]upstreamListItem, error) {

Some files were not shown because too many files have changed in this diff Show More