af974edfe9400a25e7ea5b25a9a403cdeb91dad8
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
54507d236a |
fix(watchstate): make series mark-watched atomic and unload-proof (#645)
Marking a large series as watched only marked some episodes when the user
navigated away or closed the tab. POST /watched/{id} expanded a series to
every episode and then did four sequential DB round-trips per episode — a
duration lookup, a progress upsert, a stable-identity resolution (itself an
episode lookup plus a series provider-ID lookup), and a history insert — all
on the request context with no transaction. A 200-episode series was ~800
sequential queries, each committing on its own, so a disconnect mid-loop left
everything already committed in place.
Server:
- episodeTargets now resolves durations through the existing batched
listEpisodeFiles helper instead of one file query per episode. The shared
mediaFileDurationSeconds keeps the batched and single-item paths in step.
- ResolveHistoryIdentities resolves a whole series in one episode query plus
one provider-ID query per distinct series, behind optional-capability
interfaces with per-ID fallbacks.
- New userstore.WatchedBatchWriter capability, implemented transactionally in
both Postgres and SQLite, with a per-target fallback for stores that lack
it. recordMarkWatched and jellycompat's recordMarkWatchedBatch share it.
A cancelled request now rolls back to "nothing marked" rather than stranding
a half-watched series, which is the correct all-or-nothing semantic and needs
no detached context.
Client: the watched mutation sends keepalive so the request survives
navigation and tab close, plus an optimistic played flip with rollback,
following the existing favorites.ts pattern.
Routing jellycompat through the shared path would have clobbered known
durations with 0, since that caller supplies none; the batch upsert only ever
advances a duration, matching what MarkProgressBatch did before.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
4e03f4b807 |
fix(playback): last-write-wins progress and DV P7 RPU strip on remux (#334)
* fix(playback): last-write-wins progress and DV P7 RPU strip on remux Progress: UpdateProgress (the live playback-session path) clamped position_seconds to GREATEST(new, old), so a deliberate backward seek could never persist — "rewind and stop" resumed at the stale later position on every client. Position is now last-write-wins, matching the /sync/progress path that was always unconditional. The completed latch and rewatch re-entry semantics are unchanged. Remux: profile 7 Dolby Vision remuxes drop the enhancement-layer track (-map 0:v:0 keeps only the base layer) but previously left the dangling dual-layer RPUs on the BL — broken metadata that a DV-honoring display can mis-render. Remuxes of P7 files now strip DV RPUs via the dovi_rpu bitstream filter, yielding a clean HDR10 stream (the same fallback presentation the Apple client's P7 HDR10 toggle produces). Profile 8 RPUs are kept: the BL is self-contained and DV clients render it. Adds MediaFile.PrimaryDVProfile() and threads the profile through ServeRemux callers; the proxy path (no track metadata in claims) keeps prior behavior. True P7->8.1 DV conversion needs dovi_tool alongside FFmpeg (the dovi_rpu bsf only strips/recompresses); the remux plumbing now carries the DV profile so that can slot in later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userdb): apply last-write-wins progress to the SQLite backend too Review follow-up (P2): the LWW change only covered pgstore; the SQLite userdb UpdateProgress kept the MAX clamp, so rewind-and-stop still resumed at the stale later position for sqlite-backed installs. The conflict clause now matches Postgres (position last-write-wins, completed latch and rewatch re-entry unchanged), with a backward-seek regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden LWW progress and DV RPU strip from review - Probe ffmpeg for the dovi_rpu bitstream filter once per process and fall back to a no-strip remux (the pre-existing behavior) when it is missing: on pre-7.1 ffmpeg the unknown filter aborted the process, turning every Dolby Vision profile 7 remux into a hard playback failure. - Skip zero-position heartbeats in persistProgress, mirroring the stop path and the jellycompat report path. Under last-write-wins an early zero heartbeat (e.g. before the client seeks to its resume point) would wipe the stored resume position; GREATEST previously masked this. - Carry the DV profile in stream token claims (dvp, omitempty) so standalone proxy nodes strip profile 7 RPUs the same way integrated mode does. Old tokens decode as 0 and keep prior behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
9cae868a27 |
feat(downloads): offline sync for mobile — downloads v2 (#258)
* feat(downloads): offline sync for mobile (downloads v2) Replace internal/download with a unified internal/downloads package and add fully-offline download + watch-sync support for mobile clients, across five independently-shippable phases: - Phase 0: reshape the downloads table and the /downloads contract to be device- and format-aware; add GET /downloads/capability; extend DownloadConfig (default-off keys); update the web download hooks/components in lockstep. This is the one approved pre-lock exception to the additive-only /api/v1 rule (the web app is the only consumer and is updated together). - Phase 1: managed device-library entries (create/list/PATCH/delete/serve), keyed on the X-Silo-Device-Id header. - Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that strip every presigned URL (inline thumbhashes + authenticated proxies). - Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable, leased artifact queue with startup recovery, hosted on the task manager; playback.PrepareFile emits one +faststart MP4. Adds the admin transcode toggle and per-artifact LRU cleanup. - Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a server-assigned synced_seq cursor on watch_progress; an optional clamped updated_at on POST /sync/progress and an opaque ?since= cursor on GET /progress (additive; existing callers unaffected). Security & reliability invariants, each with an acceptance test: 1. Server-owned sync ordering: ?since= delta delivery is driven only by the server-assigned synced_seq; the client clock is bounded (event_at, clamped to now+skew) and used only for last-write-wins on the caller's own profile. 2. Full profile+device authorization on every managed endpoint, with a per-profile content/library access re-check before serving any bytes/assets. 3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no crash strands a download in preparing and concurrent workers never double-encode. Migrations are timestamped Goose files: reshape downloads (device/format); download_artifacts (durable queue); watch_progress event_at/synced_seq. DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI; the invariant-1 progress test also runs against the real SQLite backend locally. Client repos (silo-android, silo-apple) consume the reshaped /downloads/* contract and the updated_at/?since= progress fields and require coordinated follow-up. Implements the maintainer-approved v1 capability proposal for offline sync (downloads v2). AI-use disclosure: implemented by Claude (Claude Code) from the approved design doc under docs/superpowers/specs, with human review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(downloads): series & season downloads + client-pull monitoring Build season downloads and a "monitor a series" capability on top of the downloads v2 (offline sync for mobile) work. Season downloads: - POST /downloads accepts season_number (with series:true) to download one season. CreateSeries/CreateSeason share one body via a listEpisodes closure and register managed entries under a shared batch_id (original-only). Episode files are resolved in a single batched query. Series monitoring (auto-download), client-driven: - New device-scoped download_subscriptions table with a Sonarr-style mode (all | future | latest_season | specific_seasons), a client-enforced delete_watched flag, and a max_storage_bytes cap. The server never deletes on-device files; retention and the hard cap are the client's, the server only soft-gates registration. - The client calls POST /downloads/subscriptions/sync on open / background refresh; the server registers the in-scope, not-yet-downloaded episodes (idempotent via the managed-entry unique index) and the device pulls them on its own schedule. No background worker and no dependency on the notifications subsystem. latest_season follows new seasons (>= subscribe-time season); future excludes the back catalog via air date. - Subscription CRUD + sync are profile+device authorized (device id from the X-Silo-Device-Id header only) with a per-request content-access re-check. The capability endpoint advertises season_download / series_monitoring / monitoring_modes. Also lands the downloads-v2 work already present in the tree: durable artifact (remux/transcode) preparation and offline watch-progress reconciliation, plus the design-spec updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync * test(downloads): fix deterministic ID collision in reconcile test Artifact IDs are time-sortable, so two artifacts created in the same moment share their first 8 chars; combined with a captured timestamp the two preparing-download IDs collided on downloads_pkey. Use the full artifact ID, which is unique per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): support sqlite userdb backend for managed downloads With the sqlite userdb backend, profiles live only in per-user SQLite stores and public.user_profiles stays empty, so user_devices' profile FK made every managed create/subscription/offline-sync request fail with an FK violation. Drop the FK (shared Postgres tables must not FK profile tables — same rule as notifications) and replace the lost cascade with an app-level purge on profile deletion, wired through ProfileHandler for both backends. DB-backed regression tests cover the no-Postgres-profile-row path and the purge cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): dispatch encode kick asynchronously triggerDrain invoked the kick inline, and the kick (taskmanager RunTask) executes the encode task on the caller's goroutine — so a POST /api/v1/downloads with a bitrate quality blocked the HTTP request on the entire queue drain, ffmpeg encodes included, delaying the 202 by minutes on an idle queue. Dispatch the kick on a goroutine; the task manager already serializes concurrent runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): enforce per-user quota on the encode pipeline Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared downloads: artifact-backed rows are created in 'preparing' (never 'queued'/'downloading'), which CountActiveByUser didn't count, and createArtifactDownload enqueued the encode job before limiter.Check, so even a 429-rejected request left a job the worker would transcode. Count 'preparing' as active and check the limiter before Ensure; managed replacements stay quota-exempt since they don't add a row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): protect ephemeral artifact links from LRU eviction HasActiveLink only counted managed (device_id IS NOT NULL) rows, so under a byte budget Cleanup could delete an artifact still referenced by a ready-but-unfetched ephemeral web download — permanently 404ing a row the API kept listing as ready (the artifact row is gone, so recovery can't re-queue it). Any non-terminal link now protects the artifact; only artifacts whose links are all cancelled/failed/revoked are evictable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): batch manifests skip bad entries instead of failing whole batch One deleted or access-filtered episode made GET /downloads/batches/{id}/manifests 404 for the entire season, so a client could no longer fetch manifests for the still-valid entries. Report unbuildable entries in a skipped[] array (revoked | not_found | error) alongside the delivered manifests, mirroring the create path's skip idiom. Also cut the batch cost: the shared series detail is resolved once per batch instead of once per episode, and buildSubtitles reuses the already-loaded media file instead of re-querying it per manifest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): wrap DO block in StatementBegin/End markers Under NO TRANSACTION goose splits statements on semicolons, so the dollar-quoted DO block failed every fresh install with 'unterminated dollar-quoted string' (SQLSTATE 42601). Already-applied databases are unaffected. Same fix is being applied to main; identical content merges cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): allow season 0 (Specials) in season downloads season_number was a plain int dispatched with '> 0', so requesting the Specials season was indistinguishable from omitting the field and silently broadened to a full-series download. Dispatch on pointer presence, treat 0 as the Specials season, and reject negatives with 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): capability quality_presets is never JSON null PresetsFor returned a nil slice when downloads are disabled or the user lacks the permission, and Capability's []string{} initialization was immediately overwritten by it — so GET /downloads/capability serialized "quality_presets": null where the contract documents an array. Normalize at the source so every caller inherits the guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): subscription sync correctness + batched registration Three subscription fixes: - A paused subscription no longer syncs: PATCHing scope (or pausing and changing scope in one request) registered episodes for a monitor the user had just stopped, inconsistently with SyncSubscriptions' guard. - SubModeFuture compares calendar days (UTC): air_date is date-only, so the strict instant comparison permanently excluded episodes airing the same day the user subscribed; episodes with no air date now fall back to their ingest time instead of never registering. - Registration is one batched fetch (GetManagedEntriesByKeys) plus one batched INSERT ... ON CONFLICT DO NOTHING RETURNING (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode — a 300-episode series cost ~600 sequential round trips per request and every no-op sync re-walked the full set. RETURNING yields exactly the new rows, so the sync response's 'registered' count now honestly reports 0 in the steady state instead of the full in-scope count on every app open. The now-unused InsertManagedEntryIfAbsent is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userstore): stamp triggers own the event_at LWW key MarkProgressBatch (jellycompat series mark-played) advanced updated_at but never event_at, and both stamp triggers only defaulted event_at when NULL — so a queued offline event with a client time between the row's old event_at and the mark could win SetProgressIfNewer and resurrect a stale resume position that then re-synced to every device. Make the triggers authoritative instead of adding a tenth hand-written SET clause: whenever an UPDATE changes updated_at without explicitly changing event_at, the trigger advances the LWW key; writes that do set event_at (offline sync's clamped client event time) keep their value. Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb migration that drops and reinstalls the trigger bodies (CREATE TRIGGER IF NOT EXISTS never replaces). Conformance tests cover both batch paths, the preserved-client-time invariant, and the v11→v12 upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps Migrations: fold the 20260621 corrective migration back into the base Downloads V2 migrations (its columns/constraints already exist there) and fix the reshape Down, which re-added the narrow status CHECK without collapsing managed-lifecycle rows first — rollback aborted on any DB with preparing/ready/revoked rows; validated against a live row. Branch databases that applied the corrective migration need its version row removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459. Code: drop the dead 'registered' status (nothing ever wrote it; the lifecycle is preparing -> ready; 'revoked' stays reserved for the planned admin revoke flow) along with unused KindDirect and ErrInvalidFormat. Sweeps: Cleanup now runs an age-based hygiene pass independent of the byte budget — cold terminally-failed artifacts (with .part leftovers), orphaned ready artifacts no download row references, and ephemeral web rows older than their convenience-record lifetime (also unpinning their artifacts and bounding GET /downloads growth). The byte budget remains the disk quota per the limits & restrictions design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff Document the contract changes from the review fixes: batch-manifest skipped[] shape, honest subscription 'registered' semantics, season 0 = Specials, always-array quality_presets, bytes_sent actual behavior, ephemeral 7-day retention, header-pairing requirement, progress-delta deletion caveat, and the ready/failed push event schema (new §9.4). Add an Android client handoff section (§11) mirroring the Apple one, register HEAD on /downloads/{id}/file for download stacks that probe before ranged GETs, and add season_number to the web create-request type. Flag the /direct-download session-token-in-URL tradeoff; a short-lived download-scoped URL is a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: consolidate download/progress helpers, prune dead code, gate sweeps Behavior-preserving consolidation from the Downloads V2 review: - appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection, shared by the HLS builder and the single-file prepare builder (the drift pattern that already bit tone-mapping once). - userstore.ResolveProgressState: one home for the min-resume/watched threshold rule, replacing five identical copies across both store backends and the offline-sync ingest. - Download file selection ranks resolutions via access.CompareQuality (adds 4320p, agrees with playback) instead of a private switch. - writeSubtitle uses the shared subtitles.SubtitleContentType mapping. - config.DefaultTranscodeDir replaces three '/tmp/silo-transcode' literals. - Read-side quality/revision defaulting helpers removed: insertArgs plus the NOT NULL/CHECK schema already guarantee the invariant. - Dead code removed: Repository.ListByUser, SubscriptionRepository. ListActiveBySeries, and the stale auto-register-worker comments (the design is client-pull; no worker exists). - Redundant left-prefix indexes dropped from the base migrations (their unique indexes serve the same prefixes). - recover()'s disk-presence sweep and the stale-row hygiene sweep run on startup then hourly instead of every 30s tick (both are O(cache size)). - gofmt/prettier fixes for pre-existing drift in handlers/playback.go and pages/Profiles.tsx. Deferred (noted for follow-ups): quality-ladder preset table collides with the drafted download limits & restrictions design, which specifies its own ladder helper; Download-literal construction consolidation and the managed-identity value object remain open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): draft download limits & restrictions design Design input for the follow-up v1 capability proposal (quality ceiling, batch size cap, per-user quantity/bandwidth overrides). Committed with downloads v2 because the remediation work explicitly defers the quality ladder refactor and revocation wiring to this spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(progress): reject malformed updated_at; clamp negative progress inputs Review findings on #258: - A malformed (non-RFC3339) updated_at in POST /sync/progress previously parsed to the zero time, which clampEventAt treated as "now" — letting a stale offline event win LWW as a fresh server-time write. The item is now rejected with a per-item error instead. - ResolveProgressState now clamps negative position/duration before classification so no backend can persist negative progress through UpdateProgress/SetProgress. - The online-write event_at invariant test is table-driven over both SetProgress and UpdateProgress, which share the same contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests Review findings on #258: - UpdateSubscription now applies the same feature/DownloadAllowed gate as CreateSubscription and SyncSubscriptions; a PATCH could previously re-activate or widen a monitor and register managed rows after an admin disabled downloads or revoked the user. - Serving download bytes (managed and ephemeral) and /direct-download now mirror playback's per-file authorization via catalog.FileAllowedByAccess: library scope and the profile's max playback quality are re-checked at serve time, with artifact-backed rows checked against the artifact's resolution (a 720p transcode of a 4K source stays servable under a 1080p ceiling). - Offline manifests for remux/transcode entries now describe the prepared artifact (container, codecs, resolution, single selected audio track) instead of the catalog source file the client never receives. - ArtifactRepository.Requeue reports ErrNotFound when the row was concurrently swept; ArtifactManager.Ensure recreates the job in that case instead of linking downloads to a dead artifact id. - "No downloadable episodes" is a sentinel (mapped to 404 no_downloadable_episodes) rather than a bare error that surfaced as 500. - Subscription season_numbers are bounds-checked (0–9999) before the int32 narrowing in the repo could silently wrap them. - HandlePatchDownload reuses requireManaged instead of hand-rolling the same managed-identity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
QuickandGitHub
|
cac435c4b9 |
Fix watch-state unwatch sync across user data and Jellyfin mappings (#179)
* Refine playback session handling and API responses * fix(watchstate): harden completed-history visibility |
||
|
|
46540dfec3 |
fix(progress): track resume points independently of watched state (#117)
* fix(progress): track resume points independently of watched state
Re-watching a finished item never re-entered Continue Watching: completion
latched completed = TRUE one-way, pinned position_seconds to the duration,
and the resume query filtered on completed = FALSE — so a rewatch heartbeat
could never surface the item again (and releasing the latch would have
erased the watched state clients display).
Adopt the Jellyfin invariant instead of guard heuristics:
- Completion resets position_seconds to 0 (UpdateProgress, SetProgress,
SetProgressAt, SetProgressIfNewer, MarkWatched, MarkProgressBatch), so
position_seconds > 0 now means "live resume point".
- completed stays a pure one-way watched latch; rewatch heartbeats re-enter
Continue Watching through plain GREATEST/MAX while the watched flag and
PlayCount survive (matching Plex and Jellyfin master).
- ListProgress("in_progress") keys on position_seconds > 0 in both stores;
the SQLite store also gains the min-resume floor the Postgres store had.
- jellycompat reports Played=true with live PositionTicks during a rewatch
(resumePositionTicks no longer zeroes played items) — the DTO shape real
Jellyfin emits since jellyfin/jellyfin#15762.
- Web mirrors the latch (playbackProgressCache), resumes rewatches at their
stored position, and shows progress bars on rewatched episodes.
- ABS audiobook surfaces keep today's behavior: finished books report 100%
via the completed flag and Continue Listening still excludes them.
- Migrations reset legacy completed rows (position pinned to duration) to
0: a Goose migration for Postgres and a user_version-gated one-time fix
for the per-user SQLite DBs.
Replaces the guard-based approach of #109, whose restart detection
(50% fraction + 60s time gap) could never release the latch for immediate
rewatches (blocked heartbeats refreshed updated_at, re-arming the gap) and
un-watched items on position-0 heartbeats.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(progress): address review — migration gate, one-way latch, missed writers/readers
Review fixes for the position-based watch-progress model:
- The per-user SQLite data fix is now migrateToV11 in the existing
versioned runMigrations chain (schemaVersion 11). The previous
standalone PRAGMA gate compared against 1, but existing DBs already
sit at user_version 10, so the reset never ran for them — and the
gate would have rewound the version. Fresh DBs short-circuit to the
current version as before.
- `completed` is now one-way across every playback/sync writer:
SetProgress (the RecordPlaybackStop path — stopping a rewatch below
the watched threshold no longer clears the watched state),
SetProgressAt, SetProgressIfNewer (both stores), and the history
import upsert, which also stops pinning completed imports to
position = duration. Mark-unwatched still releases the latch via
ClearProgress/ClearProgressBatch.
- MarkProgressBatch regains its freshness guard: a delayed batch mark
carrying an old timestamp can no longer zero a newer rewatch resume
point (the position-reset now rides the original updated_at check).
- Catalog read paths align with the new in-progress definition
(position_seconds > 0, completed-agnostic): smart-collection
in_progress filter, progress sort ratio, episode progress CTE, and
both next-up predicates.
- jellycompat derives PlayedPercentage and PlaybackPositionTicks from
the same clamped position; a played item at rest reports 100 (as the
old model did) while a rewatch reports its live fraction.
- ABS audiobook UpsertProgress stores position 0 on finish so finished
books can't surface as phantom resume entries; re-listens still move
position forward from 0 with the latch intact.
- The web optimistic cache zeroes the resume point on completion,
mirroring the server invariant until the refetch lands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Silo Server Migration
|
c085b12fd1 | Initial Silo migration |