* fix(watchsync): respect MDBList rate limits and defer syncs on 429
MDBList caps API usage at 1,000 requests/day on the free tier, and a
large-library first sync (paginated watched/watchlist fetches plus
exports chunked at 100 items per POST) could blow through it. A 429 was
treated as a generic failure: every pending chunk was marked failed and
the next scheduled run replayed the whole sync into the same limit.
- Pace MDBList requests at ~1/s per API key and retry 429s with a short
Retry-After in place; longer waits surface a typed RateLimitedError.
- Abort the remaining sync flows on the first rate-limited flow and
persist rate_limited_until on the connection; scheduled syncs skip it
until the deferral passes and manual sync returns a proper cooldown.
- Leave rate-limited exports pending instead of marking them failed so
the next run resumes where it stopped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(watchsync): address rate-limit review feedback
- Floor RateLimitedError.RetryAfter to the default deferral when in-place
retries are exhausted, so untrustworthy short Retry-After hints can't
produce a seconds-long deferral that walks straight back into the limit.
- Stamp rate_limited_until on every connection bound to the same provider
account (the quota belongs to the API key, not the profile), and re-read
connections mid-batch in SyncDueConnections so siblings deferred after
the snapshot are skipped.
- Filter deferred connections out of the live dispatch queries
(local watch events, list events, scrobbles) so real-time exports stop
burning quota during a cooldown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(watchsync): sync watchlists with Trakt/Simkl/MDBList
Extend the watch-providers feature to sync a user's watchlist, generalizing
the existing favorites pipeline rather than duplicating it.
What changed
- Generalize the favorites sync into one ListKind-parameterized pipeline
(internal/watchsync/lists.go) driving both favorites and watchlist; the
per-favorites service methods are replaced by kind-generic ones. The shadow
table watch_provider_favorite_items becomes watch_provider_list_items with a
list_kind discriminator.
- Providers: Trakt gains watchlist sync (/sync/watchlist, distinct from
favorites); Simkl gains plan-to-watch sync; MDBList is re-mapped from
favorites to watchlist (its only list is a watchlist) — its capabilities now
report import_favorites=false / import_watchlist=true, and the migration
re-binds existing MDBList connections.
- Auto-remove watched items from the watchlist: a standalone, default-on
profile preference (user_profiles.remove_watched_from_watchlist) removes a
movie when watched and a series once every episode is watched. Implemented as
watchstate.CompletionObserver (internal/watchlist.Maintainer), wired into the
manual mark-watched, playback-stop, and jellycompat mark-played paths.
- Optional MDBList sort-order mirroring: an opt-in, capability-gated toggle
mirrors MDBList's watchlist order into Silo via user_watchlist.sort_index;
ListWatchlist orders by sort_index then added_at, so both /api/v1/watchlist
and the catalog watchlist view inherit it.
- Real-time + scheduled: local add/remove pushes to connected providers
immediately (removals gated by the opt-in removals toggle); the hourly job is
the inbound/import + retry/reconcile path.
- Web: watch-provider settings gain watchlist import/export/removals and
"mirror watchlist order" toggles plus watchlist sync stats.
Why
- The favorites and watchlist pipelines are ~90% identical; generalizing keeps
one code path (per CLAUDE.md's anti-duplication guidance) instead of cloning.
API/compat
- All new fields on ConnectionStatus/Capabilities/ConnectionUpdate/SyncRun and
the web types are additive (Silo v1 additive-only rule). No existing field is
renamed, removed, or retyped.
Risks / follow-up
- MDBList capability flip is intentional and client-visible: silo-android /
silo-apple may need to surface MDBList under the watchlist (not favorites) UI.
- MDBList existing users: their MDBList list previously mirrored Silo favorites
and now mirrors Silo watchlist; the first post-migration sync is a union
(removals default off), so nothing is destructively purged.
- Order mirroring reflects the order MDBList returns from /watchlist/items
(couldn't confirm against their docs — Cloudflare-blocked); if it ever
diverges from the UI sort, a sort param is the small follow-up.
Tests: new maintainer (auto-remove) and watchlist-order unit tests; provider +
service tests updated. go build, go test (affected pkgs), migrate-validate,
verify-local-paths, web prettier/eslint/tsc all pass.
AI-use disclosure: implemented with Claude Code (Claude Opus 4.8).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(watchsync): update list shadow table references
* fix(watchsync): address review — retry/progress + error propagation
Addresses CodeRabbit review on #227:
- maintainer: propagate transient catalog lookup errors instead of silently
treating every items.GetByID failure as "maybe an episode".
- exportList: mark every queued item not confirmed sent (not_found, failed, or
omitted) so the pending loop always advances; the next run's upsert clears the
error and re-attempts, so transient failures still retry.
- removePendingListItems + realtime removal: treat Sent and NotFound as
reconciled; leave true failures pending (no last_error, which would strand
them from the removal query) so the scheduled run retries, using in-memory
dedupe to terminate the loop.
- exportLocalListItems: send the normalized items (with computed
ProviderItemKey), not the original event slice.
- UpdateConnection: clear mirrored watchlist order before persisting the disable
and propagate failures, so a failed clear can't report "disabled" while
sort_index ordering is still active.
- web: include favorite + watchlist removal counts in the exported "sent" total.
- test: align serviceFakeRepo list-state with Postgres (clear last_error on
successful transitions); add maintainer error-propagation test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
