Files
silo-server/migrations/sql/20260626174211_watch_provider_watchlist.sql
02e62767a1 feat(watchsync): sync watchlists with Trakt/Simkl/MDBList (#227)
* feat(watchsync): sync watchlists with Trakt/Simkl/MDBList

Extend the watch-providers feature to sync a user's watchlist, generalizing
the existing favorites pipeline rather than duplicating it.

What changed
- Generalize the favorites sync into one ListKind-parameterized pipeline
  (internal/watchsync/lists.go) driving both favorites and watchlist; the
  per-favorites service methods are replaced by kind-generic ones. The shadow
  table watch_provider_favorite_items becomes watch_provider_list_items with a
  list_kind discriminator.
- Providers: Trakt gains watchlist sync (/sync/watchlist, distinct from
  favorites); Simkl gains plan-to-watch sync; MDBList is re-mapped from
  favorites to watchlist (its only list is a watchlist) — its capabilities now
  report import_favorites=false / import_watchlist=true, and the migration
  re-binds existing MDBList connections.
- Auto-remove watched items from the watchlist: a standalone, default-on
  profile preference (user_profiles.remove_watched_from_watchlist) removes a
  movie when watched and a series once every episode is watched. Implemented as
  watchstate.CompletionObserver (internal/watchlist.Maintainer), wired into the
  manual mark-watched, playback-stop, and jellycompat mark-played paths.
- Optional MDBList sort-order mirroring: an opt-in, capability-gated toggle
  mirrors MDBList's watchlist order into Silo via user_watchlist.sort_index;
  ListWatchlist orders by sort_index then added_at, so both /api/v1/watchlist
  and the catalog watchlist view inherit it.
- Real-time + scheduled: local add/remove pushes to connected providers
  immediately (removals gated by the opt-in removals toggle); the hourly job is
  the inbound/import + retry/reconcile path.
- Web: watch-provider settings gain watchlist import/export/removals and
  "mirror watchlist order" toggles plus watchlist sync stats.

Why
- The favorites and watchlist pipelines are ~90% identical; generalizing keeps
  one code path (per CLAUDE.md's anti-duplication guidance) instead of cloning.

API/compat
- All new fields on ConnectionStatus/Capabilities/ConnectionUpdate/SyncRun and
  the web types are additive (Silo v1 additive-only rule). No existing field is
  renamed, removed, or retyped.

Risks / follow-up
- MDBList capability flip is intentional and client-visible: silo-android /
  silo-apple may need to surface MDBList under the watchlist (not favorites) UI.
- MDBList existing users: their MDBList list previously mirrored Silo favorites
  and now mirrors Silo watchlist; the first post-migration sync is a union
  (removals default off), so nothing is destructively purged.
- Order mirroring reflects the order MDBList returns from /watchlist/items
  (couldn't confirm against their docs — Cloudflare-blocked); if it ever
  diverges from the UI sort, a sort param is the small follow-up.

Tests: new maintainer (auto-remove) and watchlist-order unit tests; provider +
service tests updated. go build, go test (affected pkgs), migrate-validate,
verify-local-paths, web prettier/eslint/tsc all pass.

AI-use disclosure: implemented with Claude Code (Claude Opus 4.8).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(watchsync): update list shadow table references

* fix(watchsync): address review — retry/progress + error propagation

Addresses CodeRabbit review on #227:
- maintainer: propagate transient catalog lookup errors instead of silently
  treating every items.GetByID failure as "maybe an episode".
- exportList: mark every queued item not confirmed sent (not_found, failed, or
  omitted) so the pending loop always advances; the next run's upsert clears the
  error and re-attempts, so transient failures still retry.
- removePendingListItems + realtime removal: treat Sent and NotFound as
  reconciled; leave true failures pending (no last_error, which would strand
  them from the removal query) so the scheduled run retries, using in-memory
  dedupe to terminate the loop.
- exportLocalListItems: send the normalized items (with computed
  ProviderItemKey), not the original event slice.
- UpdateConnection: clear mirrored watchlist order before persisting the disable
  and propagate failures, so a failed clear can't report "disabled" while
  sort_index ordering is still active.
- web: include favorite + watchlist removal counts in the exported "sent" total.
- test: align serviceFakeRepo list-state with Postgres (clear last_error on
  successful transitions); add maintainer error-propagation test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:05:53 -04:00

128 lines
5.6 KiB
SQL

-- +goose Up
-- +goose StatementBegin
-- Profile preference: remove an item from the watchlist once it is fully
-- watched (a movie on completion, a series once every episode is watched).
-- Defaults on; it is a standalone behavior that also propagates to connected
-- watchlist providers.
ALTER TABLE public.user_profiles
ADD COLUMN remove_watched_from_watchlist boolean NOT NULL DEFAULT true;
-- Per-connection watchlist sync toggles, mirroring the favorites toggles.
ALTER TABLE public.watch_provider_connections
ADD COLUMN import_watchlist_enabled boolean NOT NULL DEFAULT false,
ADD COLUMN export_watchlist_enabled boolean NOT NULL DEFAULT false,
ADD COLUMN sync_watchlist_removals_enabled boolean NOT NULL DEFAULT false,
ADD COLUMN last_watchlist_sync_at timestamptz;
-- Per-run watchlist counters, mirroring the favorites counters.
ALTER TABLE public.watch_provider_sync_runs
ADD COLUMN inbound_watchlist_found integer NOT NULL DEFAULT 0,
ADD COLUMN inbound_watchlist_imported integer NOT NULL DEFAULT 0,
ADD COLUMN outbound_watchlist_found integer NOT NULL DEFAULT 0,
ADD COLUMN outbound_watchlist_sent integer NOT NULL DEFAULT 0,
ADD COLUMN watchlist_removals_sent integer NOT NULL DEFAULT 0;
-- The favorites shadow table generalizes into a list-item shadow table keyed by
-- a list_kind discriminator ('favorites' | 'watchlist'). Existing rows are all
-- favorites, so the new column defaults accordingly.
ALTER TABLE public.watch_provider_favorite_items
RENAME TO watch_provider_list_items;
ALTER TABLE public.watch_provider_list_items
ADD COLUMN list_kind text NOT NULL DEFAULT 'favorites';
ALTER TABLE public.watch_provider_list_items
DROP CONSTRAINT watch_provider_favorite_items_connection_media_key;
ALTER TABLE public.watch_provider_list_items
ADD CONSTRAINT watch_provider_list_items_connection_kind_media_key
UNIQUE (connection_id, list_kind, media_item_id);
DROP INDEX IF EXISTS idx_watch_provider_favorite_items_provider_key;
CREATE UNIQUE INDEX idx_watch_provider_list_items_provider_key
ON public.watch_provider_list_items (connection_id, list_kind, provider_item_key)
WHERE provider_item_key <> '';
ALTER INDEX idx_watch_provider_favorite_items_connection_remote
RENAME TO idx_watch_provider_list_items_connection_remote;
ALTER INDEX idx_watch_provider_favorite_items_connection_local
RENAME TO idx_watch_provider_list_items_connection_local;
-- Re-map MDBList. MDBList exposes a single list — its watchlist — which Silo
-- historically bound to the *favorites* abstraction. Re-bind those connections
-- to the watchlist abstraction so MDBList's watchlist mirrors Silo's watchlist.
-- The stale favorites shadow rows are dropped so the next sync rebuilds clean
-- watchlist state (removals default off, so the first sync is a union, not a
-- purge).
UPDATE public.watch_provider_connections
SET import_watchlist_enabled = import_favorites_enabled,
export_watchlist_enabled = export_favorites_enabled,
sync_watchlist_removals_enabled = sync_favorite_removals_enabled,
last_watchlist_sync_at = last_favorites_sync_at,
import_favorites_enabled = false,
export_favorites_enabled = false,
sync_favorite_removals_enabled = false
WHERE provider = 'mdblist';
DELETE FROM public.watch_provider_list_items
WHERE list_kind = 'favorites'
AND connection_id IN (
SELECT id FROM public.watch_provider_connections WHERE provider = 'mdblist'
);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
-- Re-bind MDBList connections back to the favorites abstraction.
UPDATE public.watch_provider_connections
SET import_favorites_enabled = import_watchlist_enabled,
export_favorites_enabled = export_watchlist_enabled,
sync_favorite_removals_enabled = sync_watchlist_removals_enabled,
last_favorites_sync_at = last_watchlist_sync_at
WHERE provider = 'mdblist';
-- Collapsing list_kind back into a single list requires dropping any
-- watchlist-only rows to avoid violating the restored unique constraint.
DELETE FROM public.watch_provider_list_items
WHERE list_kind <> 'favorites';
ALTER INDEX idx_watch_provider_list_items_connection_local
RENAME TO idx_watch_provider_favorite_items_connection_local;
ALTER INDEX idx_watch_provider_list_items_connection_remote
RENAME TO idx_watch_provider_favorite_items_connection_remote;
DROP INDEX IF EXISTS idx_watch_provider_list_items_provider_key;
CREATE UNIQUE INDEX idx_watch_provider_favorite_items_provider_key
ON public.watch_provider_list_items (connection_id, provider_item_key)
WHERE provider_item_key <> '';
ALTER TABLE public.watch_provider_list_items
DROP CONSTRAINT watch_provider_list_items_connection_kind_media_key;
ALTER TABLE public.watch_provider_list_items
ADD CONSTRAINT watch_provider_favorite_items_connection_media_key
UNIQUE (connection_id, media_item_id);
ALTER TABLE public.watch_provider_list_items
DROP COLUMN list_kind;
ALTER TABLE public.watch_provider_list_items
RENAME TO watch_provider_favorite_items;
ALTER TABLE public.watch_provider_sync_runs
DROP COLUMN IF EXISTS watchlist_removals_sent,
DROP COLUMN IF EXISTS outbound_watchlist_sent,
DROP COLUMN IF EXISTS outbound_watchlist_found,
DROP COLUMN IF EXISTS inbound_watchlist_imported,
DROP COLUMN IF EXISTS inbound_watchlist_found;
ALTER TABLE public.watch_provider_connections
DROP COLUMN IF EXISTS last_watchlist_sync_at,
DROP COLUMN IF EXISTS sync_watchlist_removals_enabled,
DROP COLUMN IF EXISTS export_watchlist_enabled,
DROP COLUMN IF EXISTS import_watchlist_enabled;
ALTER TABLE public.user_profiles
DROP COLUMN IF EXISTS remove_watched_from_watchlist;
-- +goose StatementEnd