Files
silo-server/internal/userdb/sqlitestore.go
T
02e62767a1 feat(watchsync): sync watchlists with Trakt/Simkl/MDBList (#227)
* feat(watchsync): sync watchlists with Trakt/Simkl/MDBList

Extend the watch-providers feature to sync a user's watchlist, generalizing
the existing favorites pipeline rather than duplicating it.

What changed
- Generalize the favorites sync into one ListKind-parameterized pipeline
  (internal/watchsync/lists.go) driving both favorites and watchlist; the
  per-favorites service methods are replaced by kind-generic ones. The shadow
  table watch_provider_favorite_items becomes watch_provider_list_items with a
  list_kind discriminator.
- Providers: Trakt gains watchlist sync (/sync/watchlist, distinct from
  favorites); Simkl gains plan-to-watch sync; MDBList is re-mapped from
  favorites to watchlist (its only list is a watchlist) — its capabilities now
  report import_favorites=false / import_watchlist=true, and the migration
  re-binds existing MDBList connections.
- Auto-remove watched items from the watchlist: a standalone, default-on
  profile preference (user_profiles.remove_watched_from_watchlist) removes a
  movie when watched and a series once every episode is watched. Implemented as
  watchstate.CompletionObserver (internal/watchlist.Maintainer), wired into the
  manual mark-watched, playback-stop, and jellycompat mark-played paths.
- Optional MDBList sort-order mirroring: an opt-in, capability-gated toggle
  mirrors MDBList's watchlist order into Silo via user_watchlist.sort_index;
  ListWatchlist orders by sort_index then added_at, so both /api/v1/watchlist
  and the catalog watchlist view inherit it.
- Real-time + scheduled: local add/remove pushes to connected providers
  immediately (removals gated by the opt-in removals toggle); the hourly job is
  the inbound/import + retry/reconcile path.
- Web: watch-provider settings gain watchlist import/export/removals and
  "mirror watchlist order" toggles plus watchlist sync stats.

Why
- The favorites and watchlist pipelines are ~90% identical; generalizing keeps
  one code path (per CLAUDE.md's anti-duplication guidance) instead of cloning.

API/compat
- All new fields on ConnectionStatus/Capabilities/ConnectionUpdate/SyncRun and
  the web types are additive (Silo v1 additive-only rule). No existing field is
  renamed, removed, or retyped.

Risks / follow-up
- MDBList capability flip is intentional and client-visible: silo-android /
  silo-apple may need to surface MDBList under the watchlist (not favorites) UI.
- MDBList existing users: their MDBList list previously mirrored Silo favorites
  and now mirrors Silo watchlist; the first post-migration sync is a union
  (removals default off), so nothing is destructively purged.
- Order mirroring reflects the order MDBList returns from /watchlist/items
  (couldn't confirm against their docs — Cloudflare-blocked); if it ever
  diverges from the UI sort, a sort param is the small follow-up.

Tests: new maintainer (auto-remove) and watchlist-order unit tests; provider +
service tests updated. go build, go test (affected pkgs), migrate-validate,
verify-local-paths, web prettier/eslint/tsc all pass.

AI-use disclosure: implemented with Claude Code (Claude Opus 4.8).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(watchsync): update list shadow table references

* fix(watchsync): address review — retry/progress + error propagation

Addresses CodeRabbit review on #227:
- maintainer: propagate transient catalog lookup errors instead of silently
  treating every items.GetByID failure as "maybe an episode".
- exportList: mark every queued item not confirmed sent (not_found, failed, or
  omitted) so the pending loop always advances; the next run's upsert clears the
  error and re-attempts, so transient failures still retry.
- removePendingListItems + realtime removal: treat Sent and NotFound as
  reconciled; leave true failures pending (no last_error, which would strand
  them from the removal query) so the scheduled run retries, using in-memory
  dedupe to terminate the loop.
- exportLocalListItems: send the normalized items (with computed
  ProviderItemKey), not the original event slice.
- UpdateConnection: clear mirrored watchlist order before persisting the disable
  and propagate failures, so a failed clear can't report "disabled" while
  sort_index ordering is still active.
- web: include favorite + watchlist removal counts in the exported "sent" total.
- test: align serviceFakeRepo list-state with Postgres (clear last_error on
  successful transitions); add maintainer error-propagation test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:05:53 -04:00

404 lines
17 KiB
Go

package userdb
import (
"context"
"database/sql"
"fmt"
"time"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// SQLiteUserStore implements userstore.UserStore using a per-user SQLite database.
type SQLiteUserStore struct {
db *sql.DB
}
// NewSQLiteUserStore wraps an existing *sql.DB as a UserStore.
func NewSQLiteUserStore(db *sql.DB) *SQLiteUserStore {
return &SQLiteUserStore{db: db}
}
// Compile-time interface check.
var _ userstore.UserStore = (*SQLiteUserStore)(nil)
var _ userstore.DeviceRegistry = (*SQLiteUserStore)(nil)
// --- Profiles ---
func (s *SQLiteUserStore) CreateProfile(_ context.Context, p userstore.Profile) error {
return CreateProfile(s.db, p)
}
func (s *SQLiteUserStore) GetProfile(_ context.Context, id string) (*userstore.Profile, error) {
return GetProfile(s.db, id)
}
func (s *SQLiteUserStore) ListProfiles(_ context.Context) ([]userstore.Profile, error) {
return ListProfiles(s.db)
}
func (s *SQLiteUserStore) UpdateProfile(_ context.Context, id string, u userstore.UpdateProfileInput) error {
return UpdateProfile(s.db, id, u)
}
func (s *SQLiteUserStore) DeleteProfile(_ context.Context, id string) error {
return DeleteProfile(s.db, id)
}
func (s *SQLiteUserStore) VerifyPIN(_ context.Context, profileID, pin string) (bool, error) {
return VerifyPIN(s.db, profileID, pin)
}
// --- Progress ---
func (s *SQLiteUserStore) UpdateProgress(_ context.Context, profileID, mediaItemID string, position, duration float64, thresholds userstore.ProgressThresholds) error {
return UpdateProgress(s.db, profileID, mediaItemID, position, duration, thresholds)
}
func (s *SQLiteUserStore) SetProgress(_ context.Context, profileID, mediaItemID string, position, duration float64, thresholds userstore.ProgressThresholds) error {
return SetProgress(s.db, profileID, mediaItemID, position, duration, thresholds)
}
func (s *SQLiteUserStore) SetProgressAt(_ context.Context, profileID, mediaItemID string, position, duration float64, completed bool, updatedAt time.Time) error {
return SetProgressAt(s.db, profileID, mediaItemID, position, duration, completed, updatedAt)
}
func (s *SQLiteUserStore) SetProgressIfNewer(_ context.Context, profileID, mediaItemID string, position, duration float64, completed bool, updatedAt time.Time) (bool, error) {
return SetProgressIfNewer(s.db, profileID, mediaItemID, position, duration, completed, updatedAt)
}
func (s *SQLiteUserStore) UpdateProgressHints(_ context.Context, profileID, mediaItemID string, hints userstore.VersionHints) error {
return UpdateProgressHints(s.db, profileID, mediaItemID, hints)
}
func (s *SQLiteUserStore) MarkWatched(_ context.Context, profileID, mediaItemID string, duration float64) error {
return MarkWatched(s.db, profileID, mediaItemID, duration)
}
func (s *SQLiteUserStore) ClearProgress(_ context.Context, profileID, mediaItemID string) error {
return ClearProgress(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) MarkProgressBatch(_ context.Context, profileID string, mediaItemIDs []string, updatedAt time.Time) error {
return MarkProgressBatch(s.db, profileID, mediaItemIDs, updatedAt)
}
func (s *SQLiteUserStore) ClearProgressBatch(_ context.Context, profileID string, mediaItemIDs []string, updatedAt time.Time) error {
return ClearProgressBatch(s.db, profileID, mediaItemIDs, updatedAt)
}
func (s *SQLiteUserStore) GetProgress(_ context.Context, profileID, mediaItemID string) (*userstore.WatchProgress, error) {
return GetProgress(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) ListProgress(_ context.Context, profileID, status string, limit, offset int) ([]userstore.WatchProgress, error) {
return ListProgress(s.db, profileID, status, limit, offset)
}
func (s *SQLiteUserStore) ListProgressByMediaItems(_ context.Context, profileID string, mediaItemIDs []string) (map[string]userstore.WatchProgress, error) {
return ListProgressByMediaItems(s.db, profileID, mediaItemIDs)
}
func (s *SQLiteUserStore) AddHistory(_ context.Context, entry userstore.WatchHistoryEntry) error {
return AddHistory(s.db, entry)
}
func (s *SQLiteUserStore) AddVisibleHistory(_ context.Context, entry userstore.WatchHistoryEntry) (userstore.WatchHistoryEntry, error) {
return AddVisibleHistory(s.db, entry)
}
func (s *SQLiteUserStore) AddHistoryIfMissing(_ context.Context, entry userstore.WatchHistoryEntry) (bool, error) {
return AddHistoryIfMissing(s.db, entry)
}
func (s *SQLiteUserStore) ListHistory(_ context.Context, profileID string, limit, offset int) ([]userstore.WatchHistoryEntry, error) {
return ListHistory(s.db, profileID, limit, offset)
}
func (s *SQLiteUserStore) ListCompletedHistory(_ context.Context, query userstore.CompletedHistoryQuery) ([]userstore.WatchHistoryEntry, error) {
return ListCompletedHistory(s.db, query)
}
func (s *SQLiteUserStore) ListCompletedHistoryItems(_ context.Context, query userstore.CompletedHistoryItemQuery) ([]userstore.CompletedHistoryItem, error) {
return ListCompletedHistoryItems(s.db, query)
}
func (s *SQLiteUserStore) VisibleHistoryTimestamps(_ context.Context, profileID string, mediaItemIDs []string, at time.Time) (map[string]string, error) {
return VisibleHistoryTimestamps(s.db, profileID, mediaItemIDs, at)
}
func (s *SQLiteUserStore) RemoveHistoryItems(_ context.Context, profileID string, mediaItemIDs []string, removedAt time.Time) error {
return RemoveHistoryItems(s.db, profileID, mediaItemIDs, removedAt)
}
func (s *SQLiteUserStore) DeleteHistoryBySource(_ context.Context, profileID string, mediaItemIDs []string, source userstore.WatchHistorySource) error {
return DeleteHistoryBySource(s.db, profileID, mediaItemIDs, source)
}
func (s *SQLiteUserStore) ListHomeDismissals(_ context.Context, profileID, surface string) ([]userstore.HomeItemDismissal, error) {
return ListHomeDismissals(s.db, profileID, surface)
}
func (s *SQLiteUserStore) UpsertHomeDismissal(_ context.Context, dismissal userstore.HomeItemDismissal) error {
return UpsertHomeDismissal(s.db, dismissal)
}
func (s *SQLiteUserStore) DeleteHomeDismissal(_ context.Context, profileID, surface, mediaItemID string) error {
return DeleteHomeDismissal(s.db, profileID, surface, mediaItemID)
}
// --- Favorites & Watchlist ---
func (s *SQLiteUserStore) AddFavorite(_ context.Context, profileID, mediaItemID string) error {
return AddFavorite(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) AddFavoriteAt(_ context.Context, profileID, mediaItemID string, addedAt time.Time) error {
return AddFavoriteAt(s.db, profileID, mediaItemID, addedAt)
}
func (s *SQLiteUserStore) RemoveFavorite(_ context.Context, profileID, mediaItemID string) error {
return RemoveFavorite(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) ListFavorites(_ context.Context, profileID string, limit, offset int) ([]userstore.Favorite, error) {
return ListFavorites(s.db, profileID, limit, offset)
}
func (s *SQLiteUserStore) ListFavoritesByMediaItems(_ context.Context, profileID string, mediaItemIDs []string) (map[string]bool, error) {
return ListFavoritesByMediaItems(s.db, profileID, mediaItemIDs)
}
func (s *SQLiteUserStore) IsFavorite(_ context.Context, profileID, mediaItemID string) (bool, error) {
return IsFavorite(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) AddToWatchlist(_ context.Context, profileID, mediaItemID string) error {
return AddToWatchlist(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) AddToWatchlistAt(_ context.Context, profileID, mediaItemID string, addedAt time.Time) error {
return AddToWatchlistAt(s.db, profileID, mediaItemID, addedAt)
}
func (s *SQLiteUserStore) RemoveFromWatchlist(_ context.Context, profileID, mediaItemID string) error {
return RemoveFromWatchlist(s.db, profileID, mediaItemID)
}
func (s *SQLiteUserStore) ReplaceWatchlistOrder(_ context.Context, profileID string, orderedMediaItemIDs []string) error {
return ReplaceWatchlistOrder(s.db, profileID, orderedMediaItemIDs)
}
func (s *SQLiteUserStore) ListWatchlist(_ context.Context, profileID string, limit, offset int) ([]userstore.WatchlistEntry, error) {
return ListWatchlist(s.db, profileID, limit, offset)
}
func (s *SQLiteUserStore) ListWatchlistByMediaItems(_ context.Context, profileID string, mediaItemIDs []string) (map[string]bool, error) {
return ListWatchlistByMediaItems(s.db, profileID, mediaItemIDs)
}
func (s *SQLiteUserStore) InWatchlist(_ context.Context, profileID, mediaItemID string) (bool, error) {
return InWatchlist(s.db, profileID, mediaItemID)
}
// RemoveWatchedFromWatchlist defaults on for the embedded sqlite backend, which
// does not persist the per-profile preference.
func (s *SQLiteUserStore) RemoveWatchedFromWatchlist(_ context.Context, _ string) (bool, error) {
return true, nil
}
// --- Collections ---
func (s *SQLiteUserStore) CreateCollection(_ context.Context, input userstore.CreateCollectionInput) (*userstore.Collection, error) {
return CreateCollection(s.db, input)
}
func (s *SQLiteUserStore) GetCollection(_ context.Context, id string) (*userstore.Collection, error) {
return GetCollection(s.db, id)
}
func (s *SQLiteUserStore) ListCollections(_ context.Context, profileID string) ([]userstore.Collection, error) {
return ListCollections(s.db, profileID)
}
func (s *SQLiteUserStore) UpdateCollection(_ context.Context, input userstore.UpdateCollectionInput) error {
return UpdateCollection(s.db, input)
}
func (s *SQLiteUserStore) DeleteCollection(_ context.Context, id string) error {
return DeleteCollection(s.db, id)
}
func (s *SQLiteUserStore) AddCollectionItem(_ context.Context, collectionID, mediaItemID string, position int) error {
return AddCollectionItem(s.db, collectionID, mediaItemID, position)
}
func (s *SQLiteUserStore) RemoveCollectionItem(_ context.Context, collectionID, mediaItemID string) error {
return RemoveCollectionItem(s.db, collectionID, mediaItemID)
}
func (s *SQLiteUserStore) ListCollectionItems(_ context.Context, collectionID string) ([]userstore.CollectionItem, error) {
return ListCollectionItems(s.db, collectionID)
}
// User-collection imports are Postgres-only; SQLite never receives sync writes.
func (s *SQLiteUserStore) ReplaceCollectionItems(_ context.Context, _ string, _ []userstore.CollectionItemReplacement) error {
return fmt.Errorf("user collection imports are not supported on the SQLite user store")
}
func (s *SQLiteUserStore) ReorderCollectionItems(_ context.Context, _ string, _ []string) error {
return fmt.Errorf("collection reordering is not supported on the SQLite user store")
}
func (s *SQLiteUserStore) ReorderCollections(_ context.Context, _ string, _ *string, _ []string) error {
return fmt.Errorf("collection reordering is not supported on the SQLite user store")
}
func (s *SQLiteUserStore) UpdateCollectionSyncState(_ context.Context, _ userstore.UpdateCollectionSyncStateInput) error {
return fmt.Errorf("user collection imports are not supported on the SQLite user store")
}
func (s *SQLiteUserStore) ListCollectionGroups(_ context.Context) ([]userstore.CollectionGroup, error) {
return nil, nil
}
func (s *SQLiteUserStore) EnsureCollectionGroup(_ context.Context, _ string) error {
return nil
}
func (s *SQLiteUserStore) CreateCollectionGroup(_ context.Context, _, _ string, _ userstore.GroupSortMode) (*userstore.CollectionGroup, error) {
return nil, fmt.Errorf("collection groups are not supported on the SQLite user store")
}
func (s *SQLiteUserStore) UpdateCollectionGroup(_ context.Context, _ string, _ *string, _ *string, _ *userstore.GroupSortMode) (*userstore.CollectionGroup, error) {
return nil, fmt.Errorf("collection groups are not supported on the SQLite user store")
}
func (s *SQLiteUserStore) DeleteCollectionGroup(_ context.Context, _ string) error {
return fmt.Errorf("collection groups are not supported on the SQLite user store")
}
func (s *SQLiteUserStore) ReorderCollectionGroups(_ context.Context, _ []string) error {
return fmt.Errorf("collection groups are not supported on the SQLite user store")
}
// --- Section Overrides ---
func (s *SQLiteUserStore) ListSectionOverrides(_ context.Context, profileID, scope, libraryID string) ([]userstore.SectionOverride, error) {
return ListSectionOverrides(s.db, profileID, scope, libraryID)
}
func (s *SQLiteUserStore) SaveSectionOverrides(_ context.Context, profileID, scope, libraryID string, overrides []userstore.SectionOverride) error {
return SaveSectionOverrides(s.db, profileID, scope, libraryID, overrides)
}
func (s *SQLiteUserStore) ResetSectionOverrides(_ context.Context, profileID, scope, libraryID string) error {
return ResetSectionOverrides(s.db, profileID, scope, libraryID)
}
// --- Settings & Preferences ---
func (s *SQLiteUserStore) GetSetting(_ context.Context, key string) (string, error) {
return GetSetting(s.db, key)
}
func (s *SQLiteUserStore) SetSetting(_ context.Context, key, value string) error {
return SetSetting(s.db, key, value)
}
func (s *SQLiteUserStore) DeleteSetting(_ context.Context, key string) error {
return DeleteSetting(s.db, key)
}
func (s *SQLiteUserStore) ListSettings(_ context.Context) ([]userstore.SettingEntry, error) {
return ListSettings(s.db)
}
func (s *SQLiteUserStore) GetDeviceSetting(_ context.Context, profileID, deviceID, key string) (*userstore.DeviceSettingEntry, error) {
return GetDeviceSetting(s.db, profileID, deviceID, key)
}
func (s *SQLiteUserStore) RegisterDevice(_ context.Context, entry userstore.DeviceEntry) error {
return RegisterDevice(s.db, entry)
}
func (s *SQLiteUserStore) ListDevices(_ context.Context) ([]userstore.DeviceEntry, error) {
return ListDevices(s.db)
}
func (s *SQLiteUserStore) SetDeviceSetting(_ context.Context, entry userstore.DeviceSettingEntry) error {
return SetDeviceSetting(s.db, entry)
}
func (s *SQLiteUserStore) DeleteDeviceSetting(_ context.Context, profileID, deviceID, key string) error {
return DeleteDeviceSetting(s.db, profileID, deviceID, key)
}
func (s *SQLiteUserStore) DeleteAllDeviceSettings(_ context.Context, profileID, deviceID string) error {
return DeleteAllDeviceSettings(s.db, profileID, deviceID)
}
func (s *SQLiteUserStore) DeleteDeviceSettingsByKey(_ context.Context, key string) error {
return DeleteDeviceSettingsByKey(s.db, key)
}
func (s *SQLiteUserStore) ListDeviceSettings(_ context.Context, key string) ([]userstore.DeviceSettingEntry, error) {
return ListDeviceSettings(s.db, key)
}
func (s *SQLiteUserStore) ListAllDeviceSettings(_ context.Context) ([]userstore.DeviceSettingEntry, error) {
return ListAllDeviceSettings(s.db)
}
func (s *SQLiteUserStore) SetSubtitlePreference(_ context.Context, pref userstore.SubtitlePreference) error {
return SetSubtitlePreference(s.db, pref)
}
func (s *SQLiteUserStore) GetSubtitlePreference(_ context.Context, profileID, seriesID string) (*userstore.SubtitlePreference, error) {
return GetSubtitlePreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) DeleteSubtitlePreference(_ context.Context, profileID, seriesID string) error {
return DeleteSubtitlePreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) SetAudioPreference(_ context.Context, pref userstore.AudioPreference) error {
return SetAudioPreference(s.db, pref)
}
func (s *SQLiteUserStore) GetAudioPreference(_ context.Context, profileID, seriesID string) (*userstore.AudioPreference, error) {
return GetAudioPreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) DeleteAudioPreference(_ context.Context, profileID, seriesID string) error {
return DeleteAudioPreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) SetSeriesPlaybackPreference(_ context.Context, pref userstore.SeriesPlaybackPreference) error {
return SetSeriesPlaybackPreference(s.db, pref)
}
func (s *SQLiteUserStore) GetSeriesPlaybackPreference(_ context.Context, profileID, seriesID string) (*userstore.SeriesPlaybackPreference, error) {
return GetSeriesPlaybackPreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) DeleteSeriesPlaybackPreference(_ context.Context, profileID, seriesID string) error {
return DeleteSeriesPlaybackPreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) GetLibraryPlaybackPreference(_ context.Context, profileID string, libraryID int) (*userstore.LibraryPlaybackPreference, error) {
return GetLibraryPlaybackPreference(s.db, profileID, libraryID)
}
func (s *SQLiteUserStore) ListLibraryPlaybackPreferences(_ context.Context, profileID string) ([]userstore.LibraryPlaybackPreference, error) {
return ListLibraryPlaybackPreferences(s.db, profileID)
}
func (s *SQLiteUserStore) UpsertLibraryPlaybackPreference(_ context.Context, pref userstore.LibraryPlaybackPreference) error {
return UpsertLibraryPlaybackPreference(s.db, pref)
}
func (s *SQLiteUserStore) DeleteLibraryPlaybackPreference(_ context.Context, profileID string, libraryID int) error {
return DeleteLibraryPlaybackPreference(s.db, profileID, libraryID)
}