- Forward start, pause, resume, and stop events with stable media identities
- Persist and retry terminal scrobbles across teardown and restart paths
- Reject ambiguous playback-report route matches
The watchlist catalog page silently returned the stored list order while
its sort dropdown claimed "Date Added", and mirroring a provider's list
order (e.g. MDBList) was off by default, so synced watchlists appeared
in first-sync-time order with no way to tell what was happening.
- Web: the watchlist source now uses the same source-order sentinel as
collections — the dropdown shows "List Order" as the default, and an
explicit "Date Added" pick sends sort=added_at instead of being
stripped (previously indistinguishable from the default).
- Server: on personal lists (watchlist/favorites) an explicit added_at
sort now takes the source-order path, where added_at means "date
added to the list"; the query executor path sorted by the library's
created_at instead. History keeps the executor path since its ID
loading ignores the sort.
- watchsync: new connections default sync_watchlist_order_enabled to
true, with a migration flipping existing rows to match. Providers
without the provides_watchlist_order capability ignore the flag at
sync time.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(observability): OpenTelemetry logs+traces with secret redaction
Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing
stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with
no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged.
Bootstrap (internal/telemetry):
- Setup() builds one shared resource, a TracerProvider (parent-based trace-id
ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator
from env. It installs NO MeterProvider — metrics stay on Prometheus, and the
built-in no-op global MeterProvider keeps the trace instrumentation libs from
double-emitting. Shutdown is deferred with a flush timeout.
- Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the
shared LevelVar and best-effort so a failing collector can't break the console
or DB branches. stderr + opslog stay untouched.
Secret redaction (internal/logredact):
- A slog.Handler masks secret-keyed attributes (password, token, api_key,
authorization, cookie, ...) — including .With-bound attrs, nested groups,
secret-keyed group subtrees, and values behind a LogValuer — on the console
and OTLP sinks, with a no-op fast path when a record has no secret keys.
opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one
marker list.
Rotation is infra-managed (no custom file sink): container runtime for stderr,
collector/backend for OTLP, opslog partition-pruning for the DB. Documented in
docs/architecture/observability.md.
Verification: go build ./..., go vet, gofmt -l — clean; go test
./internal/telemetry/ ./internal/logredact/ -race pass.
AI-use disclosure: implemented with AI assistance (Claude Code), including
adversarial reviews that hardened the bootstrap and fixed two redaction leak
paths; reviewed by the author.
* refactor(observability): slog context+component sweep, sloglint gate (phase 3)
Part of #265. Builds on the OTel bootstrap + redaction commit.
Standardizes every log call site onto the context-carrying slog variants so
records correlate with the active OpenTelemetry trace, and locks the standard
in with a machine gate so future code (human- or AI-authored) can't drift back.
- Call-site sweep: converted the remaining slog.<Level>(...) calls to the
slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope
(background/init calls with no ctx are left as-is), across 183 files. Applied
via a type-aware AST codemod. Log levels and message strings are preserved
verbatim; a component attr (canonical per-package name) is added to direct
package-level slog calls. Bound-logger calls keep their existing .With
bindings. The main.go and telemetry package conversions rode with their file
in the previous commit to keep each file within a single commit.
- Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg,
key-naming-case=snake, no-mixed-args. After the sweep all four report zero
violations repo-wide (tests included), so make lint / CI now blocks any
regression to the non-context form. The gate ships with the sweep because it
cannot be green until the legacy sites are converted.
Metrics remain on Prometheus; no behavior change to /metrics or Grafana.
Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4
rules) 0 violations repo-wide; log levels verified unchanged.
AI-use disclosure: implemented with AI assistance (Claude Code), including the
codemod; reviewed by the author.
* fix(observability): honor per-signal OTLP protocol and secret WithGroup names
Two Codex review findings on PR #290:
- telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the
generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector
setups (e.g. HTTP logs + gRPC traces) build the right exporter.
- logredact: entering a group whose name is secret-bearing (e.g.
WithGroup("authorization")) now masks every leaf in that subtree,
matching how slog.Group("authorization", ...) is masked as a whole.
* fix(observability): address review feedback on telemetry bootstrap
- Telemetry setup failure no longer kills boot: Setup returns usable
no-op providers alongside the error and main logs and continues with
telemetry disabled, honoring the best-effort contract.
- Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_*
variants); unsupported values fall back to parentbased_traceidratio.
- Attach node identity as semconv service.instance.id instead of the
non-semconv node.name.
- Rename opslog retention-scope log attrs to target_component/target_level
so they no longer collide with the canonical component routing key, and
tag those lines with component=opslog.
- Fix stale levelGated comment casing; use WarnContext in the telemetry
shutdown defer; document the LogValuer double-resolve on the redaction
slow path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(watchsync): respect MDBList rate limits and defer syncs on 429
MDBList caps API usage at 1,000 requests/day on the free tier, and a
large-library first sync (paginated watched/watchlist fetches plus
exports chunked at 100 items per POST) could blow through it. A 429 was
treated as a generic failure: every pending chunk was marked failed and
the next scheduled run replayed the whole sync into the same limit.
- Pace MDBList requests at ~1/s per API key and retry 429s with a short
Retry-After in place; longer waits surface a typed RateLimitedError.
- Abort the remaining sync flows on the first rate-limited flow and
persist rate_limited_until on the connection; scheduled syncs skip it
until the deferral passes and manual sync returns a proper cooldown.
- Leave rate-limited exports pending instead of marking them failed so
the next run resumes where it stopped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(watchsync): address rate-limit review feedback
- Floor RateLimitedError.RetryAfter to the default deferral when in-place
retries are exhausted, so untrustworthy short Retry-After hints can't
produce a seconds-long deferral that walks straight back into the limit.
- Stamp rate_limited_until on every connection bound to the same provider
account (the quota belongs to the API key, not the profile), and re-read
connections mid-batch in SyncDueConnections so siblings deferred after
the snapshot are skipped.
- Filter deferred connections out of the live dispatch queries
(local watch events, list events, scrobbles) so real-time exports stop
burning quota during a cooldown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(historyimport): import the Plex account watchlist alongside watch history
The Plex import migrated only watch history; the user's saved watchlist
had to be rebuilt by hand (#245).
- PlexClient gains FetchWatchlist: pages the account-level watchlist on
the Plex discover API (discover.provider.plex.tv). It authenticates
with the plex.tv ACCOUNT token — the PIN/OAuth session token, which
resolvePlexAuth now threads through plexAuth.AccountToken (manual-token
imports pass the user token, which doubles as the account token).
- Watchlist entries become import Records flagged Watchlisted, carrying
identity only (movie/show → KindMovie/KindSeries, guids parsed) and no
watch state. They ride the existing matcher (series matching already
exists), and matched entries are added to the importing profile's
watchlist via the idempotent AddToWatchlistAt — re-imports do not
duplicate. A watchlist fetch failure downgrades to a run warning so the
history import still completes.
- Run summaries gain a watchlist_added counter (new column + repo
plumbing + client/admin UI cards).
Fixes#245
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(historyimport): count only newly inserted watchlist rows in WatchlistAdded
AddToWatchlistAt now reports whether a row was actually inserted (the
insert is ON CONFLICT DO NOTHING / INSERT OR IGNORE), and the import
summary increments WatchlistAdded only for genuine inserts, so
re-importing the same Plex account no longer inflates the count.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
* fix(watchsync): omit empty Trakt episode ids so series mark-watched targets the right show
omitempty on a struct value is a no-op in encoding/json, so the history
export sent all-zero episode ids ({tmdb:0,tvdb:0}); Trakt matched the
degenerate id to one default show, mis-recording every watched series.
Make episode IDs a *traktIDs pointer and attach it only when a real id
exists, else use the show + season/number fallback (mirroring scrobble).
Adds a debug log on the show-fallback path and payload tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(watchsync): sync TV episodes to Trakt via valid nested shows payload
Episode history exports were dropped by Trakt because they were emitted
into the flat episodes[] array with a bogus sibling show object plus
season/number keys — a shape the Trakt API does not accept, so it
silently discarded them (200/201 with no history recorded). Movies were
unaffected since they always carry their own external IDs.
Two coordinated changes:
- watchstate/identity.go: ResolveHistoryIdentity now carries the
episode's own imdb/tmdb/tvdb IDs (already stored on the episodes
table) so episodes with real IDs export via the flat episodes[].ids
form, matching how movies work.
- watchsync/providers/trakt/provider.go: episodes without their own ID
now serialize into the correct nested shows[].seasons[].episodes[]
structure keyed by the show's IDs, merging episodes by show/season.
Same fix applied to the history-remove payload. Empty-payload guards
account for the new shows[] list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(watchstate): keep episode identity when episode has its own IDs
Address CodeRabbit review on PR #254: the episode identity builder
dropped the whole identity whenever series IDs were empty, so episodes
with a valid episode IMDb/TMDB/TVDB ID but no series IDs never reached
the flat episodes[].ids Trakt path. Only require series IDs when the
episode has no IDs of its own (nested show fallback).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(watchsync): sync watchlists with Trakt/Simkl/MDBList
Extend the watch-providers feature to sync a user's watchlist, generalizing
the existing favorites pipeline rather than duplicating it.
What changed
- Generalize the favorites sync into one ListKind-parameterized pipeline
(internal/watchsync/lists.go) driving both favorites and watchlist; the
per-favorites service methods are replaced by kind-generic ones. The shadow
table watch_provider_favorite_items becomes watch_provider_list_items with a
list_kind discriminator.
- Providers: Trakt gains watchlist sync (/sync/watchlist, distinct from
favorites); Simkl gains plan-to-watch sync; MDBList is re-mapped from
favorites to watchlist (its only list is a watchlist) — its capabilities now
report import_favorites=false / import_watchlist=true, and the migration
re-binds existing MDBList connections.
- Auto-remove watched items from the watchlist: a standalone, default-on
profile preference (user_profiles.remove_watched_from_watchlist) removes a
movie when watched and a series once every episode is watched. Implemented as
watchstate.CompletionObserver (internal/watchlist.Maintainer), wired into the
manual mark-watched, playback-stop, and jellycompat mark-played paths.
- Optional MDBList sort-order mirroring: an opt-in, capability-gated toggle
mirrors MDBList's watchlist order into Silo via user_watchlist.sort_index;
ListWatchlist orders by sort_index then added_at, so both /api/v1/watchlist
and the catalog watchlist view inherit it.
- Real-time + scheduled: local add/remove pushes to connected providers
immediately (removals gated by the opt-in removals toggle); the hourly job is
the inbound/import + retry/reconcile path.
- Web: watch-provider settings gain watchlist import/export/removals and
"mirror watchlist order" toggles plus watchlist sync stats.
Why
- The favorites and watchlist pipelines are ~90% identical; generalizing keeps
one code path (per CLAUDE.md's anti-duplication guidance) instead of cloning.
API/compat
- All new fields on ConnectionStatus/Capabilities/ConnectionUpdate/SyncRun and
the web types are additive (Silo v1 additive-only rule). No existing field is
renamed, removed, or retyped.
Risks / follow-up
- MDBList capability flip is intentional and client-visible: silo-android /
silo-apple may need to surface MDBList under the watchlist (not favorites) UI.
- MDBList existing users: their MDBList list previously mirrored Silo favorites
and now mirrors Silo watchlist; the first post-migration sync is a union
(removals default off), so nothing is destructively purged.
- Order mirroring reflects the order MDBList returns from /watchlist/items
(couldn't confirm against their docs — Cloudflare-blocked); if it ever
diverges from the UI sort, a sort param is the small follow-up.
Tests: new maintainer (auto-remove) and watchlist-order unit tests; provider +
service tests updated. go build, go test (affected pkgs), migrate-validate,
verify-local-paths, web prettier/eslint/tsc all pass.
AI-use disclosure: implemented with Claude Code (Claude Opus 4.8).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(watchsync): update list shadow table references
* fix(watchsync): address review — retry/progress + error propagation
Addresses CodeRabbit review on #227:
- maintainer: propagate transient catalog lookup errors instead of silently
treating every items.GetByID failure as "maybe an episode".
- exportList: mark every queued item not confirmed sent (not_found, failed, or
omitted) so the pending loop always advances; the next run's upsert clears the
error and re-attempts, so transient failures still retry.
- removePendingListItems + realtime removal: treat Sent and NotFound as
reconciled; leave true failures pending (no last_error, which would strand
them from the removal query) so the scheduled run retries, using in-memory
dedupe to terminate the loop.
- exportLocalListItems: send the normalized items (with computed
ProviderItemKey), not the original event slice.
- UpdateConnection: clear mirrored watchlist order before persisting the disable
and propagate failures, so a failed clear can't report "disabled" while
sort_index ordering is still active.
- web: include favorite + watchlist removal counts in the exported "sent" total.
- test: align serviceFakeRepo list-state with Postgres (clear last_error on
successful transitions); add maintainer error-propagation test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(collections): add profile-scoped display filters
* refactor(collections): dedup display-filter helpers per review
Address code-review feedback on the profile-scoped display filters
without changing behavior:
- Widen CompletedHistoryItemMap to accept ProgressCompletionStore and
drop the duplicate completedHistoryItemMapForProgress copy.
- Extract the duplicated MDBList candidate retry loop into a generic
collectionutil.FetchMDBListWithFallback helper, used by both the user
and library collection syncers, and cover it with unit tests.
- Reuse validateOptionalLibraryIDs in HandleUpdateCollection instead of
an inline positive-ID loop.
- Import the shared COLLECTION_{WATCH,MEDIA}_FILTER_OPTIONS in the
template config form rather than redefining them locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(collections): sanitize query_definition library_ids fallback
readSourceConfigLibraryIDs validated source_config.library_ids (finite,
positive, truncated, deduplicated) but returned the query_definition
fallback raw, so legacy rows could surface zero/negative/duplicate IDs
that the backend now rejects on save. Extract a shared sanitizer and
apply it to both paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(docs): This makes the agents annoying to work with
* Improve playback session handling
* Support collection source order in catalog filters
* fix(collections): address display filter review feedback
* refactor(catalog): remove duplicate collection query params
* Hide episode media scope for collection overlays
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(security): encrypt server-owned credentials at rest
Introduce AES-256-GCM at-rest encryption (HKDF-derived from a required
SECRET_KEY) for server-owned credentials, with row-bound AAD, a versioned
enc:v1: envelope, and an idempotent startup backfill.
- internal/secret: cipher + RowAAD/SettingsAAD + the startup backfill engine.
- SECRET_KEY required at bootstrap; cipher threaded as an explicit dependency.
- server_settings: EncryptedSettingsRepo decorator over the audited
SensitiveSettingKeys (also drives admin redaction); the config watcher and
watch-sync settings reads decrypt too.
- Arr keys inline-encrypted; the ambiguous SecretResolver indirection removed
from requests/autoscan.
- Per-table columns encrypted: subtitles, watch-sync, webhook-sync (not
webhook_secret), history-import, and the jellycompat session's bridged Silo
access/refresh tokens.
- Startup backfill (resolve-then-encrypt for arr refs) is best-effort and
primary-node gated.
Equality-looked-up secrets and plugin_runtime_configs.config_value are out of
scope (need hashing / cross-repo design) — see
docs/architecture/secret-encryption.md.
Refs #45
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(compose): require SECRET_KEY in docker-compose
The server now fatals without SECRET_KEY, so the integrated service (and the
commented distributed proxy/transcode examples) pass it through with a
fail-fast guard matching the existing MEDIA_ROOT pattern. Distributed worker
nodes must use the SAME key as the primary to decrypt shared data.
Generate with: openssl rand -base64 48.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): encrypt history import session credentials
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
