From adeda3c87cd064b95d3b4f13024535d1e7fccdc2 Mon Sep 17 00:00:00 2001 From: Quick <31828688+Quick104@users.noreply.github.com> Date: Thu, 11 Jun 2026 10:38:29 -0400 Subject: [PATCH] docs(notifications): add notification system design plans Imported the Continuum-era notification specs (durable inbox + websocket foundation, APNs relay, FCM relay, outbound webhooks) and amended them for Silo. Amendments from the 2026-06-11 review: wire contracts normalized to Silo naming, back-catalog seeding and per-series burst suppression, durable dispatch outbox, cross-library episode dedupe, forward-sync wake API, ticket-based websocket handshake, relay threat-model additions (egress IP, keyed collapse IDs), and webhook 4xx/SSRF hardening. Includes a self-contained design-decisions.html visual overview. Co-Authored-By: Claude Fable 5 --- .../notifications/00-architecture-overview.md | 388 +++++++ .../01-release-events-and-inbox.md | 1022 +++++++++++++++++ .../plans/notifications/02-apns-relay.md | 891 ++++++++++++++ .../plans/notifications/03-fcm-relay.md | 748 ++++++++++++ .../notifications/04-outbound-webhooks.md | 731 ++++++++++++ .../superpowers/plans/notifications/README.md | 88 ++ .../plans/notifications/design-decisions.html | 281 +++++ 7 files changed, 4149 insertions(+) create mode 100644 docs/superpowers/plans/notifications/00-architecture-overview.md create mode 100644 docs/superpowers/plans/notifications/01-release-events-and-inbox.md create mode 100644 docs/superpowers/plans/notifications/02-apns-relay.md create mode 100644 docs/superpowers/plans/notifications/03-fcm-relay.md create mode 100644 docs/superpowers/plans/notifications/04-outbound-webhooks.md create mode 100644 docs/superpowers/plans/notifications/README.md create mode 100644 docs/superpowers/plans/notifications/design-decisions.html diff --git a/docs/superpowers/plans/notifications/00-architecture-overview.md b/docs/superpowers/plans/notifications/00-architecture-overview.md new file mode 100644 index 00000000..f5fa147b --- /dev/null +++ b/docs/superpowers/plans/notifications/00-architecture-overview.md @@ -0,0 +1,388 @@ +# Silo Notifications — Architecture Overview + +**Date:** 2026-04-28 +**Status:** Draft +**Scope:** Cross-cutting design covering all delivery channels: durable in-app inbox, realtime websocket, Apple Push (APNs), Android Push (FCM), and outbound webhooks. +**Companion docs:** +- [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) — Foundation +- [`02-apns-relay.md`](./02-apns-relay.md) — Apple push +- [`03-fcm-relay.md`](./03-fcm-relay.md) — Android push +- [`04-outbound-webhooks.md`](./04-outbound-webhooks.md) — Discord and generic webhooks + +## Goal + +Silo should notify a profile when a new episode of a series they care about becomes available in their library. "Care about" means at least one of: the series is favorited, watchlisted, in continue-watching, or has a profile-eligible next-up cursor. + +The notification should reach the profile through whichever delivery channels the profile has configured: an in-app inbox with realtime push to connected clients, mobile push notifications to Apple and Android devices, and outbound webhooks to user-chosen destinations like a Discord channel. + +## Why this is hard + +Three constraints pull in different directions: + +1. **Self-hosted privacy.** Silo is self-hosted media. Notification content (episode titles, posters, profile names, library identity) must never leave the user's server unless the user has explicitly opted into a destination that requires it. +2. **Mobile push gatekeeping.** Apple and Google require pushes to the official App Store / Play Store builds to be authenticated by the credentials of the developer account that signed and published the app. There is no fully self-hosted, zero-Silo-infrastructure path to push to the official mobile apps. +3. **Multi-profile fairness.** A single user account on a Silo server can have multiple profiles (kid, adult, parent, etc.). Notifications must be scoped to the active profile. Devices, push tokens, webhooks, and preferences are all profile-scoped. + +The design resolves these by: + +- Routing all notification work through a durable per-profile inbox that lives on the user's own server. Everything else is a transport on top of this inbox. +- Operating a privacy-preserving relay for APNs and FCM that holds the official-app credentials and accepts only opaque push requests. Self-hosted servers opt in. The relay never sees notification content. +- Profile-scoping every database row, every API surface, every websocket subscription, and every push registration. + +## Mental model + +``` + ┌─────────────────────────────────────────────────┐ + │ │ + ingest/scan ────► release_events (one row per newly-available │ + │ episode in a library) │ + │ │ │ + │ ▼ │ + │ fanout worker │ + │ │ │ + │ │ loads profile_series_interest │ + │ │ applies eligibility + preferences │ + │ ▼ │ + │ notification_deliveries (one row per │ + │ eligible profile per release) │ + │ │ │ + │ │ AFTER COMMIT │ + │ ▼ │ + │ channel dispatcher ◄── fans out to channels: │ + │ │ + │ ┌───────────────┬───────────────┬─────────┐ │ + │ │ │ │ │ │ + │ ▼ ▼ ▼ ▼ │ + │ websocket apple push android webhooks│ + │ (in-app) (APNs relay push (per │ + │ or custom) (FCM) profile)│ + │ │ │ │ │ │ + └───┼───────────────┼───────────────┼─────────┼───┘ + ▼ ▼ ▼ ▼ + connected iOS / tvOS / Android Discord / + browser tabs macOS apps apps user URL + and apps (wake then (wake) (content + fetch metadata) included) +``` + +Three invariants make this work: + +- **The durable row is the source of truth.** Realtime websocket events, mobile pushes, and webhooks are all triggered *after* the `notification_deliveries` row commits. If any transport fails, the inbox row still exists and shows up on next reconnect or refresh. +- **The fanout worker decides recipients exactly once.** Recipients are derived from a compact `profile_series_interest` index keyed by `(library_id, series_id)`. Per-channel transports are purely about delivery — they never make recipient decisions. +- **Dispatch enqueue is durable (outbox).** The same fanout transaction that inserts `notification_deliveries` also inserts `pending` per-target attempt rows for the push and webhook channels (`push_delivery_attempts` / `webhook_delivery_attempts`). The post-commit dispatcher *claims* those rows for immediate delivery; recovery workers sweep stale `pending` rows. A crash between delivery commit and dispatch therefore delays pushes/webhooks instead of silently dropping them. The websocket channel deliberately has no outbox row — it is best-effort because the inbox snapshot covers reconnect. See `01-release-events-and-inbox.md` "Transaction and Concurrency Rules". + +## Channels at a glance + +| Channel | Where it lives | Trust model | Content visibility | +|---|---|---|---| +| **In-app inbox** | User's own server, accessed via authenticated REST API | Same auth as the rest of Silo | Full content | +| **Realtime websocket** | User's own server, `/events/ws` channel `notifications` | Same auth, profile-scoped | Full content | +| **Apple Push (APNs)** | Silo-operated relay (or admin-supplied custom APNs credentials) | Opt-in. Self-hosters trust the relay to forward opaque requests. | Opaque payload only — app fetches content from user server after wake | +| **Android Push (FCM)** | Silo-operated relay (or admin-supplied custom FCM credentials) | Same as APNs | Opaque data-only message — app fetches content from user server after wake | +| **Outbound webhooks** | User-chosen destination URL (e.g., Discord, Slack, custom service) | Profile chose the destination; trust is implicit | Full content included by default. Discord type renders native embeds; generic type sends signed JSON | + +## Triggers (what generates a notification) + +A notification is generated when a new episode becomes available in a library and at least one profile has interest in the series: + +- **Favorite** — the series is favorited by the profile. +- **Watchlist** — the series is on the profile's watchlist. +- **Continue watching** — the profile has in-progress watch state for the series. +- **Next up** — the new episode is at or beyond the profile's `next_expected_episode_key` cursor and the profile has progression state for the series. + +A profile may match multiple reasons for a single release event. The `notification_deliveries` row records all matching reasons in a `reason_flags` JSONB column; the fanout produces exactly one delivery per `(profile_id, release_event_id)` regardless of how many reasons matched. A second uniqueness rule spans libraries: because Silo media items are catalog-level (`media_item_libraries` is a junction), the same episode landing in two libraries (e.g., "TV" and "TV 4K") shares one `episode_id`, and at most one `episode.available` delivery is created per `(profile_id, episode_id)` — dual-quality library setups do not double-notify. + +**Back-catalog protection:** "newly available" must mean *newly released to this server*, not *newly seen by the notifications feature*. The first scan of a newly created library and the one-time feature-enable backfill **seed** `episode_availability` without creating release events, and the fanout worker applies a per-series burst cap for bulk additions to existing libraries. Without this, importing a 200-episode back-catalog of a series with hundreds of interested profiles would generate tens of thousands of deliveries and pushes in a single scan. Full rules in `01-release-events-and-inbox.md` "Seeding and Burst Suppression". + +**v1 explicitly does not support:** + +- Movie additions. Silo has no movie-availability detector parallel to the per-episode one in this design; without it, there's nothing to fan out from for movies. Users *can* watchlist movies that don't yet exist in their library (via TMDB lookups), but the missing piece is the ingest-time signal that a watchlisted movie has now arrived. v2 can add a `movie_availability` table and analogous release events; the fanout/inbox/channel layers in this design are movie-ready by construction. +- Metadata-only updates (poster refreshes, title corrections). +- Generic library-changed notifications. +- Aggregated notifications like "3 new episodes available." Per-episode deliveries first; aggregation is a presentation concern that can stack on top. + +## Mode terminology + +The design uses three vocabularies that correspond to different layers; they are intentionally not the same word. This is the canonical mapping: + +| Layer | Terms | Where it lives | +|---|---|---| +| **Profile-level push mode** (UI-facing) | `off`, `in_app_only`, `private_push` | `push_devices.push_mode` | +| **APNs wire mode** (relay request `mode` field) | `private_alert`, `background_wake` | Sent to APNs relay or used in custom APNs payload builder | +| **FCM wire mode** (relay request `mode` field) | `private_data`, `background_wake` | Sent to FCM relay or used in custom FCM payload builder | + +Mapping: + +- Profile `private_push` → APNs `private_alert` for visible wakes, FCM `private_data` for visible wakes. +- Profile `in_app_only` → no remote push at all (websocket / inbox only). +- Profile `off` → no notifications for this device whatsoever. +- The `background_wake` wire mode is reserved for low-noise sync (per-channel spec details). It is not exposed at profile level in v1. + +## Preference model + +Preferences are profile-scoped and flat. There is no per-channel × per-reason matrix. + +``` +notification_preferences (per profile) +├── enabled bool — global kill switch +├── notify_favorites bool +├── notify_watchlist bool +├── notify_continue_watching bool +└── notify_next_up bool +``` + +Per-webhook reason gating uses **the same four boolean columns** on `notification_webhooks` (`notify_favorites`, `notify_watchlist`, `notify_continue_watching`, `notify_next_up`) — not a single structured `reason_filters` JSON field. + +**Precedence:** profile preferences are a **hard gate** applied during fanout (a delivery row is not created if all matching reasons are disabled at the profile level). Per-webhook flags and per-device push mode are **additional filters** applied during dispatch — they can narrow what fires for a specific destination but cannot re-enable a reason the profile has globally disabled. + +Each push device has a `push_mode` (`off`, `in_app_only`, `private_push`) that gates whether mobile push fires for that specific device. Each webhook has its own `enabled` flag and per-webhook `notify_favorites` / `notify_watchlist` / `notify_continue_watching` / `notify_next_up` booleans for finer control. + +The fanout worker uses `notification_preferences` to decide whether to *create* a `notification_delivery` at all. The channel dispatcher uses per-device and per-webhook flags to decide which transports fire after the row commits. + +This means: if a profile turns off `notify_continue_watching`, a delivery is never created for a continue-watching-only match, and no transports fire. If a profile leaves the reason flags on but disables a specific Discord webhook, the inbox row + websocket + push still fire normally — the webhook just doesn't. + +## Mobile push: why a relay is necessary + +Apple and Google operate the only push infrastructure that can wake a closed app on iOS and Android. To send a push to the official Silo app on the App Store or Play Store, the request must be authenticated by the credentials of the developer account that signed and published that app build: + +- **APNs:** Apple Developer Team ID + Auth Key ID + `.p8` private key. The auth key is account-scoped (not topic-scoped); it can sign JWTs that authorize push to any bundle topic the team owns. The `apns-topic` request header selects which topic the push targets, and Apple rejects requests whose topic is not owned by the signing team. +- **FCM:** Firebase service account JSON whose private key signs a JWT exchanged for an OAuth2 access token. The service account is scoped to a specific Firebase project, and the project owns one or more registered Android app package names. (Package + SHA-1 cert fingerprint registration matters for Firebase Authentication and App Check, but for plain FCM v1 push the service account + package name + matching FCM registration token are what's required.) + +Distributing these credentials to thousands of self-hosted Silo installations is not viable: + +1. It violates Apple's and Google's developer agreements, which prohibit sharing credentials outside the team. +2. Anyone with a copy could push to every Silo app worldwide. +3. Apple actively monitors and revokes leaked keys. + +Therefore the design separates two paths: + +### Hosted relay (`silo_relay`) — the default for official-app users + +A small Silo-operated service holds the official APNs `.p8` key and FCM service account JSON. Self-hosted servers opt in by configuring a relay API key, then call the relay with opaque push requests. The relay signs and forwards to APNs / FCM. The relay never sees notification content. + +The relay is **stateless on the request path**. Its database holds only: + +- relay accounts (one per opt-in self-hosted server installation) +- relay API keys (with `last_used_at`, revocable) +- minimal per-account allowlists (which app topics this account may push to) +- redacted operational logs (no APNs/FCM tokens, no notification content) + +It does **not** store user identities, profile data, device-to-user mappings, or notification content. Apple and Google do the actual addressing — when a push request goes out, the relay just forwards `{token, opaque_ids}` to APNs/FCM, and APNs/FCM deliver based on the device token. + +### Custom credentials (`custom_apns` / `custom_fcm`) — escape hatch + +Power users, white-label deployments, or anyone running a fork that ships its own resigned mobile app can configure their own Apple/Google developer credentials. The user's Silo server signs requests directly using those credentials and sends to APNs/FCM without going through the Silo relay. + +Custom mode is **only useful if the admin has shipped a custom-signed app build under their own developer account** with their own bundle/package ID. It does not let a self-hoster push to the official App Store / Play Store app — that's not technically possible without the official developer credentials. + +### Addressing flow walkthrough + +The most common question on this design is "how does the relay know which user/device to send to with thousands of servers and thousands of users?" The answer is: it doesn't, because Apple and Google handle addressing. + +``` +1. User installs official Silo app on iPhone from App Store. +2. App calls iOS push registration; Apple returns an opaque APNs device token + (~100 bytes, unique per app install per device). +3. App POSTs that token to the user's OWN server: + POST https://my-silo.example.com/api/v1/devices/push/apple + { device_id, apns_token, ... } +4. User's server stores in its push_devices table, scoped to the active profile. +5. New episode lands. Fanout worker creates a notification_delivery row. +6. Channel dispatcher loads the profile's enabled push_devices and POSTs + to the relay (or signs directly via custom_apns): + POST https://relay.silo.app/v1/apple/send + Authorization: Bearer rk_thisServersRelayApiKey + { + "token": "740f...", ← APNs device token from step 3 + "topic": "com.continuum.app.ios", + "environment": "production", + "mode": "private_alert", + "server_device_id": "01JOPAQUE...", ← lets app find right local server + "delivery_id": "01JDELIVERY..." ← lets app fetch notification meta + } +7. Relay signs JWT with Silo's .p8, forwards to APNs: + POST https://api.push.apple.com/3/device/740f... + apns-topic: com.continuum.app.ios + authorization: bearer + {generic opaque payload with silo.* fields} +8. Apple delivers to the iPhone holding token 740f... +9. App wakes, reads server_device_id, finds matching local server account, + GETs full notification content from the user's own server. +``` + +The relay knows: a relay API key was used (so: which self-hosted server installation), the server's egress IP address (inherent to any hosted relay — the server connects to it directly), an APNs device token, a timestamp, and opaque IDs. The relay does not know: which user, which profile, what notification content, what server URL, what library, or what media item. Even a fully compromised relay leaks only timing + device tokens + egress IPs, not notification content or identity. The egress IP is the closest thing to an identity leak in this design — for home hosting it identifies the household's connection — and is listed here so the privacy claims stay exhaustive rather than over-strong. + +## Threat model summary + +Detailed per-channel threat models live in each channel's spec. The cross-cutting summary: + +| Channel | Adversary | Worst case | +|---|---|---| +| In-app inbox + websocket | Network attacker | Existing Silo auth threat model. Notifications add no new surface. | +| APNs / FCM relay (operator) | Compromised relay or hostile relay operator | Can learn device tokens, push timing, and the server's egress IP for opted-in servers. Cannot learn user identity, notification content, server URL, or library data. Cannot fabricate meaningful pushes (only generic opaque payloads). | +| APNs / FCM (Apple / Google) | Apple / Google as platform operator | Can see app topic, device token, generic payload, timing. No user identity, no media metadata, no server URL. | +| Outbound webhooks | Webhook destination operator (e.g., Discord) | Sees full notification content (titles, posters, episode info) for the profile that configured the webhook. The profile chose this destination; this is the explicit trade. | +| Outbound webhooks | Network attacker who owns a URL submitted by the user | Can receive the profile's notifications until the webhook is removed. Mitigated by HMAC for generic webhooks, HTTPS-only, host blocklist (no localhost / RFC1918 by default), and a profile-visible "last seen" / failure status. | + +## Channel dispatcher boundary + +A single internal `Dispatcher` interface fans the durable row out to channels. Each channel implements the same minimal contract: + +```go +type Dispatcher interface { + // Called once per notification_deliveries row, AFTER the row commits. + // Per-target dispatchers (push, webhooks) do not decide their own work: + // the fanout transaction already enqueued one `pending` attempt row per + // (delivery_id, target_id), where target_id is push_device.id or + // notification_webhook.id. Dispatch claims and sends those rows; a + // recovery worker sweeps `pending` rows whose claim never happened + // (crash between commit and dispatch). Idempotency is therefore + // by (delivery_id, target_id, attempt_number). The websocket dispatcher + // has no per-target row and is instead idempotent by delivery_id alone — + // re-publishing the same delivery_id is a no-op for connected clients. + Dispatch(ctx context.Context, delivery NotificationDelivery) error +} +``` + +Implementations: + +- `WebsocketDispatcher` — publishes a `notification.created` event on the `notifications` channel, scoped to the delivery's `(user_id, profile_id)`. Best-effort; durable row is the source of truth. +- `ApplePushDispatcher` — selects enabled `push_devices` for the profile where `platform = 'apple'` and routes through the configured provider (`silo_relay` or `custom_apns`). Records each attempt in `push_delivery_attempts`. +- `AndroidPushDispatcher` — same but `platform = 'android'`, routes through `silo_relay` or `custom_fcm`. +- `WebhookDispatcher` — selects enabled `notification_webhooks` for the profile, applies per-webhook reason filters, and POSTs the channel-specific payload (Discord embed or generic JSON+HMAC). Records each attempt in `webhook_delivery_attempts`. + +Channel failures are isolated: if APNs is down, webhooks and websocket still fire. If a webhook destination is unreachable, push still fires. The fanout worker does **not** wait for any dispatcher to succeed before marking the release event processed; the durable delivery rows plus the `pending` outbox attempt rows (enqueued in the same transaction) guarantee eventual delivery for push and webhooks even across a crash. The relay-bound dispatchers additionally pace their sends client-side (token bucket) so a burst never trips the relay's rate limits — see `02`/`03` "Relay pacing". + +## Capability surface + +The frontend and mobile clients need to know what's available without guessing. A single capability endpoint returns the truth: + +```http +GET /api/v1/notifications/capability +``` + +```json +{ + "in_app": { "enabled": true }, + "apple_push": { + "available": true, + "provider": "silo_relay", + "supported_modes": ["private_push", "in_app_only"] + }, + "android_push": { + "available": false, + "provider": "off", + "supported_modes": ["in_app_only"] + }, + "webhooks": { + "available": true, + "max_per_profile": 10, + "supported_types": ["discord", "generic"] + } +} +``` + +Clients render setup UI from this response. They never have to introspect admin settings. + +## Data model overview + +The full schemas live in the per-channel specs. The shared shape: + +| Table | Purpose | Spec | +|---|---|---| +| `episode_availability` | "Episode E became available in library L." Idempotent. Seeded silently for back-catalog. | 01 | +| `release_events` | One per `(library_id, episode_id)` newly-available event. | 01 | +| `notification_library_seed_state` | Per-library marker: availability has been seeded; release events may now be emitted. | 01 | +| `profile_series_interest` | Compact recipient index keyed by `(profile_id, library_id, series_id)`. | 01 | +| `notification_deliveries` | Per-profile durable inbox row. Cross-library dedupe via partial unique `(profile_id, episode_id)`. | 01 | +| `notification_preferences` | Per-profile reason toggles + master enable. | 01 | +| `push_devices` | Per-profile-per-device push registration. Platform-tagged (`apple` / `android`). | 02 / 03 | +| `push_delivery_attempts` | Per-attempt push log. No notification content. | 02 / 03 | +| `notification_webhooks` | Per-profile webhook destinations (Discord or generic). | 04 | +| `webhook_delivery_attempts` | Per-attempt webhook log. | 04 | + +A single migration block lands the foundation tables. APNs/FCM/webhook tables can land in separate migrations as those channels ship — they don't block the v1 in-app inbox. + +## API surface overview + +Profile-scoped (require `X-Profile-Id` header): + +- `GET /api/v1/notifications` — paginated inbox (newest-first, `before` cursor). +- `GET /api/v1/notifications/sync` — forward sync (`since` cursor); the endpoint mobile clients call after a push wake. +- `GET /api/v1/notifications/{id}` — single delivery by ID. +- `GET /api/v1/notifications/unread-count` — for the badge. +- `POST /api/v1/notifications/{id}/read` — mark single read. +- `POST /api/v1/notifications/read-all` — mark all read. +- `POST /api/v1/events/ws-ticket` — mint a short-lived single-use websocket ticket bound to `(user_id, profile_id)`. +- `GET /api/v1/notifications/preferences` — read profile prefs. +- `PUT /api/v1/notifications/preferences` — update profile prefs. +- `GET /api/v1/notifications/capability` — what channels are available. +- `POST /api/v1/devices/push/apple` — register an APNs device. +- `POST /api/v1/devices/push/fcm` — register an FCM device. +- `DELETE /api/v1/devices/push/{id}` — disable a push device. +- `GET /api/v1/notifications/webhooks` — list profile's webhooks. +- `POST /api/v1/notifications/webhooks` — create a webhook. +- `PUT /api/v1/notifications/webhooks/{id}` — update a webhook. +- `DELETE /api/v1/notifications/webhooks/{id}` — delete a webhook. +- `POST /api/v1/notifications/webhooks/{id}/test` — fire a test event. + +Admin-scoped: + +- Settings registry entries for APNs and FCM provider configuration (admin-only via existing `internal/api/handlers/settings.go` patterns). + +## Realtime channel + +A new event channel `notifications` joins the existing `catalog`, `jobs`, `sessions`, `tasks`, `scans`, `history_import`, `user_state`, `plugins` channels in `internal/events/types.go`. The websocket handler must validate active profile identity and must reject `notifications` events whose `profile_id` doesn't match the connection's bound profile. + +Because browsers can't set custom headers on WebSocket connections, profile identity is carried by a **short-lived single-use ticket**: the client calls `POST /api/v1/events/ws-ticket` (normal auth + `X-Profile-Id`), receives an opaque ticket valid for ~30 seconds, and passes it as a `ticket` query parameter on the handshake. The server consumes the ticket and binds the connection to the `(user_id, profile_id)` it was minted for. A long-lived profile token must **not** be passed in the query string — self-hosted servers commonly sit behind reverse proxies whose access logs capture query strings, and a logged ticket that expired 30 seconds after minting is harmless where a logged profile token is not. + +Event types on this channel: + +- `notification.created` — a new delivery for the bound profile. +- `notification.read` — a delivery was marked read (allows multi-tab coherence). + +Snapshot on subscribe: recent unread deliveries for the bound profile, so reconnecting clients hydrate without a separate REST call. + +## Out of scope for v1 + +- Movie release notifications (no trigger surface; v2). +- Request-fulfilled notifications ("your requested item is now available"). Silo's requests system postdates this design; it is the most obvious next notification type and slots into the extensible `notification_deliveries.type` registry without schema changes. v2. +- Aggregated notifications ("3 new episodes available"; presentation layer, can stack on per-episode rows later). The per-series burst cap in `01` bounds bulk-addition volume until an aggregated delivery type exists. +- Web Push (browser PWA notifications via Push API). Possible v2 — would mirror APNs/FCM as a third push platform. +- Email or SMS delivery. +- Cross-profile / household-wide notification views. +- Admin-level system notifications ("scan complete", "library health alert"). The existing realtime events hub (`internal/events/`, with the operational publisher wrapper at `internal/notifications/hub.go`) already handles operational events; this folder's design is strictly user-facing media notifications. +- Notification templates / customization beyond Discord-vs-generic. +- Time-window quiet hours. Could be added to `notification_preferences` later; not v1. +- Locale negotiation for push payloads. APNs / FCM payloads use generic localization keys (`SILO_NOTIFICATION_GENERIC_BODY` etc.) so the OS picks language from device locale. + +## Open questions + +These are the few remaining items not yet decided. None block writing the per-channel specs. + +1. **Service-level rate limits per relay account.** Per-second and per-day caps need real numbers. Suggest 10 req/sec burst, 50,000 req/day initial quota; tune from logs. The server-side dispatchers pace their relay calls client-side (default 5 req/sec token bucket, see `02`/`03`) so a hundreds-of-users server drains bursts smoothly instead of slamming into 429s; the per-series burst cap in `01` bounds worst-case burst size. +2. **Webhook destination domain blocklist edge cases.** RFC ranges are defined in `04-outbound-webhooks.md`; remaining open question is whether IPv6 v4-mapped addresses (`::ffff:0:0/96`) and CGNAT (`100.64.0.0/10`) need any admin escape hatch beyond the global `allow_private_destinations` flag. +3. **Discord image proxy timing.** Ship in v1 (build `media.discord-cdn-proxy.silo.app`) or defer to v1.5 with text-only embeds? Recommendation in `04`: defer to v1.5; v1 ships text-only embeds. + +Resolved during review (no longer open): + +- **Relay service code lives in a separate repo** (provisional `silo-push-relay`). This folder defines the contract. +- **Migration ordering.** Foundation schema lands as one migration; per-channel schemas can land separately as their channels ship. +- **`server_device_id` lifecycle.** Stable across token rotation; rotates only when a device is removed and re-registered. + +## Phasing + +Implementation phases gated by feature flags: + +| Phase | Flag | Capability | +|---|---|---| +| 1 | `notifications.schema_enabled` | Tables exist; no writes. | +| 2 | `notifications.interest_updaters_enabled` | `profile_series_interest` updates land on favorites/watchlist/playback events. Backfill task runs. | +| 3 | `notifications.release_events_enabled` | Availability seeding backfill runs first, then ingest writes `release_events`. No fanout yet. | +| 4 | `notifications.fanout_enabled` | Fanout worker runs; `notification_deliveries` rows materialize. Websocket channel published. | +| 5 | `notifications.ui_enabled` | Inbox API + frontend badge + page exposed. | +| 6 | `notifications.webhooks_enabled` | Outbound webhooks + Discord/generic dispatchers. v1 ships **text-only** Discord embeds (no `image` or `avatar_url` fields) to avoid leaking the user's server origin to Discord. v1.5 adds the optional `media.discord-cdn-proxy.silo.app` proxy and re-enables embed images. | +| 7 | `notifications.apple_push_enabled` | APNs registration + dispatcher. Requires relay infra (or admin-supplied custom credentials). | +| 8 | `notifications.android_push_enabled` | FCM registration + dispatcher. Requires relay infra (or admin-supplied custom credentials). | + +Phases 1-6 ship without any new external infrastructure. Phases 7-8 require Silo to provision Apple Developer + Firebase developer accounts and operate the relay service. diff --git a/docs/superpowers/plans/notifications/01-release-events-and-inbox.md b/docs/superpowers/plans/notifications/01-release-events-and-inbox.md new file mode 100644 index 00000000..620c00f4 --- /dev/null +++ b/docs/superpowers/plans/notifications/01-release-events-and-inbox.md @@ -0,0 +1,1022 @@ +# Profile-Scoped Release Notifications Implementation Plan + +**Date:** 2026-04-09 (refined 2026-04-28, amended 2026-06-11) +**Status:** Draft (refined for multi-channel delivery; amended for back-catalog seeding, burst suppression, cross-library dedupe, dispatch outbox, and forward-sync API) +**Scope:** Foundational durable inbox + realtime websocket fanout for episode-availability notifications. This is the substrate every other channel sits on. +**Companion docs:** +- [`00-architecture-overview.md`](./00-architecture-overview.md) — read first for cross-cutting context +- [`02-apns-relay.md`](./02-apns-relay.md) — Apple push, sits on top of this foundation +- [`03-fcm-relay.md`](./03-fcm-relay.md) — Android push, sits on top of this foundation +- [`04-outbound-webhooks.md`](./04-outbound-webhooks.md) — Discord/generic webhooks, sits on top of this foundation + +> **For agentic workers:** REQUIRED SUB-SKILL: Use `executing-plans` to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add scalable, profile-scoped notifications for newly available episodes when the series is relevant to that profile through `next up`, `continue watching`, `favorites`, or `watchlist`. Notifications must be durable, resumable, and delivered through Silo's own inbox and websocket infrastructure first; mobile push and webhooks layer on top via the channel dispatcher boundary defined in [`00-architecture-overview.md`](./00-architecture-overview.md). + +**Architecture:** Treat episode availability as a series-scoped release event, not a direct user notification. When ingest makes an episode newly available inside a library, persist a single durable `release_event` for that `(library_id, series_id, episode_id)` tuple. Maintain a compact `profile_series_interest` index keyed by `(profile_id, library_id, series_id)` that records why a profile cares about a series and what episode it expects next. A background fanout worker consumes `release_events`, resolves eligible recipients through that index, inserts deduplicated `notification_deliveries`, and then hands each delivery to the channel dispatcher (websocket, APNs, FCM, webhooks). The inbox API and frontend badge read from durable deliveries, so every other channel is an accelerator rather than the source of truth. + +**Tech Stack:** Go 1.26, PostgreSQL migrations, existing ingest/scanner pipeline, existing realtime events websocket, React + TanStack Query frontend + +--- + +## Scope and Constraints + +- Notify only for episodic releases in v1. +- A release means "episode became available in-library", not merely "metadata says it airs today". +- Delivery unit is the profile, not the user. +- Scaling target is driven by fanout per popular series, not total catalog size. The design must remain efficient with roughly 1 million episodes, 20,000 series, and hundreds of users (roughly 1,000 profiles) on a single server — a popular series may have several hundred interested profiles per release event. +- Do not run personalized `next up` reads at fanout time. +- Do not block scan/ingest completion on notification recipient resolution or delivery. +- Do not add background mobile push in v1. + +## Success Criteria + +- One newly available episode generates exactly one durable `release_event`. +- Fanout work is proportional to "profiles interested in this series in this library". +- Re-scans, retries, and metadata refreshes do not duplicate deliveries. +- Connected clients receive live updates via websocket. +- Disconnected clients see unread notifications in an inbox on reconnect. +- The system can backfill profile interest state and repair missed fanout work without manual data cleanup. + +## Out of Scope + +- Vendor push services, APNs, Firebase, or browser push subscriptions. +- Generic "library changed" notifications. +- Movie release notifications. +- Immediate aggregation like "3 new episodes available" in v1. Store per-episode deliveries first; aggregate presentation can come later. + +--- + +## Data Model + +### New Tables + +- `episode_availability` + - Purpose: durable record that an episode is available in a given library. Rows are inserted both by live ingest (which also emits release events) and by **seeding** (initial library scans and the feature-enable backfill, which emit no events — see "Seeding and Burst Suppression"). + - Note: `episode_id` and `series_id` are catalog-level content IDs (`media_items.content_id`); library membership is the `media_item_libraries` junction. The same episode in two libraries shares one `episode_id`. + - Columns: + - `library_id integer not null` + - `episode_id text not null` + - `series_id text not null` + - `season_number integer not null` + - `episode_number integer not null` + - `episode_key integer not null` + - `available_at timestamptz not null default now()` + - `created_at timestamptz not null default now()` + - Constraints: + - primary or unique key on `(library_id, episode_id)` + - Indexes: + - `(library_id, series_id, episode_key desc)` + +- `notification_library_seed_state` + - Purpose: per-library marker that availability seeding has completed for the library. Release events are emitted **only** for libraries with a `seeded_at` value; until then, availability inserts are silent. This is what makes "newly available" mean *newly released to this server* instead of *newly seen by the notifications feature*. + - Columns: + - `library_id integer primary key references media_folders(id) on delete cascade` + - `seeded_at timestamptz not null default now()` + +- `release_events` + - Purpose: one logical release emitted when an episode first becomes available in a library. + - Columns: + - `id text not null` + - `library_id integer not null` + - `series_id text not null` + - `episode_id text not null` + - `season_number integer not null` + - `episode_number integer not null` + - `episode_key integer not null` + - `available_at timestamptz not null` + - `dedupe_key text not null` — composed as `{library_id}:{episode_id}`. Exists as an explicit column (rather than relying on a composite unique) so future event kinds can share the table with their own key shapes. + - `processed_at timestamptz` + - `suppressed_reason text` — null for fanned-out events; `'series_burst'` when the per-series burst cap consumed this event without fanout (see "Seeding and Burst Suppression"). + - `created_at timestamptz not null default now()` + - Constraints: + - unique on `dedupe_key` + - Indexes: + - `(processed_at, created_at)` + - `(library_id, series_id, created_at desc)` + +- `profile_series_interest` + - Purpose: compact recipient index used for fanout. + - Columns: + - `user_id integer not null` + - `profile_id text not null references user_profiles(id) on delete cascade` + - `library_id integer not null` + - `series_id text not null` + - `favorite boolean not null default false` + - `watchlist boolean not null default false` + - `continue_watching boolean not null default false` + - `next_up_candidate boolean not null default false` + - `last_completed_episode_key integer` + - `next_expected_episode_key integer` + - `last_notified_episode_key integer` + - `updated_at timestamptz not null default now()` + - Constraints: + - primary key on `(profile_id, library_id, series_id)` + - Indexes: + - `(library_id, series_id)` + - partial index on `(library_id, series_id)` where `favorite OR watchlist OR continue_watching OR next_up_candidate` + - `(profile_id, updated_at desc)` + +- `notification_deliveries` + - Purpose: durable inbox rows for profiles. + - Columns: + - `id text not null` + - `release_event_id text references release_events(id) on delete set null` — nullable: operational types like `webhook.auto_disabled` have no release event, and retention pruning of old `release_events` must not delete inbox rows. + - `user_id integer not null` + - `profile_id text not null references user_profiles(id) on delete cascade` + - `library_id integer` — nullable; populated for `episode.available`, null for operational types. + - `series_id text` — same nullability rule. + - `episode_id text` — same nullability rule. + - `type text not null` + - v1 known values: `episode.available` (the primary case), `webhook.auto_disabled` (operational notice posted by the webhook channel when it auto-disables a profile-owned webhook — see [`04-outbound-webhooks.md`](./04-outbound-webhooks.md)) + - extensible; new types may land in future versions. Frontend must render unknown types with a generic fallback. + - `reason_flags jsonb not null` + - For `episode.available`: keys are the four reason booleans. For `webhook.auto_disabled`: shape is `{"webhook_id": "01J...", "webhook_name": "Family Discord", "last_failure_status": 404}` — never carries reason booleans. + - `status text not null default 'delivered'` + - `read_at timestamptz` + - `delivered_at timestamptz` + - `created_at timestamptz not null default now()` + - Constraints: + - `CHECK (type <> 'episode.available' OR (release_event_id IS NOT NULL AND library_id IS NOT NULL AND series_id IS NOT NULL AND episode_id IS NOT NULL))` + - partial unique on `(profile_id, release_event_id)` where `release_event_id IS NOT NULL` + - partial unique on `(profile_id, episode_id)` where `type = 'episode.available'` — **cross-library dedupe**: the same episode landing in two libraries (e.g., "TV" and "TV 4K") produces two release events but at most one delivery per profile; the first event processed wins and later inserts no-op via `ON CONFLICT DO NOTHING`. + - Indexes: + - `(profile_id, created_at desc)` + - `(profile_id, read_at, created_at desc)` + - `(status, created_at)` + - `(created_at, id)` — supports the forward-sync cursor. + +- `notification_preferences` + - Purpose: per-profile notification controls. + - Columns: + - `profile_id text not null references user_profiles(id) on delete cascade` + - `enabled boolean not null default true` + - `notify_favorites boolean not null default true` + - `notify_watchlist boolean not null default true` + - `notify_continue_watching boolean not null default true` + - `notify_next_up boolean not null default true` + - `updated_at timestamptz not null default now()` + - Constraints: + - primary key on `profile_id` + +### Shared Helpers + +- Add a small shared helper for `episode_key = season_number * 1000000 + episode_number`. The 1,000,000 multiplier comfortably accommodates real-world worst cases: long-running daily soaps and absolute-numbered anime (One Piece's catalog representation can exceed 10,000 episodes in a single "season 1" when scanners flatten absolute numbering, exceeding a `* 10000` multiplier). The combined value still fits comfortably within PostgreSQL `integer` (up to 2,147,483,647) for any realistic season number. Scanners must not emit `episode_number >= 1_000_000`; document as an invariant and reject in catalog ingest. +- Use this helper everywhere that stores or compares episode progression and release state. + +--- + +## Implementation Surface + +### Backend Modules Likely to Change + +- `internal/libraryingest/executor.go` +- `internal/scanner/scanner.go` +- `internal/api/handlers/favorites.go` (this file owns `PersonalDataHandler` which serves both favorites and watchlist routes — `HandleAddFavorite`, `HandleAddToWatchlist`, etc.) +- `internal/watchstate/service.go` (entry points: `SetFavorite`, `ToggleFavorite`, `RecordPlaybackStop`, `RecordImportedWatch`, `RecordImportedHistory`, `RecordJellycompatMarkPlayed`) +- `internal/api/handlers/progress.go` (the sync endpoint writes to the user store directly and must also trigger interest updaters — see Task 5) +- `internal/events/types.go` (extend `AllChannels` with `ChannelNotifications`) +- `internal/api/handlers/events_ws.go` (handshake must validate active profile via a short-lived ticket; see Task 8) +- `internal/api/router.go` +- `web/src/components/RealtimeEventsProvider.tsx` +- new package under `internal/notifications` (preferred — there is already a stub package at `internal/notifications/hub.go` for the catalog/jobs realtime hub; the user-notification system is a sibling concern in the same package directory, but the existing `Hub` type is operational-events-only and should not be conflated) +- `internal/taskmanager/tasks/...` +- new migrations under `migrations/` + +### New Backend Ownership Boundaries + +- `internal/notifications` (existing package — extends with new files) + - availability repo + - release event repo + - profile interest repo + - fanout worker + - notification delivery repo + - eligibility logic + - channel dispatcher interface and implementations (see "Channel Dispatcher Boundary" below) + - websocket dispatcher implementation + - per-channel dispatcher implementations land in subsequent specs (`02-apns-relay.md`, `03-fcm-relay.md`, `04-outbound-webhooks.md`) + +- `internal/api/handlers/notifications.go` + - inbox, unread count, preferences APIs + - capability endpoint for clients + - webhook CRUD (delegated handler under `internal/api/handlers/notifications_webhooks.go` per `04-outbound-webhooks.md`) + +> **Note on the existing `internal/notifications/hub.go`:** The current `notifications.Hub` type is a thin wrapper around `internal/events/Hub` for publishing operational events on `ChannelCatalog` and `ChannelJobs` (library-changed, metadata-updated, job lifecycle). It is unrelated to the user-facing notification system designed here. Do not conflate the two. The user-notification fanout publishes on the new `ChannelNotifications` and uses its own publishers; the existing operational hub continues to own catalog/jobs realtime. + +### Channel Dispatcher Boundary + +The fanout worker creates `notification_deliveries` rows. After each row commits, it hands the delivery to a `Dispatcher`, which fans the delivery out to channels: + +```go +type Dispatcher interface { + // Called once per notification_deliveries row, AFTER the row commits. + // Per-target dispatchers (push, webhooks) do not decide their own work: + // the fanout transaction already enqueued one `pending` attempt row per + // (delivery_id, target_id). Dispatch claims and sends those rows; a + // recovery worker sweeps `pending` rows whose claim never happened. + // Idempotency is by (delivery_id, target_id, attempt_number). The + // websocket dispatcher has no per-target row and is idempotent by + // delivery_id alone. + Dispatch(ctx context.Context, delivery NotificationDelivery) error +} +``` + +V1 implementations: + +- `WebsocketDispatcher` — publishes a `notification.created` event on `ChannelNotifications` scoped to `(user_id, profile_id)`. Best-effort; the durable row is the source of truth. +- `ApplePushDispatcher` — see [`02-apns-relay.md`](./02-apns-relay.md). +- `AndroidPushDispatcher` — see [`03-fcm-relay.md`](./03-fcm-relay.md). +- `WebhookDispatcher` — see [`04-outbound-webhooks.md`](./04-outbound-webhooks.md). + +A composite `MultiDispatcher` runs all configured dispatchers in parallel with bounded concurrency. Channel failures are isolated: a downed APNs path does not block the websocket or webhook channels. + +--- + +## Canonical Semantics + +These are the exact behavioral rules the implementation should follow. + +### What Counts as a Release + +- A release is emitted only when an episode becomes newly available inside a specific library **that has completed availability seeding** (`notification_library_seed_state.seeded_at` is set — see "Seeding and Burst Suppression" below). +- "Available" means: + - the episode is resolved to a concrete episode record + - the episode has at least one playable non-missing file in that library + - the `(library_id, episode_id)` pair did not previously exist in `episode_availability` +- Metadata-only changes do not create a release. +- Repaired probes, poster refreshes, and title edits do not create a release. +- Removing an episode's file and later re-adding it (quality upgrades, re-downloads) does not re-notify: the `episode_availability` row persists across file churn. This is intentional — availability is a one-way "first released here" fact. +- If the same episode is available in two libraries, that is two distinct availability facts and can yield two distinct `release_events`. Delivery-level dedupe (below) ensures a profile is still notified at most once per episode. + +### Seeding and Burst Suppression + +The flood problem: without these rules, the first scan of a new library — or adding a 200-episode back-catalog of a favorited series — makes every episode "newly available" at once. On a server with hundreds of users, one bulk import could generate tens of thousands of deliveries and pushes in a single scan. "Newly available" must mean *newly released to this server*, not *newly seen by the notifications feature*. + +**Seeding rules (no release events):** + +- **Feature-enable seeding.** Before `notifications.release_events_enabled` may be turned on, a one-time seeding task inserts `episode_availability` rows for every currently playable episode in every library, without creating release events, and writes `notification_library_seed_state` for each library. This is a rollout prerequisite, like the interest backfill. +- **New-library seeding.** A library with no `notification_library_seed_state` row is unseeded: availability inserts during its scans create no release events. When the library's first full scan completes successfully, the scanner writes the seed marker. Subsequent scans emit release events normally. +- Seeding is idempotent (`ON CONFLICT DO NOTHING` on `episode_availability`; upsert on the seed marker). + +**Burst suppression (bounded fanout for bulk additions to seeded libraries):** + +Bulk additions to an *existing* library (a back-catalog season pack, a batch import) legitimately create many release events. The fanout worker bounds the per-profile blast radius: + +- **Settling delay.** The worker claims only release events with `created_at <= now() - settle_seconds` (default `30`, setting `notifications.fanout.settle_seconds`) so one scan's burst for a series lands in the same claim batch instead of trickling through several. +- **Per-series burst cap.** Within a claim batch, events are grouped by `(library_id, series_id)`. If a group exceeds `notifications.fanout.max_series_burst` (default `3`), the worker fans out only the `max_series_burst` events with the highest `episode_key` and marks the rest `processed_at = now(), suppressed_reason = 'series_burst'` — no deliveries, no pushes, no webhooks for the suppressed events. +- The cap is per claim batch and therefore approximate across batches; that is acceptable. A v2 aggregated delivery type ("12 episodes of X are now available") can replace suppression with a summary row without schema changes. +- Suppressed events do not update `last_notified_episode_key`; the fanned-out events do, and the guarded max-wins update makes ordering irrelevant. +- Suppression counts are logged and exported (`release_events_suppressed_total`) so admins can see what was withheld; silent truncation is not acceptable. + +### What Counts as an Interested Profile + +A profile is a candidate recipient only if all of the following are true: + +- the profile belongs to the authenticated `user_id` +- the profile can access the `library_id` where the episode became available +- the profile has a `profile_series_interest` row for the same `(library_id, series_id)` +- at least one of the interest flags is enabled and relevant + +For v1, "can access the library" should be determined from profile restrictions and the same library visibility rules already used in request-time access filtering. The index must never assume global library visibility. + +### When Each Interest Flag Should Be Set + +- `favorite` + - set when a series is explicitly favorited + - cleared when the favorite is removed +- `watchlist` + - set when a series is explicitly watchlisted + - cleared when the watchlist entry is removed +- `continue_watching` + - set when the profile has in-progress episode progress for the series + - cleared when no qualifying in-progress state remains +- `next_up_candidate` + - set when the profile has progression state that should receive future-episode notifications + - paired with `next_expected_episode_key` + - cleared when the series is no longer eligible for next-up style notifications + +### Notification Deduplication Rules + +- One `release_event` per logical `(library_id, episode_id)` availability. +- One `notification_delivery` per `(profile_id, release_event_id)`. +- **At most one `episode.available` delivery per `(profile_id, episode_id)` across libraries.** Media items are catalog-level, so the same episode in "TV" and "TV 4K" shares one `episode_id`; the partial unique index makes the first-processed release event win and later ones no-op for that profile. Dual-quality library setups must not double-notify. +- If multiple interest reasons apply, store one delivery with merged `reason_flags`. +- Reprocessing a release event must be safe and produce zero duplicates. + +### Profile Scope Rules + +- Inbox APIs are scoped to the active `X-Profile-Id`. +- Realtime notifications must carry both `user_id` and `profile_id`. +- The websocket layer must reject events whose `profile_id` does not match the request context profile. +- Admin users do not get a cross-profile notifications view in v1 unless explicitly added later. +- Because browser websocket connections cannot set arbitrary request headers in the normal `WebSocket` API, the websocket handshake must carry profile identity explicitly — via a **short-lived single-use ticket** minted over authenticated REST (see "Websocket Handshake Contract" below). Long-lived profile tokens must not appear in the handshake query string: self-hosted servers commonly sit behind reverse proxies whose access logs capture query strings. + +--- + +## End-to-End Lifecycle + +1. Ingest makes an episode newly available in a library. +2. The backend inserts a new `episode_availability` row. +3. That insert produces one durable `release_event` (only for seeded libraries — see "Seeding and Burst Suppression"). +4. A fanout worker claims unprocessed `release_events` past the settling delay and applies the per-series burst cap. +5. The worker loads candidate recipients from `profile_series_interest`. +6. The worker filters recipients using preferences and progression rules. +7. The worker bulk inserts `notification_deliveries` **and the `pending` per-target outbox attempt rows for the push and webhook channels** in the same transaction. +8. The worker marks the `release_event` processed. +9. The worker publishes realtime events for inserted deliveries and triggers the per-channel dispatchers, which claim their pending attempt rows. +10. Connected clients update immediately; disconnected clients see the inbox snapshot later. If the process crashes between steps 8 and 9, recovery workers drain the pending attempt rows. + +No step after `release_event` creation is allowed to run inline on the ingest request path. + +--- + +## Transaction and Concurrency Rules + +### Availability and Event Creation + +- Availability detection and `release_event` creation should happen in a short transaction where practical. +- Use `INSERT ... ON CONFLICT DO NOTHING RETURNING ...` patterns to detect new rows without race-prone pre-checks. +- Prefer deriving "newly available episodes" from touched content IDs or touched libraries inside the current ingest scope. +- Do not rescan the entire library to decide what changed. + +### Fanout Worker Claiming + +- Process `release_events` in batches. +- Claim events with a concurrency-safe pattern such as: + - `SELECT ... FOR UPDATE SKIP LOCKED` + - then mark a claim timestamp or process inside the transaction +- Multiple Silo nodes must be able to run the worker without duplicate fanout. +- If the process crashes after inserting deliveries but before marking the event processed, reprocessing must be harmless because delivery inserts are idempotent. + +### Delivery Insert and State Update + +- The worker should: + 1. load candidate interest rows + 2. compute eligible recipients + 3. bulk insert deliveries with `INSERT ... ON CONFLICT DO NOTHING RETURNING id, profile_id` so the worker can distinguish newly inserted rows (publish realtime + per-channel dispatch) from deduped rows (no-op) + 4. **enqueue the dispatch outbox**: for each *newly inserted* delivery, insert `pending` attempt rows — one per enabled `push_device` of the recipient profile (`push_delivery_attempts`, see `02`/`03`) and one per enabled, reason-matching `notification_webhook` (`webhook_delivery_attempts`, see `04`). Skip channels whose feature flag is off or whose tables have not landed yet; the foundation defines the pattern, the channel specs own the tables. + 5. update `last_notified_episode_key` using a guarded `UPDATE profile_series_interest SET last_notified_episode_key = $newKey WHERE profile_id = $p AND library_id = $l AND series_id = $s AND (last_notified_episode_key IS NULL OR last_notified_episode_key < $newKey)`. The `< $newKey` guard prevents two concurrent workers handling adjacent release events from clobbering each other's update — the higher key always wins regardless of commit order. + 6. mark the `release_event` processed +- Steps 3-6 should happen in one transaction so the event does not get marked processed without durable deliveries **and** durable dispatch intent. This is the outbox invariant from `00`: a crash after commit delays pushes/webhooks (recovery workers sweep stale `pending` rows) instead of silently dropping them. Without step 4, "durable rows + retry tables guarantee eventual delivery" would be false — retry workers can only retry attempts that were recorded. +- Outbox sizing: pending rows are bounded by recipients × enabled targets per event, and the per-series burst cap bounds events per scan. Chunk the inserts like the delivery inserts. +- The `RETURNING` clause in step 3 is load-bearing for two invariants: "publish only inserted rows" and "no duplicate websocket event on rescan." The dispatcher and the outbox enqueue must operate on the returned set, not the candidate set. + +### Realtime Publishing + +- Publish websocket events only after the delivery transaction commits. +- If realtime publish fails, the durable delivery still exists and will appear on reconnect. + +--- + +## Exact Recipient Resolution Rules + +### Library Visibility + +- `profile_series_interest` is library-scoped on purpose. +- When building or updating interest rows, only create rows for libraries the profile can actually see. +- If profile library restrictions change, affected `profile_series_interest` rows must be rebuilt. + +### Series Resolution + +- Favorites/watchlist against series items map directly to `series_id = media_item_id`. +- Favorites/watchlist against episode or season items should resolve to their parent `series_id`. +- Movie targets are ignored in v1 and should not create `profile_series_interest`. + +### Progression Cursor Rules + +- `last_completed_episode_key` tracks the highest sequentially completed episode for the series as represented by the profile's watch state. +- `next_expected_episode_key` should be the next unwatched episode key after the profile's completed progression, not merely "highest seen + 1" if gaps exist. +- If implementation-time computation of true gaps is expensive, use a safe conservative value that may under-notify rather than over-notify. Document that tradeoff in code comments if used. +- `last_notified_episode_key` prevents repeated notifications for the same or older episodes. + +### Home Dismissals + +- Existing home-surface dismissal state should not suppress release notifications in v1. +- Dismissing `continue watching` or `next up` rows on the home screen does not mean "stop release notifications for this series". +- Notification preferences, not home-surface dismissals, are the suppression mechanism. + +--- + +## API Contract + +These shapes should be treated as part of the plan rather than decided later. + +### `GET /api/v1/notifications` + +Query params: + +- `status=all|unread` default `all` +- `limit` default `25`, max `100` +- `before` optional RFC3339 timestamp or opaque cursor, choose the format that best matches existing API style and keep it stable + +Response shape: + +```json +{ + "notifications": [ + { + "id": "01H...", + "type": "episode.available", + "profile_id": "profile-1", + "library_id": 7, + "series_id": "series-123", + "episode_id": "episode-456", + "series_title": "Severance", + "episode_title": "Hello, Ms. Cobel", + "season_number": 2, + "episode_number": 1, + "poster_path": "metadata/posters/...", + "poster_thumbhash": "....", + "reason_flags": { + "favorite": true, + "watchlist": false, + "continue_watching": true, + "next_up": true + }, + "created_at": "2026-04-09T12:34:56Z", + "read_at": null + } + ] +} +``` + +### `GET /api/v1/notifications/sync` + +Forward sync for clients that wake from a push or reconnect after an offline gap. This is the endpoint mobile clients call after an APNs/FCM wake (see `02`/`03` "Wake And Metadata Fetch") — push delivery is not guaranteed and multiple deliveries may have accumulated, so a cursor sync beats fetching one delivery by ID. + +Query params: + +- `since` optional opaque cursor from a previous response. Encodes `(created_at, id)` and pages **ascending** (oldest first), the opposite direction of the inbox list. Omitted: returns the most recent page and a cursor for subsequent calls. +- `limit` default `50`, max `100` + +Response shape: + +```json +{ + "notifications": [ /* same row shape as GET /api/v1/notifications */ ], + "next_cursor": "opaque", + "unread_count": 3 +} +``` + +- Rows include read state so a wake-sync can render accurately without a second call; `unread_count` is included for the same reason. +- Clients persist `next_cursor` per `(server, profile)` and pass it on the next wake. + +### `GET /api/v1/notifications/{id}` + +Returns a single delivery by ID, same row shape as the list API. Profile-scoped: returns `404` if the delivery belongs to another profile. Used when a push wake carries a specific `delivery_id` and the app deep-links to one notification. + +### `GET /api/v1/notifications/unread-count` + +Response shape: + +```json +{ + "count": 3 +} +``` + +Unread semantics are `read_at IS NULL` for the active profile. The `status` field is operational metadata and should not change unread-count behavior. + +### `POST /api/v1/notifications/{id}/read` + +- Marks a single notification as read for the active profile. +- Must be idempotent. +- Returns `204 No Content`. + +### `POST /api/v1/notifications/read-all` + +- Marks all notifications as read for the active profile. +- Returns `204 No Content`. + +### Realtime Event Payload + +Channel: `notifications` + +Event name: + +- `notification.created` +- optionally later `notification.read` + +Payload shape: + +```json +{ + "id": "01H...", + "profile_id": "profile-1", + "type": "episode.available", + "library_id": 7, + "series_id": "series-123", + "episode_id": "episode-456", + "series_title": "Severance", + "episode_title": "Hello, Ms. Cobel", + "season_number": 2, + "episode_number": 1, + "reason_flags": { + "favorite": true, + "watchlist": false, + "continue_watching": true, + "next_up": true + }, + "created_at": "2026-04-09T12:34:56Z", + "read_at": null +} +``` + +The notifications snapshot should return the same object shape as the list API for recent unread rows. + +### Websocket Handshake Contract + +Profile identity is carried by a short-lived single-use ticket, not by tokens in the query string (reverse proxies log query strings; a leaked ticket that expired 30 seconds after minting is harmless, a leaked profile token is not). + +- `POST /api/v1/events/ws-ticket` — normal auth + `X-Profile-Id`. Returns `{ "ticket": "opaque", "expires_in": 30 }`. The ticket is single-use, bound server-side to `(user_id, profile_id)`, and stored in memory (or Redis when multiple nodes serve websockets). +- The frontend websocket URL builder requests a ticket immediately before connecting and passes it as a `ticket` query parameter on `/events/ws`. +- The handshake consumes the ticket and binds the connection to the `(user_id, profile_id)` the ticket was minted for. +- If the ticket is missing, expired, already used, or invalid: + - reject the notifications channel subscription + - or fail the websocket handshake for profile-scoped usage +- On reconnect, the client mints a fresh ticket; tickets are cheap. +- Do not rely on `RequireProfile` middleware for `/events/ws` in the browser path. + +--- + +## Frontend Behavior Contract + +- The unread badge should reflect only the active profile. +- Switching profiles should clear notifications query cache and resubscribe under the new `X-Profile-Id`. +- On websocket connect: + - subscribe to `notifications` + - hydrate unread rows from the snapshot + - update unread count cache +- On `notification.created`: + - prepend to cached inbox list if present + - increment unread count if `read_at == null` + - show a toast only when the active profile matches +- On reading a notification: + - update cached row state + - decrement unread count without waiting for a refetch + +Do not piggyback on `catalog` invalidation for inbox behavior. Notifications need their own query keys and event handling. + +--- + +## Execution Plan + +### Task 1: Add Release and Notification Persistence Schema + +**Files:** +- Add: a timestamped Goose migration in `migrations/sql/` created with `make migrate-create NAME=profile_release_notifications` (single file with `-- +goose Up` / `-- +goose Down` sections; do **not** create paired `.up.sql` / `.down.sql` files — that convention predates this repo's migration runner). + +- [ ] Create `episode_availability`, `notification_library_seed_state`, `release_events`, `profile_series_interest`, `notification_deliveries`, and `notification_preferences`. +- [ ] Add unique constraints, partial unique indexes (cross-library dedupe), foreign keys, and indexes exactly as described above. +- [ ] Keep `notification_deliveries.reason_flags` as `jsonb` for merged-source recording without a schema churn loop. +- [ ] Ensure the `-- +goose Down` section cleanly removes the new tables and indexes in reverse order. + +### Task 2: Introduce a Release Package and Shared Episode Key Helper + +**Files:** +- Add: `internal/notifications/types.go` +- Add: `internal/notifications/episode_key.go` +- Add: `internal/notifications/repositories.go` + +- [ ] Define domain types for availability records, release events, profile interest, preferences, and deliveries. +- [ ] Add `EpisodeKey(seasonNumber, episodeNumber int) int`. +- [ ] Add repository methods for: + - recording episode availability + - inserting release events idempotently + - selecting unprocessed release events + - loading profile interest rows by `(library_id, series_id)` + - bulk inserting deliveries + - marking release events processed + - marking notifications read + - loading inbox pages and unread counts + - loading and upserting notification preferences + +### Task 3: Detect Newly Available Episodes During Ingest + +**Files:** +- Modify: `internal/scanner/scanner.go` +- Modify: `internal/libraryingest/executor.go` +- Add or modify: `internal/notifications/availability_detector.go` + +- [ ] Add a path that identifies which episode IDs became newly available in the touched ingest scope. +- [ ] Persist availability through `episode_availability` with `ON CONFLICT DO NOTHING`. +- [ ] Only create `release_events` for newly inserted availability rows **in libraries that have a `notification_library_seed_state` row**; unseeded libraries insert availability silently. +- [ ] Write the seed marker when a new library's first full scan completes successfully. +- [ ] Run this after matching/reconcile is complete, not on raw file discovery, so the release is tied to an actual resolved episode. +- [ ] Keep this write path lightweight and transactional where practical. +- [ ] Do not perform recipient lookup or websocket publishing here. +- [ ] Ensure the implementation works for: + - full-library ingest + - subtree ingest + - single-file ingest +- [ ] For file/subtree ingest, use touched content IDs rather than broad library reconciliation queries. +- [ ] Resolve the current ingest seam limitation explicitly: the existing `Matcher` interface only returns counts, not content IDs or episode IDs. Choose one implementation path before coding: + - widen the matcher contract to return touched content IDs or release candidates + - or add a post-match repository query that derives newly available episode IDs from the just-touched ingest scope without scanning the full library +- [ ] Prefer the post-match repository query if it keeps the matcher contract stable and remains scope-bounded. + +**Implementation note:** The current ingest publish seam in [`internal/libraryingest/executor.go`](../../../../internal/libraryingest/executor.go) is the right place to persist release candidates because it already represents a completed scan/match cycle. + +### Task 4: Maintain `profile_series_interest` from Favorites and Watchlist + +**Files:** +- Modify: `internal/api/handlers/favorites.go` +- Add: `internal/notifications/interest_updater.go` + +- [ ] When a profile favorites or watchlists a series, upsert `profile_series_interest` rows for every visible library membership of that series. +- [ ] When a profile removes a favorite or watchlist item, clear only that flag; do not delete the row if other interest flags remain. +- [ ] Ignore movie items in v1. +- [ ] Resolve "target item to series" for episode or season cases so the interest key stays series-centric. +- [ ] Keep the write best-effort but logged if it fails, similar to other auxiliary user-state updates. +- [ ] Respect profile library restrictions when creating library-scoped interest rows. +- [ ] Add a helper that recomputes a single `(profile, series)` interest row from source-of-truth state so repairs and live updates share code. + +### Task 5: Maintain `profile_series_interest` from Playback and History + +**Files:** +- Modify: `internal/watchstate/service.go` +- Possibly modify: `internal/api/handlers/playback.go` +- Modify: `internal/api/handlers/progress.go` +- Add or modify: `internal/notifications/interest_progress.go` + +- [ ] Trigger the updater on watch-state **transitions**, not every progress write: an episode entering in-progress state, crossing the completion threshold, or progress rows being deleted. Progress sync endpoints fire continuously during playback on a busy server (hundreds of concurrent streams); recomputing interest on every tick is a pointless hot write path. Compare the derived flags/cursors against the existing row (or debounce per `(profile, series)` with a short TTL) and skip no-op writes. +- [ ] On episode watch progress transitions, update `continue_watching` and `next_up_candidate`. +- [ ] On episode completion, update: + - `last_completed_episode_key` + - `next_expected_episode_key` + - `next_up_candidate` +- [ ] Clear `continue_watching` when the profile has no in-progress state left for the series. +- [ ] Keep these updates series-centric and library-aware. +- [ ] Do not query the full catalog at fanout time to derive these values later. +- [ ] Decide one implementation path and encode it in code: + - either fully recompute a profile-series row from source-of-truth progress on every mutation + - or apply an incremental updater with a shared repair path +- [ ] Prefer recompute-per-series on mutation if the series-local query cost is acceptable; it is simpler and less drift-prone than trying to patch every field incrementally. +- [ ] Cover every live progress mutation path, not just `watchstate.Service`. The current sync endpoint in `internal/api/handlers/progress.go` writes directly to the user store, so it must also trigger the shared profile-series updater or be refactored behind the same abstraction. + +**Decision:** `next_up_candidate` should mean "this profile is eligible for next-episode notifications on this series", while `next_expected_episode_key` is the precise cursor used to decide if a newly available episode should notify. + +### Task 6: Add Backfill / Repair Tasks for Interest State and Availability Seeding + +**Files:** +- Add: `internal/taskmanager/tasks/rebuild_release_interest.go` +- Add: `internal/taskmanager/tasks/seed_episode_availability.go` +- Modify: `internal/api/router.go` + +- [ ] Add the availability seeding task: insert `episode_availability` for every currently playable episode in every library (batched, `ON CONFLICT DO NOTHING`), write `notification_library_seed_state` per library, emit zero release events. Must complete before `notifications.release_events_enabled` is turned on. +- [ ] Add a hidden task that incrementally rebuilds `profile_series_interest` from existing favorites, watchlist, and watch progress data. +- [ ] Process in batches to avoid long-running transactions. +- [ ] Make the task rerunnable and idempotent. +- [ ] Wire it into the task manager similarly to existing scheduled tasks. +- [ ] Keep the initial rollout safe by allowing the task to run before fanout is enabled. +- [ ] Add an admin-invokable entry point if there is already a pattern for manually triggering tasks in this repo. + +### Task 7: Implement the Fanout Worker + +**Files:** +- Add: `internal/notifications/fanout_worker.go` +- Add: `internal/notifications/fanout_logic.go` +- Modify: `internal/api/router.go` +- Possibly modify: `cmd/silo/main.go` + +- [ ] Add a worker loop that loads unprocessed `release_events` in batches, honoring the settling delay (`created_at <= now() - settle_seconds`). +- [ ] Nudge the worker on insert instead of relying on tight polling: ingest publishes a wake signal (Postgres `LISTEN/NOTIFY` or a Redis pub — Redis is already a dependency) after writing release events; the worker also polls at a relaxed fallback interval (15-30s). The nudge schedules a claim at `settle_seconds` so notifications still feel near-realtime. +- [ ] Group claimed events by `(library_id, series_id)` and apply the per-series burst cap: fan out only the `max_series_burst` highest `episode_key` events per group; mark the rest processed with `suppressed_reason = 'series_burst'`. +- [ ] For each fanned-out event, load `profile_series_interest` rows by `(library_id, series_id)`. +- [ ] Apply eligibility rules: + - notify if `favorite` + - notify if `watchlist` + - notify if `continue_watching` + - notify if `next_up_candidate` and `episode_key >= next_expected_episode_key` + - suppress if `last_notified_episode_key >= episode_key` + - suppress if preferences disable all matching reasons +- [ ] Bulk insert `notification_deliveries` with `ON CONFLICT DO NOTHING` (both partial uniques participate: per-release-event and cross-library per-episode). +- [ ] Enqueue the dispatch outbox in the same transaction: `pending` attempt rows per enabled push device / reason-matching webhook for each newly inserted delivery (skip channels whose flags are off or whose tables haven't landed). +- [ ] Update `last_notified_episode_key` for rows that actually produced deliveries. +- [ ] Mark the `release_event` processed only after durable delivery and outbox rows are created. +- [ ] Keep worker throughput visible through structured logs and counters, including suppression counts. +- [ ] Use `FOR UPDATE SKIP LOCKED` or an equivalent claim pattern so multiple nodes can process safely. +- [ ] Publish realtime events only for deliveries inserted in the current transaction, not for deduped rows. +- [ ] Batch processing defaults: + - claim up to `100` release events at a time + - load recipients in memory per event + - bulk insert deliveries and outbox rows in chunks if recipient counts are large + +### Task 8: Add a Notifications Realtime Channel and Snapshot + +**Files:** +- Modify: `internal/events/types.go` +- Modify: `internal/api/handlers/events_ws.go` +- Add or modify: `internal/events/publishers.go` +- Modify: `web/src/components/RealtimeEventsProvider.tsx` +- Modify: `web/src/api/client.ts` or a nearby profile-token source if needed + +- [ ] Add `ChannelNotifications` to `internal/events/types.go`. Must be appended to the `AllChannels` slice (currently: catalog, jobs, sessions, tasks, scans, history_import, user_state, plugins) so subscription enumeration finds it. +- [ ] Update `allowedChannelsForRole` in `internal/api/handlers/events_ws.go` to permit authenticated users (and admins) to subscribe to `ChannelNotifications`. +- [ ] Update the snapshot switch in `events_ws.go` to return a real snapshot payload for `ChannelNotifications` (recent unread deliveries for the bound profile, capped at e.g. 25 rows, identical row shape to the inbox list API in Task 9). Not `null`. +- [ ] Publish a profile-scoped realtime event when new `notification_deliveries` are created. Include both `user_id` and `profile_id` on the event payload. +- [ ] Ensure the websocket filter continues to respect `UserID`, and add profile scoping for notifications. +- [ ] Add the `POST /api/v1/events/ws-ticket` endpoint (normal auth + `X-Profile-Id`) returning a ~30-second single-use ticket bound to `(user_id, profile_id)`, and extend the websocket handshake to consume the `ticket` query parameter and bind the connection. `/events/ws` is not currently using `RequireProfile` and the browser websocket path does not send `X-Profile-Id`; long-lived profile tokens must not be passed in the query string (see "Websocket Handshake Contract"). +- [ ] Store tickets in memory for single-node deployments; use Redis when multiple nodes serve websockets. +- [ ] Add explicit tests for profile mismatch rejection and expired/reused-ticket rejection. + +**Important safety note:** The existing websocket path filters on `UserID` but not generally on `ProfileID`. Notification events should include both, and the handler should reject deliveries for mismatched profiles. + +### Task 8a: Wire Worker Startup and Shutdown + +**Files:** +- Modify: `cmd/silo/main.go` +- Possibly modify: `internal/api/router.go` + +- [ ] Construct release repositories and services when DB access is available. +- [ ] Start the fanout worker under the main application context. +- [ ] Ensure graceful shutdown waits for in-flight worker loops to exit cleanly. +- [ ] Do not start the worker when the feature flag is disabled. +- [ ] Keep the long-running fanout worker separate from taskmanager scheduled tasks. Use taskmanager only for rebuild, repair, and cleanup passes. + +### Task 9: Add Inbox and Read APIs + +**Files:** +- Add: `internal/api/handlers/notifications.go` +- Modify: `internal/api/router.go` + +- [ ] Add: + - `GET /api/v1/notifications` + - `GET /api/v1/notifications/sync` (forward cursor; the mobile wake-fetch endpoint required by `02`/`03`) + - `GET /api/v1/notifications/{id}` + - `GET /api/v1/notifications/unread-count` + - `POST /api/v1/notifications/{id}/read` + - `POST /api/v1/notifications/read-all` +- [ ] Mount these routes behind the same auth + profile middleware pattern used by other profile-scoped endpoints so `X-Profile-Id` is mandatory. +- [ ] Return enough display data for the frontend to render a useful row without an extra lookup round trip: + - series title + - episode title if available + - poster/thumbhash + - season/episode numbers + - created timestamp + - reason flags +- [ ] Keep pagination cursor-based or limit/offset-based, whichever matches local patterns best. +- [ ] Add endpoints for preferences if they are needed to make the feature operable in v1: + - `GET /api/v1/notifications/preferences` + - `PUT /api/v1/notifications/preferences` + +### Task 9a: Add Notification Preferences API + +**Files:** +- Add or modify: `internal/api/handlers/notifications.go` +- Modify: `internal/api/router.go` + +- [ ] Expose profile-scoped preferences for enable/disable and per-reason toggles. +- [ ] Default missing rows to all-enabled behavior. +- [ ] Keep preference writes idempotent. + +### Task 10: Add Frontend Inbox, Badge, and Live Updates + +**Files:** +- Modify: `web/src/components/RealtimeEventsProvider.tsx` +- Add: `web/src/hooks/queries/notifications.ts` +- Add: `web/src/pages/Notifications.tsx` +- Modify app shell/sidebar files as needed + +- [ ] Add query keys and hooks for inbox pages and unread counts. +- [ ] Subscribe the frontend to the `notifications` channel. +- [ ] Hydrate snapshot state on connect and update unread counts on live events. +- [ ] Add a sidebar badge or header badge for unread count. +- [ ] Add a dedicated notifications page. +- [ ] Add a lightweight live toast for connected clients. +- [ ] Mark notifications read when opened or through explicit actions. +- [ ] Add profile-scoped preferences UI if the API is included in v1. +- [ ] Clear notification caches when auth state or active profile changes. + +### Task 11: Add Operational Repair Paths and Feature Flags + +**Files:** +- Add or modify: `internal/notifications/repair.go` +- Modify: config or settings files if runtime flags are preferred + +- [ ] Add a repair path that finds unprocessed `release_events` or events with missing deliveries. +- [ ] Add a rebuild path for corrupted or stale `profile_series_interest`. +- [ ] Gate rollout with feature flags or settings: + - `notifications.release_events_enabled` + - `notifications.fanout_enabled` + - `notifications.ui_enabled` + - `notifications.preferences_enabled` if preference UI/API is shipped separately +- [ ] Enable in stages: schema -> backfill -> event creation -> fanout -> UI. +- [ ] Add a retention policy decision and implement it: + - keep read notifications for a bounded window such as 90 days + - keep processed `release_events` for a bounded debugging window + - add cleanup tasks if retention is not indefinite + +--- + +## Eligibility Rules + +These are implementation decisions, not open questions. + +- `favorite`: notify whenever a newly available episode lands for a favorited series. +- `watchlist`: notify whenever a newly available episode lands for a watchlisted series. +- `continue_watching`: notify whenever a newly available episode lands for a series the profile is actively watching. +- `next_up`: notify only when the new episode is at or beyond the profile's `next_expected_episode_key`. +- Multiple reasons produce one delivery with merged `reason_flags`. +- Re-scans and repeated availability writes must not produce duplicate deliveries. + +--- + +## Scaling Notes + +- The hot fanout query must be `profile_series_interest WHERE library_id = ? AND series_id = ?`. +- Do not use the full `episodes` table in delivery-time recipient resolution. +- Do not execute [`internal/catalog/nextup_repo.go`](../../../../internal/catalog/nextup_repo.go) per recipient. +- The design should scale linearly with "interested profiles for this one series", which is acceptable even for popular shows. +- With roughly 20,000 series, series-scoped state remains compact and cache-friendly compared to episode-scoped per-profile tracking. +- Size expectations: + - `episode_availability` can grow with available episodic catalog and should be indexed narrowly + - `profile_series_interest` grows with engaged profile-series pairs, not total episodes + - `notification_deliveries` is the fastest-growing table and needs retention and pagination discipline +- Fanout throughput should be measured in: + - release events per minute + - recipients per event + - deliveries inserted per second +- The worker should remain correct if a hit show has very large recipient counts. Use batching rather than one enormous insert statement when needed. +- Hundreds-of-users arithmetic (sanity check, ~1,000 profiles): a popular release with 300 interested profiles inserts 300 delivery rows plus outbox rows — trivial for PostgreSQL. The dangerous shapes are bulk imports (bounded by seeding + the per-series burst cap) and relay throughput (bounded by client-side pacing in `02`/`03`). Steady state of ~100 new episodes/day × ~20 interested profiles each is ~2,000 deliveries/day; retention keeps `notification_deliveries` in the low millions worst case. + +--- + +## Rollout Plan + +1. Deploy schema only. +2. Deploy `profile_series_interest` updaters and the rebuild task with fanout disabled. +3. Run the interest backfill and verify row counts. +4. Run the availability seeding task and verify every library has a `notification_library_seed_state` row. +5. Enable `release_events` creation only. +6. Verify that new episodes create one durable event each and that back-catalog rescans create none. +7. Enable fanout worker. +8. Verify durable deliveries without frontend changes. +9. Enable websocket channel and inbox UI. +10. Enable preferences API/UI if included. +11. Monitor metrics and repair task output before widening rollout. + +--- + +## Backfill Strategy + +The backfill is part of the implementation, not an optional follow-up. + +### Initial Backfill Sources + +- `user_favorites` +- `user_watchlist` +- `user_watch_progress` +- enough series metadata to resolve episode progress rows back to `series_id` +- profile library restriction settings + +### Backfill Algorithm + +- Iterate profiles in batches, ordered by `profile_id` (any stable ordering works; profile_id is convenient). +- Persist a checkpoint row keyed by task name (e.g., in an existing `task_state` / `kv_state` table, or a small dedicated `notification_backfill_state` table with columns `task text primary key`, `last_processed_profile_id text`, `started_at`, `updated_at`, `completed_at`). Decide between the two during Task 6 implementation; the simplest option is a single dedicated table since this is the only checkpoint the notification system needs. +- For each profile (in `WHERE profile_id > $checkpoint ORDER BY profile_id LIMIT $batch_size` order): + - load visible libraries + - load favorited/watchlisted series + - load episodic progress rows and resolve to series + - recompute one `profile_series_interest` row per `(library_id, series_id)` +- Upsert the rebuilt rows. +- After each batch, update the checkpoint row with the highest `profile_id` processed; commit. A crash between batches resumes from the checkpoint with at most one batch of repeated work (cheap because upserts are idempotent). +- Mark the checkpoint `completed_at` when the iteration sees zero remaining profiles. Subsequent reruns are no-ops unless `completed_at` is reset. +- Optionally prune rows that no longer have any interest flags and no progression cursor state. + +### Backfill Safety Rules + +- Backfill must be idempotent (recomputing a profile's interest rows produces the same upsert outcome regardless of how many times it runs). +- Backfill must be resumable via the checkpoint described above. A crash mid-batch leaves work to redo for at most one batch; a crash between batches resumes exactly where the checkpoint left off. +- Backfill must not emit notifications. +- Fanout must remain disabled until the first backfill has completed successfully or reached an acceptable coverage threshold. + +--- + +## Verification Plan + +Per repo guidance, keep verification proportional and backend-focused. + +- [ ] Unit test episode key helpers and fanout eligibility logic. +- [ ] Repository tests for: + - idempotent `episode_availability` insert + - idempotent `release_events` insert + - bulk delivery dedupe + - unread count and read APIs +- [ ] Worker tests for: + - one release -> many deliveries + - repeat processing -> no duplicates + - `next_up` cursor gating + - merged reason flags + - multi-node claim safety + - profile mismatch suppression in websocket delivery + - unseeded library -> availability rows but zero release events; seed marker flips behavior + - per-series burst cap: N+5 events for one series -> N fanned out, 5 suppressed with `suppressed_reason` + - cross-library dedupe: same episode released in two libraries -> one delivery per profile + - outbox recovery: deliveries committed with `pending` attempt rows and no dispatch -> recovery worker sends them +- [ ] API tests for: + - inbox list scoping to active profile + - unread count + - mark-read idempotency + - preferences read/write if included +- [ ] Manual verification: + - favorite a show + - add a new episode + - confirm one `release_event`, one `notification_delivery`, one websocket event, one unread badge increment + - repeat scan and confirm no duplicate delivery + - switch profiles and confirm unread counts and inbox rows change correctly +- [ ] Load verification: + - simulate a popular series with many interested profiles + - confirm fanout batch time and insert counts remain bounded + - confirm ingest completion time does not materially regress when release events are enabled + +--- + +## Metrics and Observability + +- `release_events_created_total` +- `release_events_processed_total` +- `release_events_suppressed_total` (burst cap; labeled by reason) +- `notification_outbox_recovered_total` (pending attempt rows claimed by recovery instead of the inline dispatcher) +- `notification_availability_seeded_total` +- `notification_recipients_selected_total` +- `notification_deliveries_inserted_total` +- `notification_deliveries_deduped_total` +- `notification_fanout_duration_ms` +- `notification_ws_publish_failures_total` +- `notification_repair_runs_total` +- `notification_interest_rebuild_rows_total` +- `notification_unread_count_queries_total` +- `notification_preferences_updates_total` + +Log structured attributes: +- `library_id` +- `series_id` +- `episode_id` +- `recipient_count` +- `inserted_count` +- `deduped_count` +- `duration_ms` +- `profile_id` for API and websocket logs where appropriate + +--- + +## Risks and Safeguards + +- **Risk:** fanout leaks notifications across profiles. + - **Safeguard:** carry both `user_id` and `profile_id` on deliveries and websocket events; enforce profile filtering in handlers. + +- **Risk:** rescans emit duplicate notifications. + - **Safeguard:** dedupe at both `episode_availability` and `notification_deliveries`. + +- **Risk:** profile interest rows drift from source-of-truth user state. + - **Safeguard:** backfill and repair tasks, plus idempotent updaters on every source mutation. + +- **Risk:** ingest slows down due to notification work. + - **Safeguard:** ingest only records availability and release events; fanout is entirely asynchronous. + +- **Risk:** feature ships with incomplete interest state and under-notifies. + - **Safeguard:** run the backfill before enabling fanout and keep a repair task available. + +- **Risk:** notification tables grow without bound. + - **Safeguard:** define retention up front and implement cleanup tasks. + +- **Risk:** a partial worker failure marks events processed too early. + - **Safeguard:** process deliveries and processed-state updates in one transaction. + +- **Risk:** a crash between delivery commit and channel dispatch silently loses pushes/webhooks. + - **Safeguard:** the outbox — `pending` per-target attempt rows committed in the fanout transaction; recovery workers sweep stale pending rows. + +- **Risk:** a back-catalog import floods inboxes and the push relay on a server with hundreds of users. + - **Safeguard:** availability seeding for new libraries and feature enablement; per-series burst cap with logged suppression for bulk additions to existing libraries. + +- **Risk:** dual-quality libraries ("TV" + "TV 4K") double-notify every release. + - **Safeguard:** partial unique index on `(profile_id, episode_id)` for `episode.available` deliveries. + +--- + +## Defaults and Assumptions + +- V1 covers episodes only. +- V1 stores one delivery per new episode per profile. +- Aggregated phrasing like "3 new episodes available" is a presentation or v2 delivery concern. +- Availability is determined by actual in-library episode presence, not future calendar metadata. +- Existing `catalog` websocket invalidation remains separate from the new notification channel. +- Preference defaults are all-enabled unless a profile has explicitly saved overrides. +- Home-surface dismissals are independent from notification suppression. + +--- + +## Launch Checklist + +- [ ] Migrations applied successfully. +- [ ] Interest backfill task deployed. +- [ ] Interest backfill completed or reached acceptable coverage. +- [ ] Availability seeding completed; every library has a seed marker. +- [ ] Release event creation verified on a staging ingest, including: adding a brand-new library emits zero events; adding a back-catalog season to an existing library emits events but fanout is capped. +- [ ] Fanout worker enabled in staging and dedupe validated with repeated scans. +- [ ] Inbox APIs validated with multiple profiles on the same user. +- [ ] Realtime events validated with multiple tabs and profile switching. +- [ ] Retention/cleanup path scheduled or documented. +- [ ] Metrics visible in logs or dashboards before production rollout. diff --git a/docs/superpowers/plans/notifications/02-apns-relay.md b/docs/superpowers/plans/notifications/02-apns-relay.md new file mode 100644 index 00000000..28b8bf43 --- /dev/null +++ b/docs/superpowers/plans/notifications/02-apns-relay.md @@ -0,0 +1,891 @@ +# Privacy-Preserving APNs Relay Spec + +**Date:** 2026-04-27 (refined 2026-04-28) +**Status:** Draft (refined; open questions resolved) +**Scope:** Silo server push integration, Apple clients, opt-in central APNs relay, custom APNs provider configuration +**Depends On:** +- [`00-architecture-overview.md`](./00-architecture-overview.md) — read first for cross-cutting context +- [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) — foundational durable inbox + fanout +**Sibling spec:** +- [`03-fcm-relay.md`](./03-fcm-relay.md) — Android push using the same threat model and architecture + +## Summary + +Silo should support remote Apple notifications through either an opt-in +central APNs relay or admin-configured APNs credentials, while keeping +notification metadata on the user's own Silo server. + +The hosted relay should be a constrained transport service. It should receive +only the minimum fields needed to submit an APNs request: + +- an APNs device token +- APNs environment and app topic +- a push mode +- an opaque server/device correlation key +- an opaque delivery identifier +- optional badge count + +Neither provider path may send titles, item names, usernames, profile names, +library names, server URLs, artwork URLs, watched-state metadata, or +notification body text through APNs. + +When the Apple device wakes, the app should fetch real notification metadata +directly from the user's Silo server using the existing authenticated +server connection. + +## Problem + +Reliable Apple remote notifications require APNs. A self-hosted Silo server +cannot directly wake an iPhone, iPad, or Apple TV app in the background without +going through Apple's push infrastructure. + +That creates two product concerns: + +1. Privacy: notification contents should not be exposed to Apple or a Silo + operated relay. +2. Operational friction: most server admins should not need to create and manage + their own Apple developer account, APNs key, bundle topic, and push service. + +Silo should offer a practical default that preserves the self-hosted trust +model as much as the platform allows. + +## Decision + +Add configurable Apple push providers for the official Apple app: + +- `off`: no remote Apple push +- `silo_relay`: Silo's hosted APNs relay +- `custom_apns`: admin-supplied APNs credentials used directly by the server + +The relay submits APNs requests using Silo-controlled Apple credentials for +the official app bundle. A self-hosted server admin may opt in to use this relay. +The relay's API should intentionally prevent notification content from being +sent through the relay. + +If `custom_apns` is selected, the user's Silo server submits directly to +APNs using admin-provided Apple developer credentials. This avoids the Silo +relay entirely, but it must use the same minimal payload and metadata-fetch +rules as the hosted relay path. + +The default push mode should be **Private Alert**: + +- APNs displays a generic Silo notification. +- The APNs payload contains only opaque identifiers. +- The app fetches the actual notification row from the user's server when it is + opened or granted background execution. + +The implementation should also support **Background Wake** for low-noise sync, +with the explicit understanding that iOS background pushes are opportunistic and +not a reliable user-visible notification mechanism. + +## Goals + +- Make Apple push possible without every admin managing APNs credentials. +- Keep actual notification metadata on the user's Silo server. +- Make relay participation explicit and disabled by default. +- Make APNs credentials configurable for admins who want to avoid the hosted + relay. +- Keep the relay stateless for device subscriptions where possible. +- Keep APNs payloads opaque and generic. +- Keep the hosted relay and custom APNs paths behaviorally equivalent from the + app's point of view. +- Fit on top of the durable notification inbox design. + +## Non-Goals + +- Replacing APNs for Apple remote push. +- Shipping notification content through the central relay. +- Requiring Firebase or Google services for this Apple path. +- Solving Android push in this spec. +- Adding marketing, analytics, or delivery-tracking exports to the relay. +- Hiding APNs itself from Apple devices. Custom APNs still uses Apple APNs. +- Making silent background delivery as reliable as visible push. + +## Threat Model + +### What The User's Server Knows + +The user's Silo server knows: + +- the notification content +- the profile and user recipients +- the registered local devices +- APNs device tokens for those devices +- push delivery attempts and failures + +This is acceptable because the server is already the authority for the user's +media, profiles, sessions, and notification inbox. + +### What The Central Relay May Know + +The relay may see: + +- an opt-in relay account or install identifier +- the server's egress IP address (inherent to any hosted relay — the server + connects to it directly; for home hosting this identifies the household's + connection, making it the closest thing to an identity leak in this design) +- APNs device tokens submitted in send requests +- request timestamps +- coarse push mode, such as `private_alert` or `background_wake` +- opaque `collapse_id` values (per-server-keyed HMACs of series identity; the + relay can group one server's pushes into series-equivalence classes but + cannot recover which series, nor correlate across servers — see "Send Apple + Push" validation) +- APNs response status and APNs request IDs +- opaque delivery identifiers + +The relay must not receive enough information to know what media item, +notification type, profile, user, library, server hostname, or server URL caused +the push. + +### What Apple May Know + +Apple/APNs may see: + +- the official app topic +- the target device token +- APNs headers +- request timing +- the generic APNs payload + +Apple must not receive media titles, usernames, library names, server URLs, +artwork URLs, or other notification metadata in the payload. + +### Residual Metadata Leakage + +This design cannot hide that a device received a Silo push at a particular +time, nor the egress IP of the server sending relay requests. It can only hide +the meaning and content of the push. Admins for whom IP exposure matters can +route relay traffic through their existing VPN/proxy egress; the relay path is +plain HTTPS and needs no special handling. + +## User And Admin Controls + +### Server Admin Control + +The server admin must explicitly choose an Apple push provider. + +Suggested setting: + +```text +notifications.apple_push.provider = off | silo_relay | custom_apns +``` + +Default: + +```text +off +``` + +If `silo_relay` is selected, the admin configures a relay API key and +accepts a clear privacy notice: + +```text +Silo Relay can wake Apple devices through APNs, but notification details +stay on this server. The relay receives APNs device tokens, timestamps, opaque +delivery IDs, and generic push mode only. +``` + +If `custom_apns` is selected, the admin configures their own Apple developer +APNs credentials: + +```text +notifications.apple_push.custom.team_id +notifications.apple_push.custom.key_id +notifications.apple_push.custom.private_key +notifications.apple_push.custom.default_environment = production | sandbox +notifications.apple_push.custom.allowed_topics = [...] +``` + +The server then sends directly to APNs and does not call the Silo relay. +The same private payload rules still apply. + +### Device User Control + +Each Apple device must opt in independently: + +- the OS notification permission must be granted +- the app must be signed into a Silo server +- the active profile must enable push notifications for that device +- the server admin must have enabled an Apple push provider + +Admins must not be able to silently enroll every profile/device into remote push +without the device having granted OS notification permission. + +### Profile Control + +Push preferences should sit on top of the durable notification preferences from +the inbox design. + +Suggested profile/device modes: + +- `off`: never send remote push to this device +- `in_app_only`: websocket and inbox only +- `private_push`: generic APNs wake/alert, fetch details from server +- `full_preview`: out of scope for v1; would require explicit user consent and + a separate privacy warning + +V1 should ship only `off`, `in_app_only`, and `private_push`. + +## System Architecture + +```mermaid +sequenceDiagram + participant App as Apple App + participant Server as User Silo Server + participant Relay as Silo APNs Relay + participant APNs as Apple APNs + + App->>App: Request OS notification permission + App->>App: Register for remote notifications + App->>Server: Register device token and opaque server_device_id + Server->>Server: Store encrypted token and device preferences + Server->>Server: Create durable notification_delivery + Server->>Relay: Send minimal APNs relay request + Relay->>APNs: Submit generic APNs payload + APNs->>App: Deliver private alert or background wake + App->>Server: Fetch notification metadata directly + Server->>App: Return inbox rows for active profile +``` + +## Ownership Boundaries + +### This Repository + +Silo owns: + +- admin settings for selecting a push provider +- profile and device notification preferences +- APNs token registration from Apple clients +- durable storage of device registrations +- push fanout from `notification_deliveries` +- relay client integration +- retry, backoff, and APNs error handling +- inbox sync APIs used after wake + +### Apple Client Repository + +The Apple app owns: + +- APNs registration +- notification permission UX +- device registration API calls +- local server/device correlation +- wake handling +- fetching metadata from the user's server +- rendering local/in-app notification details + +### Central Relay Service + +The central relay owns: + +- APNs provider credentials for official Silo bundle topics +- relay API authentication +- request validation and rate limiting +- APNs request submission +- redacted operational logs +- minimal delivery status reporting + +The relay should live outside this repo unless the project later decides to +co-locate the service code. This repo should define the contract and implement +the self-hosted server side. + +### Custom APNs Mode + +When `custom_apns` is selected, there is no central relay in the send path. + +The user's Silo server owns: + +- APNs provider authentication using the configured team ID, key ID, and private + key +- topic allowlisting for the official Silo app bundle IDs it intends to + support +- sandbox vs production environment selection +- APNs request construction +- APNs response handling +- APNs credential rotation and secret storage + +Custom APNs mode must not unlock richer payloads. It exists to let admins avoid +the Silo relay, not to bypass the privacy-preserving notification contract. + +## Data Model + +This spec assumes `notification_deliveries` exists from the durable notification +inbox design. + +### `push_devices` + +Purpose: profile/device-scoped remote push registration. Shared across Apple and Android — see [`03-fcm-relay.md`](./03-fcm-relay.md) for the FCM-specific columns. + +Columns: + +- `id text primary key` +- `user_id integer not null` (matches `users.id integer`) +- `profile_id text not null` +- `device_id text not null` +- `platform text not null` — `'apple'` or `'android'`. Required; the row is invalid without it. +- `provider text not null` — for Apple rows: `'silo_relay'` or `'custom_apns'`. For Android rows: `'silo_relay'` or `'custom_fcm'`. (Effectively the active configured provider at registration time; updates rewrite the row.) +- `apns_environment text` — `'production'` / `'sandbox'`. Nullable; populated only for `platform = 'apple'`. +- `apns_topic text` — bundle topic. Nullable; populated only for `platform = 'apple'`. +- `apns_token_ciphertext bytea` — nullable; populated only for `platform = 'apple'`. +- `apns_token_hash text` — nullable; populated only for `platform = 'apple'`. +- `fcm_token_ciphertext bytea` — nullable; populated only for `platform = 'android'`. Defined here so the table is forward-compatible with [`03-fcm-relay.md`](./03-fcm-relay.md); the FCM spec adds the rest of the Android-specific columns. +- `fcm_token_hash text` — nullable; Android only. +- `fcm_project_id text` — nullable; Android only. +- `fcm_package_name text` — nullable; Android only. +- `server_device_id text not null` — random opaque value, generated server-side at registration. Stable across token rotation; rotates only when the device is removed and re-registered. +- `push_mode text not null default 'private_push'` — profile-level mode (see [`00-architecture-overview.md`](./00-architecture-overview.md) "Mode terminology"). +- `enabled boolean not null default true` +- `last_seen_at timestamptz` +- `last_success_at timestamptz` +- `last_failure_at timestamptz` +- `last_failure_code text` +- `created_at timestamptz not null default now()` +- `updated_at timestamptz not null default now()` + +Constraints: + +- unique on `(profile_id, device_id, platform)` — a device can register exactly once per platform per profile, regardless of which provider the admin happens to have configured. This matters because admins may flip between `silo_relay` and `custom_apns` mid-rotation; a `provider`-scoped unique key would leave stale rows behind. +- unique on `(server_device_id)` +- `CHECK ((platform = 'apple' AND apns_token_hash IS NOT NULL AND fcm_token_hash IS NULL) OR (platform = 'android' AND fcm_token_hash IS NOT NULL AND apns_token_hash IS NULL))` — exactly the platform-appropriate token columns are populated. + +Notes: + +- `server_device_id` must be random and opaque. It is the value included in the + APNs payload so the app can map the push back to the correct local server + account. +- Store APNs tokens encrypted at rest when the local encryption facility exists. +- Store `apns_token_hash` for dedupe and diagnostics without logging raw tokens. + +### `push_delivery_attempts` + +Purpose: operational record of push attempts without notification content. + +Columns: + +- `id text primary key` +- `notification_delivery_id text not null` +- `push_device_id text not null` +- `provider text not null` +- `attempt_number integer not null` — 1-based counter incremented on each retry. Used as the `Idempotency-Key` suffix for relay calls (see Send Apple Push), and lets diagnostics distinguish "10 attempts of one delivery" from "10 deliveries each first-attempted". +- `relay_request_id text` +- `apns_id text` — Apple's APNs ID returned in the response. Nullable; null for FCM rows. +- `fcm_message_name text` — FCM v1 message name (see [`03-fcm-relay.md`](./03-fcm-relay.md)). Nullable; null for APNs rows. +- `status text not null` — `'pending'`, `'delivered'`, `'retrying'`, `'failed'`, `'device_disabled'`. `pending` rows are the dispatch outbox: the fanout transaction in [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) inserts attempt 1 as `pending` in the same transaction as the delivery; the push dispatcher claims and sends them post-commit. +- `failure_code text` +- `attempted_at timestamptz not null default now()` +- `next_retry_at timestamptz` + +Constraints: + +- unique on `(notification_delivery_id, push_device_id, attempt_number)` to make the relay's `Idempotency-Key` deterministic and prevent double-claims of an attempt. +- add an index on `(status, next_retry_at)` for retry workers. The same index serves outbox recovery: a sweeper claims `pending` rows older than ~60s (crash between delivery commit and dispatch) with `FOR UPDATE SKIP LOCKED`. + +Retention: + +- keep successful attempts for a short operational window, such as 14 days +- keep failed attempts longer, such as 30 to 90 days, to diagnose device issues + +## Server API Surface + +### Register Apple Push Device + +```http +POST /api/v1/devices/push/apple +``` + +Profile-scoped. Requires normal auth plus active profile context. + +Request: + +```json +{ + "device_id": "apple-device-local-id", + "apns_token": "hex-or-base64-token", + "apns_environment": "production", + "apns_topic": "com.continuum.app.ios", + "push_mode": "private_push" +} +``` + +Response: + +```json +{ + "id": "01J...", + "server_device_id": "01JOPAQUE...", + "enabled": true, + "push_mode": "private_push" +} +``` + +Rules: + +- `apns_topic` must be allowlisted by app build/channel. +- `apns_environment` must be `sandbox` or `production`. +- registration is idempotent by `(profile_id, device_id, provider)`. +- token rotation updates the encrypted token and token hash. +- a disabled server push provider should still allow registration, but should + report that remote push is unavailable in a capability field if the frontend + needs it. + +### Disable Push Device + +```http +DELETE /api/v1/devices/push/{id} +``` + +Profile-scoped and idempotent. + +The server may either delete the row or mark `enabled = false`. Marking disabled +is preferred for diagnostics and future re-enable flows. + +### Push Capability + +```http +GET /api/v1/notifications/capability +``` + +The full envelope is defined in [`00-architecture-overview.md`](./00-architecture-overview.md). The Apple-relevant slice is: + +```json +{ + "apple_push": { + "available": true, + "provider": "silo_relay", + "available_providers": ["silo_relay", "custom_apns"], + "supported_modes": ["private_push", "in_app_only"], + "privacy_mode": "metadata_fetch" + } +} +``` + +The full response also includes `in_app`, `android_push` (see [`03-fcm-relay.md`](./03-fcm-relay.md)), and `webhooks` (see [`04-outbound-webhooks.md`](./04-outbound-webhooks.md)). + +This lets clients present truthful setup UI without guessing whether the admin +has enabled remote push. + +## Relay API Contract + +The relay API should be intentionally narrow. The user server should not send an +arbitrary APNs payload. + +### Send Apple Push + +```http +POST /v1/apple/send +Authorization: Bearer +Idempotency-Key: :: +``` + +Request: + +```json +{ + "token": "apns-token", + "environment": "production", + "topic": "com.continuum.app.ios", + "mode": "private_alert", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY...", + "badge": null, + "collapse_id": "01JOPAQUE_COLLAPSE" +} +``` + +Response: + +```json +{ + "request_id": "01JRELAY...", + "apns_id": "550e8400-e29b-41d4-a716-446655440000", + "status": "accepted" +} +``` + +Validation: + +- `token` is required and must be plausible for APNs. +- `environment` must be `sandbox` or `production`. The relay forwards `production` requests to `https://api.push.apple.com` and `sandbox` requests to `https://api.development.push.apple.com` (Apple's current canonical hostnames; older docs/libraries also reference `api.sandbox.push.apple.com`, but `api.development.push.apple.com` is the current Apple-documented form). +- `topic` must be allowlisted for the relay account. +- `mode` must be `private_alert` or `background_wake`. +- `server_device_id` and `delivery_id` must be opaque values with length limits (recommend ≤128 chars, ULID-shaped). +- `collapse_id` must be opaque and ≤64 bytes — Apple enforces a 64-byte cap on `apns-collapse-id`. Reject longer values with HTTP 400. +- `collapse_id` derivation (server-side rule, not relay-enforced): the server computes `collapse_id = base32(HMAC-SHA256(server_collapse_secret, series_id))` truncated to 26 chars, where `server_collapse_secret` is a random per-server secret generated at first use. Never send raw or plainly-hashed `series_id` — an unkeyed hash would let the relay (or Apple) dictionary-match popular series IDs. Residual: the relay can still group one server's pushes into per-series equivalence classes (that is what collapse is for); the per-server key prevents recovering the series or correlating across servers. +- `badge` is optional. Default should be omitted to avoid leaking unread counts through APNs unless the user/admin explicitly enables badge sync. +- no free-form notification title or body fields are accepted. +- no image URL, media ID, username, server hostname, or server URL field is accepted. + +### Relay-Built APNs Payloads + +For `private_alert`, the relay constructs: + +```json +{ + "aps": { + "alert": { + "title-loc-key": "SILO_NOTIFICATION_TITLE", + "loc-key": "SILO_NOTIFICATION_GENERIC_BODY" + }, + "sound": "default" + }, + "silo": { + "v": 1, + "wake": "notifications.changed", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY..." + } +} +``` + +For `background_wake`, the relay constructs: + +```json +{ + "aps": { + "content-available": 1 + }, + "silo": { + "v": 1, + "wake": "notifications.changed", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY..." + } +} +``` + +Headers: + +- `apns-topic`: selected from the allowlisted `topic` +- `apns-push-type`: `alert` for `private_alert`, `background` for + `background_wake` +- `apns-priority`: `10` for `private_alert`, `5` for `background_wake` +- `apns-collapse-id`: optional opaque `collapse_id` + +The relay should not allow callers to override this payload shape in v1. + +## Custom APNs Contract + +`custom_apns` should share the same internal send model as `silo_relay`, +but replace the relay HTTP call with a direct APNs provider request. + +The server-side push sender should accept only a structured internal request: + +```json +{ + "token": "apns-token", + "environment": "production", + "topic": "com.continuum.app.ios", + "mode": "private_alert", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY...", + "badge": null, + "collapse_id": "01JOPAQUE_COLLAPSE" +} +``` + +The sender builds the APNs payload locally using the exact same payload shapes +defined above for `private_alert` and `background_wake`. + +Configuration: + +- `team_id`: Apple developer team ID. +- `key_id`: APNs auth key ID. +- `private_key`: APNs `.p8` private key, stored as a secret. +- `default_environment`: `production` or `sandbox`. +- `allowed_topics`: explicit bundle topics this server is allowed to send for. + +Rules: + +- The admin may configure credentials, environment, and topics. +- The server must not expose a free-form APNs JSON payload setting. +- The server must not expose custom title/body templates for remote push in v1. +- Topic values must match the device registration topic and the configured + allowlist. +- Sandbox tokens must be sent to the sandbox endpoint and production tokens to + the production endpoint. +- APNs auth tokens should be cached briefly and regenerated before expiry. +- Credential validation should provide a test-send or dry-run diagnostic that + does not include notification content. + +## Wake And Metadata Fetch + +When the app receives a push: + +1. Read `silo.v`, `server_device_id`, and `delivery_id`. +2. Find the local server account that owns `server_device_id`. +3. If background execution is available, call the server immediately. +4. If the app is opened from the notification, call the server before rendering + the target screen. +5. Fetch notification metadata from the user's server. +6. Render the real notification from server data. + +Suggested fetch: + +```http +GET /api/v1/notifications/sync?since= +``` + +or: + +```http +GET /api/v1/notifications/{delivery_id} +``` + +The sync endpoint is preferred because push delivery is not guaranteed and +multiple notification deliveries can be coalesced. + +If the app cannot reach the server, it should keep the generic notification and +retry the inbox sync later. + +## Fanout Rules + +Push fanout should happen after durable inbox delivery commits. + +Flow: + +1. `notification_deliveries` row is created. **In the same transaction**, the + fanout worker enqueues one `pending` `push_delivery_attempts` row per + enabled `push_device` of the recipient profile (the dispatch outbox — see + [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) + "Transaction and Concurrency Rules"). +2. The realtime websocket event is published for connected clients. +3. The push dispatcher claims the `pending` attempt rows post-commit + (`FOR UPDATE SKIP LOCKED`); a recovery sweeper claims rows older than ~60s + whose dispatch never ran. +4. Each claimed attempt sends through the configured provider: + - `silo_relay`: send the minimal relay request + - `custom_apns`: build the same minimal APNs payload locally and send it + directly to APNs +5. APNs errors update device state and attempt status. + +Do not block notification delivery creation on APNs or relay availability. + +### Relay pacing + +The dispatcher paces relay calls client-side with a token bucket (default 5 +req/sec, setting `notifications.push.relay_send_rate`), shared across the APNs +and FCM dispatchers since both consume the same relay account quota. On a +server with hundreds of users, a popular release can enqueue several hundred +push attempts at once; pacing drains the queue smoothly instead of slamming +into the relay's rate limit and burning retries on 429s. `TooManyRequests` +from the relay is still honored with backoff — pacing is the steady-state +mechanism, backoff is the safety net. The per-series burst cap in `01` bounds +worst-case queue depth. Custom APNs sends are paced separately and more +generously (Apple's own limits are far higher than any single-server relay +quota). + +## APNs Error Handling + +The server should handle relay/APNs failures without exposing notification +content. + +Recommended handling: + +- `BadDeviceToken` / `InvalidToken`: disable the token and require re-registration. (Apple has migrated some responses from `BadDeviceToken` to `InvalidToken`; treat them equivalently.) +- `Unregistered`: disable the token. +- `DeviceTokenNotForTopic`: disable and log a build/topic mismatch. +- `TooManyRequests`: retry with backoff. +- `ExpiredProviderToken`: applies only to `custom_apns`. The cached APNs JWT has expired. Regenerate the JWT immediately and retry the request once before backing off. The relay path never surfaces this code because the relay manages its own JWTs. +- `TooManyProviderTokenUpdates`: applies only to `custom_apns`. Apple has rate-limited JWT regeneration. Stop regenerating; reuse the existing token; retry after backoff. Indicates a bug in the JWT cache (regenerating more often than every 20 minutes). +- relay unavailable, APNs unavailable, or network timeout: retry with backoff. +- malformed request: mark failed and surface admin diagnostics. + +Errors should be visible to admins as operational status, not to normal users +unless their device needs reauthorization. + +## Privacy Requirements + +The implementation must satisfy these requirements before shipping: + +- Relay requests contain no notification title or body. +- Relay requests contain no media identifiers. +- Relay requests contain no server hostname or base URL. +- Relay requests contain no profile, username, library, collection, or item names. +- Relay requests *do* carry `apns-topic` (e.g., `com.continuum.app.ios`) and `mode` (`private_alert` / `background_wake`). These are platform/build identifiers that Apple already sees; documenting them here so the privacy claim list is exhaustive rather than over-strong. +- Direct APNs payloads contain no notification title or body beyond generic + localizable keys. +- Direct APNs payloads contain no media identifiers, server hostname, server URL, + profile name, username, library name, collection name, or item name. +- Relay logs redact raw APNs tokens. +- Relay logs redact authorization headers. +- Relay logs do not persist full request payloads. +- Server logs redact APNs tokens and relay API keys. +- Badge count is disabled by default. +- Push registration and remote push provider use are opt-in. +- Device unregister disables further push attempts for that device. + +## Security Requirements + +- Relay API uses TLS only. +- Relay API keys are scoped to a relay account or install identifier. +- Relay API keys can be revoked without changing the user's Silo server + authentication. +- Relay requests are rate limited by API key and coarse token hash. +- Relay supports idempotency keys to prevent duplicate APNs sends during retry. +- Relay does not accept arbitrary APNs payload JSON in v1. +- Server stores relay API keys as secrets. +- Server stores custom APNs private keys as secrets. +- Server never logs custom APNs private keys, generated provider tokens, or APNs + auth headers. +- Server stores APNs tokens encrypted at rest where local secret storage exists. +- Admin diagnostics must not print raw APNs tokens. + +## Settings And UX + +Admin settings should explain the tradeoff plainly: + +```text +Apple Push Provider + +Off + No Apple remote push. Devices still receive in-app realtime updates while open. + +Silo Relay + Uses Silo's APNs relay to wake Apple devices. Notification details stay on + this server. The relay receives APNs device tokens, timestamps, and opaque + delivery IDs. + +Custom APNs + Advanced. Send directly to Apple APNs using your own Apple developer + credentials. Notification details still stay on this server. +``` + +Device settings should describe the user-visible mode: + +```text +Private Push + Show a generic Silo notification, then fetch details from your server when + this device wakes. +``` + +Do not claim the central relay is fully self-hosted. The truthful claim is: + +```text +Notification content stays on your server. Apple, and the Silo relay if +selected, may still process generic wake messages needed for Apple push +delivery. +``` + +## Implementation Plan + +### Task 1: Add Push Provider Settings + +Files likely involved: + +- `internal/api/handlers/settings.go` +- `web/src/lib/settingsManifest.ts` +- settings UI files as needed + +Add server/admin settings for: + +- provider selection +- relay endpoint +- relay API key +- custom APNs team ID +- custom APNs key ID +- custom APNs private key +- custom APNs environment +- custom APNs topic allowlist +- badge sync enabled or disabled +- relay send rate (`notifications.push.relay_send_rate`, shared with FCM pacing) + +Default provider must be `off`. + +### Task 2: Add Push Device Registration + +Files likely involved: + +- new migration under `migrations/` +- new package or service under `internal/notifications` +- `internal/api/handlers/notifications.go` +- `internal/api/router.go` + +Add profile-scoped Apple push device registration, token rotation, and disable +APIs. + +### Task 3: Add Apple Push Provider Clients + +Files likely involved: + +- `internal/notifications/apple_relay.go` +- `internal/notifications/apple_apns.go` +- config/settings accessors + +Implement the narrow `/v1/apple/send` client. The client should not accept a +free-form title/body/content payload. + +Implement the direct APNs client for `custom_apns` using the same internal +request shape and payload builder. + +### Task 4: Add Push Fanout Worker + +Files likely involved: + +- `internal/notifications/push_fanout.go` +- notification delivery creation path from the durable inbox implementation + +Trigger push fanout after `notification_deliveries` commit. Use retries and +record `push_delivery_attempts`. + +### Task 5: Add Apple Client Registration And Wake Handling + +Files live in the Apple client repository, not this server repo. + +Required client behavior: + +- request notification permission +- register APNs token with the user's server +- store `server_device_id` locally +- handle token rotation +- fetch notification metadata after wake/open +- unregister or disable on sign-out/profile removal + +### Task 6: Add Admin Diagnostics + +Expose high-level status: + +- provider enabled/disabled +- number of registered Apple devices +- last relay success +- last relay failure code +- custom APNs credential presence and topic/environment status +- last custom APNs success +- last custom APNs failure code +- token/topic mismatch warnings + +Do not expose raw APNs tokens. + +## Validation Plan + +Use minimal verification while the work is still design-only. When implemented, +verify: + +- relay-disabled servers never call the relay +- `custom_apns` servers never call the relay +- device registration is profile-scoped and idempotent +- APNs tokens are redacted in logs +- relay request bodies contain no titles, body text, item IDs, server URLs, or + profile names +- direct APNs payloads contain no titles, body text, item IDs, server URLs, or + profile names beyond the generic localizable notification keys +- custom APNs private keys and generated provider tokens are redacted in logs +- creating one `notification_delivery` sends at most one push per enabled device +- APNs token rotation updates the stored token without duplicating devices +- `Unregistered` and `BadDeviceToken` disable the affected device +- app wake fetches notification metadata from the user's server +- server offline after push leaves only the generic notification visible + +## Resolved Questions (was: Open Questions) + +These four open questions from the original draft were resolved during the multi-channel refinement: + +- **Topics: per-platform.** Official Silo builds use separate APNs topics for iOS, tvOS, and macOS. Each is a different bundle ID with its own provisioning. The `apns_topic` allowlist on a relay account enumerates the per-platform topics the account may push to. +- **Badge counts: disabled in v1.** Badge count leaks unread volume through APNs. v1 ships without badge updates; an explicit per-profile opt-in to badge sync can land in v2 alongside a privacy notice. +- **Relay statefulness: stateless v1.** No stored token aliases. The relay accepts raw APNs tokens on every request. Token aliases add complexity for a benefit (reduced repeat-token exposure to relay logs) that is small compared to "redact tokens from logs." Add aliases later if abuse-pattern analysis or rate-limit pressure justifies it. +- **Relay deployment repo: out of scope for this spec.** The relay implementation lives in a separate repo (provisional name `silo-push-relay`). This document defines the contract the user-facing Silo server implements; the relay's internal architecture (deployment platform, dependency tree, etc.) is the relay repo's concern. + +## Remaining Open Questions + +- **Multi-bundle topic allowlist UX.** When an admin configures custom APNs, how do they declare allowlisted topics — free-text textarea, or a structured list? Recommendation: structured list with platform tags (iOS / tvOS / macOS) so the UI can show which platforms have working push. +- **APNs JWT cache lifetime.** APNs JWTs are valid for 60 minutes per Apple's docs but may be reused across requests. Should the custom APNs path cache and reuse, or sign per-request? Recommendation: cache with regeneration at 50 minutes to leave headroom, matching Apple's recommendation. diff --git a/docs/superpowers/plans/notifications/03-fcm-relay.md b/docs/superpowers/plans/notifications/03-fcm-relay.md new file mode 100644 index 00000000..098d4af4 --- /dev/null +++ b/docs/superpowers/plans/notifications/03-fcm-relay.md @@ -0,0 +1,748 @@ +# Privacy-Preserving FCM Relay Spec + +**Date:** 2026-04-28 +**Status:** Draft +**Scope:** Silo server push integration for Android, opt-in central FCM relay, custom FCM provider configuration +**Depends On:** +- [`00-architecture-overview.md`](./00-architecture-overview.md) +- [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) +- [`02-apns-relay.md`](./02-apns-relay.md) — sibling spec; structurally identical with FCM-specific differences + +## Summary + +Silo should support remote Android notifications through either an opt-in central FCM relay or admin-configured Firebase Cloud Messaging credentials, while keeping notification metadata on the user's own Silo server. + +The hosted relay should be a constrained transport service. It should receive only the minimum fields needed to submit a Firebase Cloud Messaging v1 API request: + +- an FCM registration token +- the target Firebase project ID and Android package name (allowlisted) +- a push mode +- an opaque server/device correlation key +- an opaque delivery identifier +- optional badge count (disabled by default) + +Neither provider path may send titles, item names, usernames, profile names, library names, server URLs, artwork URLs, watched-state metadata, or notification body text through FCM. + +When the Android device wakes, the app should fetch real notification metadata directly from the user's Silo server using the existing authenticated server connection. + +## Why this spec mirrors APNs + +The reasoning, threat model, and fanout flow are identical to the APNs case. Apple and Google both gatekeep mobile push at the app-bundle level: only the developer who signed and published the app can authenticate pushes to it. The same arguments for a hosted relay apply unchanged. The differences are entirely about FCM's protocol and credentials: + +- **Auth:** FCM v1 API uses **OAuth2 with a Google service account**, not a JWT signed with a `.p8` key. +- **Endpoint:** Pushes go to `https://fcm.googleapis.com/v1/projects/{project_id}/messages:send`. +- **Project model:** A Firebase project owns the Android app's package name + SHA-1 cert fingerprint mapping. The relay holds one service account JSON per official Firebase project. +- **Topics vs tokens:** FCM supports both topic broadcasts and per-token sends. Silo uses **per-token sends only** — topic broadcasts would leak audience information. +- **Message types:** FCM has `notification` and `data` messages. The privacy design uses **data-only messages** so the OS can't render content the relay shouldn't have constructed. + +## Decision + +Add configurable Android push providers for the official Silo Android app: + +- `off`: no remote Android push +- `silo_relay`: Silo's hosted FCM relay +- `custom_fcm`: admin-supplied FCM service account JSON used directly by the server + +The relay submits FCM v1 requests using Silo-controlled Firebase credentials for the official app package. A self-hosted server admin may opt in to use this relay. The relay's API should intentionally prevent notification content from being sent through the relay. + +If `custom_fcm` is selected, the user's Silo server calls FCM v1 directly using admin-provided service account JSON. This avoids the Silo relay entirely, but it must use the same minimal payload and metadata-fetch rules as the hosted relay path. + +The default push mode should be **Private Data Wake**: + +- FCM delivers a data-only message. +- The data payload contains only opaque identifiers. +- The Android app handles the message via `FirebaseMessagingService.onMessageReceived` and constructs a generic local notification while it fetches the actual notification row from the user's server. + +The implementation should also support **Background Wake** for low-noise sync, with the explicit understanding that Android background data messages are subject to Doze, App Standby, and battery-optimization restrictions and are not a reliable user-visible notification mechanism. + +## Goals + +- Make Android push possible without every admin managing their own Firebase project. +- Keep actual notification metadata on the user's Silo server. +- Make relay participation explicit and disabled by default. +- Make FCM credentials configurable for admins who want to avoid the hosted relay. +- Keep the relay stateless for device subscriptions where possible. +- Keep FCM payloads opaque and generic (data-only). +- Keep the hosted relay and custom FCM paths behaviorally equivalent from the app's point of view. +- Fit on top of the durable notification inbox design. + +## Non-Goals + +- Replacing FCM for Android remote push. +- Shipping notification content through the central relay. +- Requiring Apple push services for this Android path. +- Supporting Huawei Mobile Services (HMS) push, Amazon Device Messaging, or other Android-adjacent push systems in v1. Could be added later as parallel providers. +- Adding marketing, analytics, or delivery-tracking exports to the relay. +- Hiding FCM itself from Android devices. Custom FCM still uses Google FCM. +- Making silent background delivery as reliable as visible notifications. + +## Threat Model + +### What The User's Server Knows + +The user's Silo server knows: + +- the notification content +- the profile and user recipients +- the registered local devices +- FCM registration tokens for those devices +- push delivery attempts and failures + +This is acceptable because the server is already the authority for the user's media, profiles, sessions, and notification inbox. + +### What The Central Relay May Know + +The relay may see: + +- an opt-in relay account or install identifier +- the server's egress IP address (inherent to any hosted relay — the server connects to it directly; see the identical note in [`02-apns-relay.md`](./02-apns-relay.md)) +- FCM registration tokens submitted in send requests +- request timestamps +- coarse push mode, such as `private_data` or `background_wake` +- opaque `collapse_key` values (per-server-keyed HMACs of series identity; same derivation rule as APNs `collapse_id`) +- FCM response status and FCM message names +- opaque delivery identifiers + +The relay must not receive enough information to know what media item, notification type, profile, user, library, server hostname, or server URL caused the push. + +### What Google May Know + +Google / FCM may see: + +- the official app package name and Firebase project +- the target FCM registration token +- FCM v1 request headers +- request timing +- the generic FCM data payload (which contains only opaque identifiers) + +Google must not receive media titles, usernames, library names, server URLs, artwork URLs, or other notification metadata in the payload. In particular, the design uses **data-only messages**, never `notification` messages, so Google's servers don't carry rendering content. + +### Residual Metadata Leakage + +This design cannot hide that a device received a Silo push at a particular time, nor the egress IP of the server sending relay requests. It can only hide the meaning and content of the push. Additionally, on Android, FCM delivery may be delayed by Doze / App Standby; an adversary observing wake patterns may infer rough activity but not content. + +## User And Admin Controls + +### Server Admin Control + +The server admin must explicitly choose an Android push provider. + +Suggested setting: + +```text +notifications.android_push.provider = off | silo_relay | custom_fcm +``` + +Default: + +```text +off +``` + +If `silo_relay` is selected, the admin configures a relay API key and accepts a clear privacy notice: + +```text +Silo Relay can wake Android devices through Firebase Cloud Messaging, +but notification details stay on this server. The relay receives FCM +registration tokens, timestamps, opaque delivery IDs, and generic push mode +only. +``` + +If `custom_fcm` is selected, the admin configures their own Firebase service account JSON: + +```text +notifications.android_push.custom.service_account_json +notifications.android_push.custom.project_id +notifications.android_push.custom.allowed_packages = [...] +``` + +The server then sends directly to FCM and does not call the Silo relay. The same private payload rules still apply. + +### Device User Control + +Each Android device must opt in independently: + +- the OS notification permission must be granted (Android 13+ requires `POST_NOTIFICATIONS` runtime permission) +- the app must be signed into a Silo server +- the active profile must enable push notifications for that device +- the server admin must have enabled an Android push provider + +Admins must not be able to silently enroll every profile/device into remote push without the device having granted OS notification permission. + +### Profile Control + +Push preferences sit on top of the durable notification preferences from the inbox design. The push-mode field on a `push_devices` row is shared across platforms (`apple` and `android`) so the same `push_mode` enum applies. + +Suggested profile/device modes: + +- `off`: never send remote push to this device +- `in_app_only`: websocket and inbox only +- `private_push`: generic FCM data wake, fetch details from server +- `full_preview`: out of scope for v1 + +V1 should ship only `off`, `in_app_only`, and `private_push`. + +## System Architecture + +```mermaid +sequenceDiagram + participant App as Android App + participant Server as User Silo Server + participant Relay as Silo FCM Relay + participant FCM as Google FCM + + App->>App: Request POST_NOTIFICATIONS permission (API 33+) + App->>App: Get FCM registration token via FirebaseMessaging.getToken() + App->>Server: Register registration token and opaque server_device_id + Server->>Server: Store encrypted token and device preferences + Server->>Server: Create durable notification_delivery + Server->>Relay: Send minimal FCM relay request + Relay->>FCM: Submit data-only message via FCM v1 API + FCM->>App: Deliver data message; FirebaseMessagingService.onMessageReceived + App->>Server: Fetch notification metadata directly + Server->>App: Return inbox rows for active profile +``` + +## Ownership Boundaries + +### This Repository + +Silo owns: + +- admin settings for selecting an Android push provider +- profile and device notification preferences +- FCM token registration from Android clients +- durable storage of device registrations +- push fanout from `notification_deliveries` +- relay client integration +- retry, backoff, and FCM error handling +- inbox sync APIs used after wake + +### Android Client Repository + +The Android app owns: + +- FCM registration via Firebase SDK +- notification permission UX (`POST_NOTIFICATIONS` on API 33+) +- device registration API calls +- local server/device correlation +- `FirebaseMessagingService.onMessageReceived` handling +- fetching metadata from the user's server +- rendering local notifications via `NotificationManager` after the wake + +### Central Relay Service + +The central relay owns: + +- FCM service account credentials for the official Silo Firebase project(s) +- relay API authentication +- request validation and rate limiting +- FCM v1 request submission with OAuth2 access token caching +- redacted operational logs +- minimal delivery status reporting + +The relay should live outside this repo unless the project later decides to co-locate the service code. This repo defines the contract and implements the self-hosted server side. The relay implementation can share most code with the APNs relay; only the upstream provider client differs. + +### Custom FCM Mode + +When `custom_fcm` is selected, there is no central relay in the send path. + +The user's Silo server owns: + +- FCM v1 OAuth2 token generation from the configured service account JSON +- access token caching (Google's OAuth2 access tokens are short-lived; cache and refresh on expiry) +- package allowlisting for the official Silo app package(s) it intends to support +- FCM v1 request construction (data-only) +- FCM response handling +- FCM credential rotation and secret storage + +Custom FCM mode must not unlock richer payloads. It exists to let admins avoid the Silo relay, not to bypass the privacy-preserving notification contract. + +## Data Model + +The `push_devices` and `push_delivery_attempts` schemas are owned by [`02-apns-relay.md`](./02-apns-relay.md) and are platform-tagged from the start (`platform = 'apple' | 'android'`) so this spec adds no new columns. The Android-relevant columns on `push_devices` are: + +- `platform text not null` — must be `'android'` for FCM rows. +- `provider text not null` — `'silo_relay'` or `'custom_fcm'`. +- `fcm_token_ciphertext bytea` — required for Android rows. +- `fcm_token_hash text` — required for Android rows; used for dedupe/diagnostics without logging raw tokens. +- `fcm_project_id text` — the Firebase project the token was minted under. +- `fcm_package_name text` — the Android package name (e.g., `com.continuum.app.android`). +- All `apns_*` columns must be null for Android rows. The `CHECK` constraint defined in `02-apns-relay.md` enforces this exclusivity. + +Unique constraints (defined in `02-apns-relay.md`): `(profile_id, device_id, platform)` and `(server_device_id)`. Both are intentionally platform-scoped, not provider-scoped — the same Android device cannot register twice even if the admin flips between `silo_relay` and `custom_fcm`. + +`push_delivery_attempts` columns relevant to FCM (also owned by 02): + +- `provider text not null` — for FCM rows: `'silo_relay'` or `'custom_fcm'`. +- `apns_id text` — null for FCM rows. +- `fcm_message_name text` — populated on success with FCM v1's response `name` (e.g., `"projects/continuum-prod-android/messages/0:..."`); null for APNs rows. +- `attempt_number integer not null` — 1-based; participates in the unique constraint and in the relay `Idempotency-Key`. + +Retention is identical to APNs: ~14 days for successes, 30-90 for failures. + +## Server API Surface + +### Register Android Push Device + +```http +POST /api/v1/devices/push/fcm +``` + +Profile-scoped. Requires normal auth plus active profile context. + +Request: + +```json +{ + "device_id": "android-device-local-id", + "fcm_token": "long-fcm-registration-token", + "fcm_project_id": "continuum-prod-android", + "fcm_package_name": "com.continuum.app.android", + "push_mode": "private_push" +} +``` + +Response: + +```json +{ + "id": "01J...", + "server_device_id": "01JOPAQUE...", + "enabled": true, + "push_mode": "private_push" +} +``` + +Rules: + +- `fcm_package_name` must be allowlisted for the configured provider's allowed_packages list. +- `fcm_project_id` must match the configured Firebase project for the active provider. +- registration is idempotent by `(profile_id, device_id, platform)` where platform is implied as `android`. +- token rotation updates the encrypted token and token hash. FCM tokens rotate periodically; the app should call this endpoint whenever `FirebaseMessagingService.onNewToken` fires. +- a disabled server push provider should still allow registration, but should report that remote push is unavailable in the capability response. + +### Disable Push Device + +Shared with APNs, see [`02-apns-relay.md`](./02-apns-relay.md): + +```http +DELETE /api/v1/devices/push/{id} +``` + +### Push Capability + +Extended from the APNs spec to include Android: + +```http +GET /api/v1/notifications/capability +``` + +Response: + +```json +{ + "in_app": { "enabled": true }, + "apple_push": { + "available": true, + "provider": "silo_relay", + "available_providers": ["silo_relay", "custom_apns"], + "supported_modes": ["private_push", "in_app_only"] + }, + "android_push": { + "available": true, + "provider": "silo_relay", + "available_providers": ["silo_relay", "custom_fcm"], + "supported_modes": ["private_push", "in_app_only"] + }, + "webhooks": { + "available": true, + "max_per_profile": 10, + "supported_types": ["discord", "generic"] + } +} +``` + +## Relay API Contract + +The relay API should be intentionally narrow. The user server should not send an arbitrary FCM payload. + +### Send Android Push + +```http +POST /v1/fcm/send +Authorization: Bearer +Idempotency-Key: :: +``` + +Request: + +```json +{ + "token": "fcm-registration-token", + "project_id": "continuum-prod-android", + "package_name": "com.continuum.app.android", + "mode": "private_data", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY...", + "badge": null, + "collapse_key": "01JOPAQUE_COLLAPSE" +} +``` + +Response: + +```json +{ + "request_id": "01JRELAY...", + "fcm_message_name": "projects/continuum-prod-android/messages/0:1234567890123456%abcdef", + "status": "accepted" +} +``` + +Validation: + +- `token` is required and must be plausible for FCM (FCM tokens are typically 152-180+ chars, base64url-ish). +- `project_id` must match the configured Firebase project for this relay account's allowed packages. +- `package_name` must be allowlisted for the relay account. +- `mode` must be `private_data` or `background_wake`. Note: this is the **wire mode** (FCM-side); the profile-level mode is `private_push`. See [`00-architecture-overview.md`](./00-architecture-overview.md) "Mode terminology" for the mapping. +- `server_device_id`, `delivery_id`, and `collapse_key` must be opaque values with length limits. +- `collapse_key` derivation (server-side rule, not relay-enforced): identical to the APNs `collapse_id` rule in [`02-apns-relay.md`](./02-apns-relay.md) — `base32(HMAC-SHA256(server_collapse_secret, series_id))` truncated, using the same per-server secret. Never send raw or plainly-hashed `series_id`. +- `badge` is optional. Default should be omitted to avoid leaking unread counts unless the user/admin explicitly enables badge sync. +- no free-form notification title or body fields are accepted. +- no image URL, media ID, username, server hostname, or server URL field is accepted. + +### Relay-Built FCM Payloads + +For `private_data`, the relay constructs: + +```json +{ + "message": { + "token": "fcm-registration-token", + "data": { + "v": "1", + "wake": "notifications.changed", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY..." + }, + "android": { + "priority": "high", + "collapse_key": "01JOPAQUE_COLLAPSE" + } + } +} +``` + +For `background_wake`, the relay constructs: + +```json +{ + "message": { + "token": "fcm-registration-token", + "data": { + "v": "1", + "wake": "notifications.changed", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY..." + }, + "android": { + "priority": "normal", + "collapse_key": "01JOPAQUE_COLLAPSE" + } + } +} +``` + +Notes: + +- The relay sends **data-only messages** (no top-level `notification` field) so Google never carries rendering content. The Android app constructs the local notification in `onMessageReceived`. +- `priority: high` is used for `private_data` so the message wakes the app even under Doze. High-priority data messages are subject to FCM's anti-abuse quotas — if a server hits the quota, deliveries fall back to normal priority and may be deferred. +- `collapse_key` allows multiple pushes for the same series to coalesce when the device is offline, mirroring `apns-collapse-id`. +- The relay does not allow callers to override this payload shape in v1. + +## Custom FCM Contract + +`custom_fcm` should share the same internal send model as `silo_relay`, but replace the relay HTTP call with a direct FCM v1 API request. + +The server-side push sender should accept only a structured internal request (same shape as the relay request body above): + +```json +{ + "token": "fcm-registration-token", + "project_id": "continuum-prod-android", + "package_name": "com.continuum.app.android", + "mode": "private_data", + "server_device_id": "01JOPAQUE...", + "delivery_id": "01JDELIVERY...", + "badge": null, + "collapse_key": "01JOPAQUE_COLLAPSE" +} +``` + +The sender builds the FCM v1 payload locally using the exact same payload shapes defined above for `private_data` and `background_wake`. + +Configuration: + +- `service_account_json`: full Firebase service account JSON, stored as a secret. +- `project_id`: Firebase project ID. +- `allowed_packages`: explicit Android package names this server is allowed to send for. + +Rules: + +- The admin may configure credentials, project ID, and allowed packages. +- The server must not expose a free-form FCM JSON payload setting. +- The server must not expose custom title/body templates for remote push in v1. +- Package values must match the device registration package and the configured allowlist. +- OAuth2 access tokens should be cached and regenerated before expiry (Google access tokens are typically valid for 1 hour). +- Credential validation should provide a test-send or dry-run diagnostic that does not include notification content. FCM v1 supports `validate_only: true` requests for this purpose. + +## Wake And Metadata Fetch + +When the app receives an FCM data message: + +1. `FirebaseMessagingService.onMessageReceived(remoteMessage)` fires. +2. Read `data.v`, `data.server_device_id`, and `data.delivery_id`. +3. Find the local server account that owns `server_device_id`. +4. If the app is in the foreground, update in-app inbox state directly. +5. If background: + - Construct a generic local notification using `NotificationManager` with a localized "New Silo notification" title. + - Schedule a `WorkManager` job to fetch the actual metadata from the user's server. + - When the fetch returns, update the visible notification with real content (or replace it). +6. If the app is opened from the notification, call the server before rendering the target screen. + +Suggested fetch: + +```http +GET /api/v1/notifications/sync?since= +``` + +or: + +```http +GET /api/v1/notifications/{delivery_id} +``` + +The sync endpoint is preferred because push delivery is not guaranteed and multiple notification deliveries can be coalesced. + +If the app cannot reach the server, it should keep the generic notification and retry the inbox sync when connectivity returns. + +## Fanout Rules + +Push fanout should happen after durable inbox delivery commits. Same flow as APNs: + +1. `notification_deliveries` row is created. In the same transaction, the fanout worker enqueues one `pending` `push_delivery_attempts` row per enabled Android `push_device` (the dispatch outbox — see [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md)). +2. The realtime websocket event is published for connected clients. +3. The push dispatcher claims the `pending` attempt rows post-commit, partitioned by platform; a recovery sweeper claims rows older than ~60s whose dispatch never ran. +4. Each claimed attempt sends through the configured Android provider: + - `silo_relay`: send the minimal relay request. + - `custom_fcm`: build the same minimal FCM v1 payload locally and send directly. +5. FCM errors update device state and attempt status. + +Do not block notification delivery creation on FCM or relay availability. + +Relay pacing is shared with APNs: one token bucket (`notifications.push.relay_send_rate`, default 5 req/sec) covers both dispatchers because both consume the same relay account quota. See [`02-apns-relay.md`](./02-apns-relay.md) "Relay pacing". + +## FCM Error Handling + +The server should handle relay/FCM failures without exposing notification content. + +FCM v1 error codes worth handling explicitly: + +- `UNREGISTERED` (404): the token is no longer valid (app uninstalled or data cleared). Disable the device and require re-registration. +- `INVALID_ARGUMENT` (400): malformed request. Mark failed and surface admin diagnostics. Should be rare given the constrained payload. +- `SENDER_ID_MISMATCH` (403): the token was minted under a different Firebase project. Disable and log. +- `QUOTA_EXCEEDED` (429): retry with backoff, **honoring the `Retry-After` response header** if present (FCM uses standard HTTP semantics here). Common when high-priority data-message quotas are hit; FCM specifically throttles high-priority data messages that don't surface a user-visible notification on the device. Consider falling back to `priority: normal` after repeated 429s within a cooldown window. +- `UNAVAILABLE` (503): FCM is temporarily down. Retry with backoff. +- `INTERNAL` (500): retry with backoff. +- `THIRD_PARTY_AUTH_ERROR` (401): credential failure (relay or custom_fcm). Surface in admin diagnostics; do not retry until the admin fixes it. +- relay unavailable, FCM unavailable, or network timeout: retry with backoff. + +Errors should be visible to admins as operational status, not to normal users unless their device needs reauthorization. + +## Privacy Requirements + +The implementation must satisfy these requirements before shipping: + +- Relay requests contain no notification title or body. +- Relay requests contain no media identifiers. +- Relay requests contain no server hostname or base URL. +- Relay requests contain no profile, username, library, collection, or item names. +- Direct FCM payloads contain no notification title or body. Specifically: **no top-level `notification` object** in the FCM v1 message — data-only messages only. +- Direct FCM payloads contain no media identifiers, server hostname, server URL, profile name, username, library name, collection name, or item name. +- Relay logs redact raw FCM tokens. +- Relay logs redact authorization headers and OAuth2 access tokens. +- Relay logs do not persist full request payloads. +- Server logs redact FCM tokens, custom service account JSON contents, and generated OAuth2 access tokens. +- Badge count is disabled by default. +- Push registration and remote push provider use are opt-in. +- Device unregister disables further push attempts for that device. + +## Security Requirements + +- Relay API uses TLS only. +- Relay API keys are scoped to a relay account or install identifier. +- Relay API keys can be revoked without changing the user's Silo server authentication. +- Relay requests are rate limited by API key and coarse token hash. +- Relay supports idempotency keys to prevent duplicate FCM sends during retry. +- Relay does not accept arbitrary FCM payload JSON in v1. +- Server stores relay API keys as secrets. +- Server stores custom FCM service account JSON as a secret. +- Server never logs custom FCM service account JSON contents, generated OAuth2 access tokens, or FCM authorization headers. +- Server stores FCM registration tokens encrypted at rest where local secret storage exists. +- Admin diagnostics must not print raw FCM tokens or service account JSON. +- Custom FCM service account JSON must validate as a real Google service account JSON before being accepted (parse + sanity check `type: "service_account"` and required key fields). + +## Settings And UX + +Admin settings should explain the tradeoff plainly: + +```text +Android Push Provider + +Off + No Android remote push. Devices still receive in-app realtime updates while open. + +Silo Relay + Uses Silo's FCM relay to wake Android devices. Notification details stay on + this server. The relay receives FCM tokens, timestamps, and opaque delivery IDs. + +Custom FCM + Advanced. Send directly to Google FCM using your own Firebase service account. + Notification details still stay on this server. Only useful if you've published + your own signed Android app variant under your own Firebase project. +``` + +Device settings (shared with APNs) should describe the user-visible mode: + +```text +Private Push + Show a generic Silo notification, then fetch details from your server when + this device wakes. +``` + +Do not claim the central relay is fully self-hosted. The truthful claim is: + +```text +Notification content stays on your server. Google FCM, and the Silo relay +if selected, may still process generic wake messages needed for Android push +delivery. +``` + +## Implementation Plan + +### Task 1: Add Android Push Provider Settings + +Files likely involved: + +- `internal/api/handlers/settings.go` +- `web/src/lib/settingsManifest.ts` (or admin-scope equivalent) +- settings UI files as needed + +Add server/admin settings for: + +- provider selection +- relay endpoint +- relay API key +- custom FCM service account JSON +- custom FCM project ID +- custom FCM allowed packages +- badge sync enabled or disabled (shared with Apple settings) + +Default provider must be `off`. + +### Task 2: Extend Push Device Registration + +Files likely involved: + +- migration extending `push_devices` from the APNs spec to add FCM-specific columns and a check constraint +- `internal/notifications` package +- `internal/api/handlers/notifications.go` +- `internal/api/router.go` + +Add profile-scoped Android push device registration, token rotation, and disable APIs. + +If the APNs spec lands first, this task is purely additive (new columns nullable, new endpoint, expanded `platform` enum). + +### Task 3: Add Android Push Provider Clients + +Files likely involved: + +- `internal/notifications/fcm_relay.go` +- `internal/notifications/fcm_direct.go` +- config/settings accessors + +Implement the narrow `/v1/fcm/send` relay client. The client should not accept a free-form notification payload. + +Implement the direct FCM v1 client for `custom_fcm` using: + +- `golang.org/x/oauth2/google` for service-account-based OAuth2 token generation, or an equivalent that doesn't add a heavy Google SDK dependency. +- Cached access token with refresh-before-expiry. +- Standard `net/http` for the FCM v1 endpoint. + +Both clients use the same internal request shape and payload builder so the dispatcher code is uniform. + +### Task 4: Extend Push Fanout Worker + +Files likely involved: + +- `internal/notifications/push_fanout.go` (shared with APNs) + +Trigger Android push fanout for devices with `platform = 'android'`. Use retries and record `push_delivery_attempts` with `fcm_message_name` populated on success. + +### Task 5: Add Android Client Registration And Wake Handling + +Files live in the Android client repository, not this server repo. + +Required client behavior: + +- request `POST_NOTIFICATIONS` permission on Android 13+ +- get FCM registration token via `FirebaseMessaging.getInstance().getToken()` +- register token with the user's server +- handle `FirebaseMessagingService.onNewToken` for token rotation +- store `server_device_id` locally, scoped to the active server account +- handle `onMessageReceived` for both foreground and background data messages +- fetch notification metadata after wake/open +- unregister or disable on sign-out / profile removal / app data clear + +### Task 6: Add Admin Diagnostics + +Expose high-level status (extending the APNs diagnostics surface): + +- provider enabled/disabled +- number of registered Android devices +- last relay success +- last relay failure code +- custom FCM credential presence and project/package status +- last custom FCM success +- last custom FCM failure code +- token/package/project mismatch warnings + +Do not expose raw FCM tokens or service account JSON. + +## Validation Plan + +Use minimal verification while the work is still design-only. When implemented, verify: + +- relay-disabled servers never call the relay +- `custom_fcm` servers never call the relay +- device registration is profile-scoped and idempotent +- FCM tokens are redacted in logs +- relay request bodies contain no titles, body text, item IDs, server URLs, or profile names +- direct FCM payloads contain no `notification` object and no titles, body text, item IDs, server URLs, or profile names +- custom FCM service account JSON is redacted in logs +- generated OAuth2 access tokens are redacted in logs +- creating one `notification_delivery` sends at most one push per enabled device +- FCM token rotation updates the stored token without duplicating devices +- `UNREGISTERED` and `SENDER_ID_MISMATCH` disable the affected device +- `QUOTA_EXCEEDED` triggers backoff +- app wake fetches notification metadata from the user's server +- server offline after push leaves only the generic notification visible + +## Open Questions + +- Should official Silo Android builds use one Firebase project per build channel (debug / staging / prod), or share? +- Should badge counts (Android notification dots / numbers) be allowed in `private_push`, or deferred? +- Should the relay store hashed FCM token aliases to avoid sending raw tokens on every request? Defer to v2 unless rate-limit pressure justifies it. +- Should the server fall back to `priority: normal` after repeated `QUOTA_EXCEEDED` to keep deliveries flowing at the cost of latency? Recommendation: yes, with a cooldown timer. +- Should we also add a Huawei Mobile Services (HMS) push path for non-Google Android devices? Out of scope v1; could mirror this spec as `04-hms-relay.md` later. diff --git a/docs/superpowers/plans/notifications/04-outbound-webhooks.md b/docs/superpowers/plans/notifications/04-outbound-webhooks.md new file mode 100644 index 00000000..2f84fcdf --- /dev/null +++ b/docs/superpowers/plans/notifications/04-outbound-webhooks.md @@ -0,0 +1,731 @@ +# Outbound Webhooks Spec + +**Date:** 2026-04-28 +**Status:** Draft +**Scope:** Profile-scoped outbound webhook destinations for Silo notifications. Native Discord embed type and generic JSON+HMAC type. +**Depends On:** +- [`00-architecture-overview.md`](./00-architecture-overview.md) +- [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) + +## Summary + +Silo should let each profile send their notifications to user-chosen webhook destinations: a Discord channel, a Slack incoming webhook, a personal automation service, or any HTTP endpoint that accepts JSON. Webhooks are configured per profile, support per-webhook reason filters, and include full notification content by default — the profile chose the destination, so trust is implicit. + +Two webhook types ship in v1: + +- **Discord** — native Discord embed payload. The user pastes a Discord channel webhook URL; Silo POSTs Discord-formatted embeds. +- **Generic** — canonical Silo JSON, HMAC-SHA256 signed via `X-Silo-Signature`. Suitable for Slack-incoming-webhook-style targets, custom user automations, or any HTTP endpoint. + +## Why this is different from APNs / FCM + +The privacy model inverts. APNs and FCM relay paths route through infrastructure operators (Apple, Google, Silo's relay) that the user did not directly choose. Therefore those paths carry no notification content — only opaque IDs. + +Outbound webhooks go to a destination the **profile explicitly configured**. By choosing the URL, the profile is consenting to send notification content to that endpoint. There is no privacy benefit to gating content out of webhooks the profile set up themselves; doing so would just make the feature useless ("New episode! [check Silo]" is no better than what's already in the app). + +The trust model is therefore: + +- Profile chose the URL → notification content is included by default. +- Discord embeds get series title, episode title, season/episode numbers, and poster. +- Generic webhooks get the same data in canonical JSON, plus an HMAC signature derived from a per-webhook signing secret. +- Silo still enforces privacy guardrails the profile cannot opt out of: HTTPS only, no localhost / RFC1918 destinations by default, redacted logs, retry/disable on persistent failure. + +## Decision + +Add per-profile webhook destinations, configurable through the profile's notification preferences UI: + +- Profile creates a webhook by pasting a URL and optionally a name and reason filters. +- Webhook type is detected from the URL or selected explicitly: `discord` (matches Discord webhook URL pattern) or `generic`. +- Silo POSTs a typed payload to the URL whenever a `notification_deliveries` row commits for that profile and the webhook's reason filters allow it. +- Failed deliveries are retried with exponential backoff and disabled after a configurable consecutive-failure threshold. +- Each webhook has a profile-visible status: enabled, last success, last failure code, consecutive failure count. + +## Goals + +- Let a profile send Silo notifications to Discord, Slack, or any user-chosen URL. +- Render natively in Discord without the user copy-pasting JSON templates. +- Provide a stable, signed, well-documented generic format for everything else. +- Keep the feature profile-scoped: a profile only sees and controls their own webhooks. +- Add no new infrastructure dependencies (no relay needed; webhooks are direct outbound HTTP from the user's Silo server). +- Make failure modes legible to the profile so they can fix a broken webhook themselves. + +## Non-Goals + +- Server-wide / admin-level webhooks for system events (scan complete, library health, etc.). Out of scope v1; Silo's existing realtime hub already handles operational events. +- Webhook destinations that require OAuth (Slack apps, Discord bots, Microsoft Teams Adaptive Cards). Webhook-style endpoints with an unauth'd POST URL are sufficient for v1. +- Per-event-type custom payload templates. The Discord embed shape and the generic JSON shape are fixed in v1; richer customization is v2. +- Webhook retry windows beyond 24 hours. After ~24h of consecutive failures, the webhook is auto-disabled and the profile is notified in-app. +- Multiple webhook destinations per webhook row (e.g., one row firing to two URLs). One URL per row. +- Two-way webhooks. Silo only POSTs out; it does not consume webhook responses for state. + +## Trust Model + +The profile chose the destination URL. By creating a webhook, the profile consents to send full notification content (titles, posters, episode metadata) to that URL. + +What we still don't trust the profile about: + +- **HTTPS:** Required. `http://` URLs are rejected. +- **Local / private destinations:** Rejected by default. Specifically, the URL host must resolve to an address outside all of the following ranges: + - **IPv4 private/special:** `0.0.0.0/8`, `10.0.0.0/8`, `100.64.0.0/10` (CGNAT / RFC6598), `127.0.0.0/8`, `169.254.0.0/16`, `172.16.0.0/12`, `192.0.0.0/24` (IETF protocol assignments), `192.0.2.0/24` / `198.51.100.0/24` / `203.0.113.0/24` (TEST-NET-1/2/3), `192.88.99.0/24` (deprecated 6to4 anycast), `192.168.0.0/16`, `198.18.0.0/15` (benchmarking, RFC 2544), `224.0.0.0/4` (multicast), `240.0.0.0/4` (reserved future use) + - **IPv6 private/special:** `::/128` (unspecified), `::1/128` (loopback), `fc00::/7` (ULA), `fe80::/10` (link-local), `2001:db8::/32` (documentation), `64:ff9b::/96` (NAT64) + - **IPv4-mapped IPv6:** `::ffff:0:0/96` — the most likely real-world bypass. A literal `::ffff:127.0.0.1` resolves to loopback when the connection is established but bypasses naive IPv4-only checks. The validator must unwrap v4-mapped addresses and re-check against the IPv4 deny set. + - **DNS names that resolve to any of the above** at registration time *and* at delivery time (DNS rebinding mitigation; see below). + + An admin-only setting `notifications.webhooks.allow_private_destinations = true` may be set for dev environments. It applies globally to the server and is intended only for development. +- **Server URL leakage:** Webhook payloads must **not** include the user's Silo server hostname, base URL, or absolute artwork URLs that include the server origin. Posters in Discord embeds use a profile-token-signed proxy URL (or, for v1, the embed omits images if the server URL would leak). Generic webhooks include only relative paths and let the receiver fetch from the server using their own knowledge of the server URL. +- **Relay leakage:** The webhook delivery path runs entirely on the user's own Silo server; no Silo-operated infrastructure is involved. + +## User Flow + +``` +1. Profile opens Settings -> Notifications -> Webhooks. +2. Profile clicks "Add webhook". +3. Pastes a URL like https://discord.com/api/webhooks/123/abc. +4. Silo auto-detects Discord type from URL pattern. +5. Profile names the webhook ("Family Discord") and toggles + reason filters: favorites on, watchlist on, continue_watching off, next_up off. +6. Profile clicks "Test", Silo POSTs a sample notification. +7. Discord channel shows the test embed; profile clicks "Save". +8. From now on, matching new-episode notifications post to Discord. +9. If the webhook URL becomes invalid (Discord deletes it), Silo: + a. Marks each 404'd delivery failed immediately (non-retryable 4xx); + transient network failures instead retry with backoff for ~24h. + b. After 3 consecutive non-retryable-4xx deliveries (or the retry + threshold for persistent network failures), auto-disables the webhook. + c. Posts an in-app notification to the profile: "Your 'Family Discord' + webhook stopped working. Last error: 404. Edit settings to fix." +``` + +## Data Model + +### `notification_webhooks` + +Purpose: profile-scoped webhook destination. + +Columns: + +- `id text primary key` +- `user_id integer not null` (matches `users.id integer`) +- `profile_id text not null` +- `name varchar(64) not null` — user-friendly label, capped to bound row size +- `type text not null` — `'discord'` or `'generic'` +- `url_ciphertext bytea not null` — destination URL, encrypted at rest +- `url_host varchar(253) not null` — denormalized host (no path) for diagnostics and validation logs (253 = max DNS host length) +- `signing_secret_ciphertext bytea` — null for `discord` (Discord webhooks don't sign); required for `generic` +- `enabled boolean not null default true` +- `notify_favorites boolean not null default true` +- `notify_watchlist boolean not null default true` +- `notify_continue_watching boolean not null default true` +- `notify_next_up boolean not null default true` +- `consecutive_failures integer not null default 0` +- `disabled_reason varchar(256)` — populated when auto-disabled +- `last_success_at timestamptz` +- `last_failure_at timestamptz` +- `last_failure_status integer` +- `last_failure_message varchar(256)` — short, non-sensitive diagnostic, e.g., "404 Not Found" +- `created_at timestamptz not null default now()` +- `updated_at timestamptz not null default now()` + +Constraints: + +- `(profile_id, name)` is unique per profile to prevent duplicate-name confusion. +- `CHECK (type IN ('discord', 'generic'))`. +- `CHECK (type = 'discord' OR signing_secret_ciphertext IS NOT NULL)` — generic webhooks must have a signing secret. + +Indexes: + +- `(profile_id)` for the listing endpoint. +- `(profile_id, enabled)` partial index where `enabled = true` for the dispatcher hot path. + +Notes: + +- URLs and signing secrets are encrypted using the same local secret-storage facility as other sensitive columns (e.g., APNs tokens). If that facility doesn't exist yet, this spec lands the columns as plain `bytea` containing UTF-8 bytes for v1; an explicit follow-up adds the encryption layer. +- `disabled_reason` is set when the system auto-disables; cleared when the profile re-enables. + +### `webhook_delivery_attempts` + +Purpose: operational record of webhook attempts. + +Columns: + +- `id text primary key` +- `notification_delivery_id text not null references notification_deliveries(id) on delete cascade` +- `webhook_id text not null references notification_webhooks(id) on delete cascade` +- `attempt_number integer not null` +- `attempted_at timestamptz not null default now()` +- `next_retry_at timestamptz` +- `http_status integer` +- `outcome text not null` — `'pending'`, `'delivered'`, `'retrying'`, `'failed'`, `'auto_disabled'` +- `failure_message varchar(256)` — short diagnostic (e.g., HTTP status text, DNS error class). Must not include payload contents. + +Constraints: + +- `(webhook_id, notification_delivery_id, attempt_number)` is unique to prevent double-claims under retry concurrency. +- `CHECK (outcome IN ('pending', 'delivered', 'retrying', 'failed', 'auto_disabled'))`. +- index on `(webhook_id, attempted_at desc)` for per-webhook history listing. +- index on `(outcome, next_retry_at)` for the retry worker. + +Retention: + +- keep `delivered` rows for ~7 days +- keep `failed`/`auto_disabled` rows for ~30 days for profile-visible debugging +- keep `pending`/`retrying` rows until they resolve + +## Server API Surface + +All endpoints are profile-scoped and require `X-Profile-Id` middleware. + +### List Webhooks + +```http +GET /api/v1/notifications/webhooks +``` + +Response: + +```json +{ + "webhooks": [ + { + "id": "01J...", + "name": "Family Discord", + "type": "discord", + "url_host": "discord.com", + "enabled": true, + "notify_favorites": true, + "notify_watchlist": true, + "notify_continue_watching": false, + "notify_next_up": false, + "last_success_at": "2026-04-27T18:32:11Z", + "last_failure_at": null, + "last_failure_status": null, + "last_failure_message": null, + "consecutive_failures": 0, + "disabled_reason": null + } + ] +} +``` + +Note the response **never includes the full URL or signing secret**. It returns only `url_host` so the profile can identify which Discord/Slack/etc. the webhook points at without leaking the secret URL token (which functions as the auth credential for Discord webhooks). + +### Create Webhook + +```http +POST /api/v1/notifications/webhooks +``` + +Request: + +```json +{ + "name": "Family Discord", + "url": "https://discord.com/api/webhooks/123456/abcdef-token", + "type": "discord", + "notify_favorites": true, + "notify_watchlist": true, + "notify_continue_watching": false, + "notify_next_up": false +} +``` + +Rules: + +- `type` is optional; if omitted, the server detects Discord URLs via pattern (`https://discord.com/api/webhooks/{id}/{token}` or `https://discordapp.com/...`). All other URLs default to `generic`. +- For `generic`, the server generates a random signing secret (32 bytes, base64) and returns it **once** in the response. The profile is responsible for storing it on the receiving service. +- URL must pass private-destination guards. +- URL must use `https`. +- `name` length capped at 64 chars; reject empty or whitespace-only. +- Per-profile cap: max 10 webhooks (configurable via admin setting; documented as `notifications.webhooks.max_per_profile`). + +Response: + +```json +{ + "id": "01J...", + "name": "Family Discord", + "type": "discord", + "url_host": "discord.com", + "enabled": true, + "notify_favorites": true, + "notify_watchlist": true, + "notify_continue_watching": false, + "notify_next_up": false, + "signing_secret": null +} +``` + +For `generic`: + +```json +{ + "id": "01J...", + "name": "My Slack Webhook", + "type": "generic", + "url_host": "hooks.slack.com", + "enabled": true, + "notify_favorites": true, + "notify_watchlist": true, + "notify_continue_watching": true, + "notify_next_up": true, + "signing_secret": "base64-encoded-secret-shown-once" +} +``` + +`signing_secret` is **only** returned at create time and is never re-fetchable. Rotation requires a separate endpoint. + +### Update Webhook + +```http +PUT /api/v1/notifications/webhooks/{id} +``` + +Request fields are all optional; included fields are updated: + +```json +{ + "name": "Renamed", + "enabled": true, + "notify_favorites": false, + "url": "https://discord.com/api/webhooks/.../new-token" +} +``` + +Updating the URL re-validates against the private-destination guard and resets `consecutive_failures` to 0. + +### Delete Webhook + +```http +DELETE /api/v1/notifications/webhooks/{id} +``` + +Idempotent. Cascades to `webhook_delivery_attempts`. + +### Rotate Signing Secret (generic only) + +```http +POST /api/v1/notifications/webhooks/{id}/rotate-secret +``` + +Generates and returns a new signing secret. The profile must update the receiving service to use the new secret. + +### Test Webhook + +```http +POST /api/v1/notifications/webhooks/{id}/test +``` + +Synchronously POSTs a sample payload to the destination and returns the result: + +```json +{ + "ok": true, + "http_status": 204, + "duration_ms": 187 +} +``` + +or: + +```json +{ + "ok": false, + "http_status": 404, + "duration_ms": 234, + "message": "404 Not Found" +} +``` + +The test payload is identical in shape to a real notification but is clearly marked (e.g., embed footer text "Silo test notification" for Discord; `"test": true` field in generic). Test sends do not consume the retry/auto-disable counters. + +## Payload Formats + +### Discord + +The Discord webhook API documents the `POST /api/webhooks/{id}/{token}` endpoint, which accepts a JSON body with optional `content` (plain text) and `embeds` (rich cards). Silo sends embeds only; no `content` field, so the message renders cleanly without a leading text line. + +**v1 payload (text-only, no images):** + +```json +{ + "embeds": [ + { + "title": "Severance — S2 E1: Hello, Ms. Cobel", + "description": "New episode available on Silo", + "color": 5814783, + "footer": { + "text": "Silo • Severance" + }, + "timestamp": "2026-04-28T12:34:56Z", + "fields": [ + { "name": "Reason", "value": "Favorited & Continue Watching", "inline": true }, + { "name": "Season", "value": "2", "inline": true }, + { "name": "Episode", "value": "1", "inline": true } + ] + } + ], + "username": "Silo" +} +``` + +**v1.5 payload (with image proxy, additive):** + +In v1.5, the builder adds `image` (poster) and `avatar_url` (Silo mark) fields, both pointing at `media.discord-cdn-proxy.silo.app` URLs. **In v1, these fields MUST be omitted entirely** — the builder must not fall back to absolute URLs that include the user's server origin. The privacy contract is broken if Discord sees the user's server URL in any embed field. + +Notes on the Discord payload: + +- **v1 builder rule:** the `image` field, the embed's `url` field, and the top-level `avatar_url` field must be omitted. Discord renders the embed without an image; the channel webhook's configured avatar is used. There is no fallback to the user's server URL. +- **v1.5 builder rule:** `image.url` and `avatar_url` use a CDN proxy hosted at `media.discord-cdn-proxy.silo.app`. The proxy is a small Silo-operated service that takes a short-lived signed URL token (issued by the user's server when constructing the webhook payload) and server-side streams the corresponding asset from the user's server. The leak vector being mitigated is *Discord's own infrastructure* fetching and caching the embed image — Discord's CDN (`media.discordapp.net`) fetches once from whatever origin the embed names; end-users only ever see the cached `media.discordapp.net` URL. The proxy ensures the origin Discord fetches from is the proxy, not the user's server. +- **Discord embed limits** (apply to both v1 and v1.5; payload builder must enforce): + - 6,000 char total across `title` + `description` + `field.name` + `field.value` + `footer.text` per embed + - `title`: 256 chars + - `description`: 4,096 chars + - `fields`: 25 max; `field.name` 256 chars, `field.value` 1,024 chars + - `footer.text`: 2,048 chars + - 10 embeds per message (Silo sends one) + - **Truncation policy:** prefer to truncate `description` first (with ellipsis), then field values. Never truncate `title`. If even truncated content exceeds the 6,000-char total, drop fields right-to-left until under cap. +- `color` is a Discord embed accent color encoded as decimal RGB. Silo picks per-reason: favorite=5814783 (Silo brand purple), watchlist=3066993 (green), continue_watching=15844367 (yellow), next_up=15158332 (red). +- `username` overrides the Discord webhook's default name to "Silo". +- The webhook URL itself is the auth — Discord's webhook tokens are bearer credentials in the URL path. This is why we never return the URL on read. + +The Discord embed structure follows Discord's webhook API. See [Discord embed object docs](https://discord.com/developers/docs/resources/channel#embed-object) for the full schema. + +### Generic + +JSON body, signed via HMAC-SHA256 over the raw bytes Silo sends. Receivers verify against the literal bytes they received — no canonicalization required on either side. + +Headers: + +- `Content-Type: application/json` +- `User-Agent: Silo-Webhook/1.0` +- `X-Silo-Event: notification.created` +- `X-Silo-Webhook-Id: 01J...` — the webhook row ID +- `X-Silo-Delivery-Id: 01J...` — the underlying `notification_deliveries.id` +- `X-Silo-Timestamp: 1714299296` — Unix epoch seconds, integer +- `X-Silo-Signature: t=1714299296,v1=` + - HMAC-SHA256 of the byte string `{X-Silo-Timestamp}.{request body bytes}` using the per-webhook signing secret. The result is hex-encoded. + - Format follows Stripe's signing convention so receivers can use existing libraries (Stripe Go SDK, `stripe-signature` ports, etc.). + - **The timestamp value in the body's `timestamp` field is informational and may be RFC3339 for human readability; only the `X-Silo-Timestamp` header value (Unix epoch integer) participates in the HMAC computation.** + +Body: + +```json +{ + "event": "notification.created", + "delivery_id": "01J...", + "webhook_id": "01J...", + "timestamp": "2026-04-28T12:34:56Z", + "version": 1, + "test": false, + "profile_id": "profile-1", + "library_id": 7, + "type": "episode.available", + "reason_flags": { + "favorite": true, + "watchlist": false, + "continue_watching": true, + "next_up": true + }, + "series": { + "id": "series-123", + "title": "Severance" + }, + "episode": { + "id": "episode-456", + "title": "Hello, Ms. Cobel", + "season_number": 2, + "episode_number": 1 + } +} +``` + +Notes on the generic payload: + +- The body is canonicalized (sorted keys, UTF-8 encoded) before HMAC computation so the receiver can verify deterministically. +- `profile_id` is included so the receiver can route by profile (one Slack channel per profile, etc.). It is the profile's own ID and is acceptable to share with the destination the profile chose. +- No server URL, no absolute poster URLs, no library name. The receiver cannot link this back to the user's server hostname unless the profile has separately told them. +- `version: 1` allows future schema changes without breaking existing receivers. +- `test: true` distinguishes test sends from real sends. + +### Signature Verification (generic) + +Receivers verify by: + +``` +parts = X-Silo-Signature.split(",") +ts = parts.find("t=").value // Unix epoch integer (string form) +v1 = parts.find("v1=").value // hex string + +raw_body = literal request body bytes (do NOT re-parse and re-serialize JSON) +expected = hex(hmac_sha256(secret, ts + "." + raw_body)) + +if constant_time_compare(v1, expected) and abs(now_epoch - parse_int(ts)) < 300: + accept +else: + reject +``` + +Key rules: + +- Sign and verify the **literal request body bytes**, not a re-canonicalized form. This eliminates JSON-canonicalization ambiguity (number formatting, escape rules, key ordering) and matches Stripe's pattern. +- `ts` and `now_epoch` are both Unix epoch seconds (integer). The timestamp window of 300 seconds (5 minutes) prevents replay; receivers may tighten if they wish. +- Use a constant-time comparator (`hmac.equal`, `crypto/subtle.ConstantTimeCompare`, etc.) to avoid timing attacks. + +### Signing Secret Rotation + +`POST /api/v1/notifications/webhooks/{id}/rotate-secret` generates a new signing secret and returns it once. After rotation: + +- The new secret takes effect immediately for all subsequent deliveries. +- **Pending and retrying delivery attempts re-sign with the current secret on each retry**, not the secret that was active when the attempt was first enqueued. This means receivers who have updated to the new secret will accept the retried delivery; receivers still on the old secret will reject it. The expected operational pattern is: profile rotates the secret and updates their receiver atomically (or near-atomically); brief retry-window mismatches are acceptable. +- Silo does **not** keep the old secret around after rotation. There is no dual-acceptance window. + +## Delivery Semantics + +### Send path + +1. The fanout transaction in [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md) commits the `notification_deliveries` row **and**, in the same transaction, one `webhook_delivery_attempts` row with `outcome = 'pending'` per enabled, reason-matching webhook (the dispatch outbox). Two filters apply at enqueue time: + - **Type deny list.** `webhook.auto_disabled` deliveries never enqueue webhook attempts — a webhook auto-disable notice must never be re-dispatched as a webhook, otherwise a broken webhook would generate an auto-disable notification that fires another webhook attempt that fails again, looping. Other notification types may join this list as they're added. + - **Per-webhook reason filter** (`notify_favorites`, etc.). No matching reason, no attempt row. +2. `WebhookDispatcher.Dispatch(delivery)` runs post-commit and claims the delivery's `pending` attempt rows (`FOR UPDATE SKIP LOCKED`). +3. For each claimed attempt: + - Construct the payload (Discord embed or generic JSON). + - POST the payload with a 10-second total timeout. + - On success (HTTP 2xx): mark attempt `delivered`, update webhook `last_success_at`, reset `consecutive_failures = 0`. + - On failure: mark attempt `retrying` with `next_retry_at`, increment `consecutive_failures` on the webhook. +4. The retry worker reads `webhook_delivery_attempts WHERE (outcome = 'retrying' AND next_retry_at <= now()) OR (outcome = 'pending' AND attempted_at <= now() - interval '60 seconds')` — the second arm is outbox recovery for attempts whose post-commit dispatch never ran (process crash). +5. When `consecutive_failures` exceeds the threshold (default 10) over a span > 24h, the webhook is auto-disabled (`enabled = false`, `disabled_reason = '...'`), and an in-app notification is posted to the profile via the in-app inbox. + +### Preference precedence (relationship to `notification_preferences`) + +Profile-level `notification_preferences.notify_*` flags are a **hard gate** applied during fanout (see [`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md)). A delivery row is never created if all matching reasons are disabled at the profile level — so the dispatcher never sees those events. + +Per-webhook `notify_*` flags are an **additional filter** applied during dispatch. They can narrow what fires for a specific webhook, but they cannot re-enable a reason the profile has globally disabled. The frontend must reflect this: if a profile turns off `notify_continue_watching` globally, the per-webhook continue-watching checkbox should appear disabled with explanatory text rather than implying the user can re-enable it for one destination. + +### Retry schedule + +Exponential backoff, capped at 24h total: + +| Attempt | Delay since first attempt | +|---|---| +| 1 | 0 (immediate) | +| 2 | 30s | +| 3 | 2m | +| 4 | 10m | +| 5 | 30m | +| 6 | 2h | +| 7 | 6h | +| 8 | 12h | +| 9 | 18h | +| 10 | 24h, then auto-disable | + +Each attempt has a 10s total timeout. + +**4xx skip-retry:** 4xx HTTP responses (except `408 Request Timeout`, `425 Too Early`, `429 Too Many Requests`) are deterministic destination-side rejections — retrying *this delivery* won't help, so the attempt is marked `failed` immediately without walking the 10-attempt schedule. Auto-disable, however, requires **3 consecutive deliveries** to fail with a non-retryable 4xx (tracked via `consecutive_failures` plus the failure-status class) before the webhook is disabled. A single 4xx is not proof the webhook is dead: destination-side WAF/CDN blips (Discord behind Cloudflare) intermittently return 403/400 for valid webhooks, and instant disable on one such blip would force pointless manual re-enables. Three consecutive deterministic rejections is strong evidence the URL is actually gone. The profile is notified in-app per "Auto-Disable Notification" below. + +`429 Too Many Requests` honors the `Retry-After` header if present (overriding the schedule above for that attempt). + +### Concurrency + +The dispatcher fans webhook deliveries in parallel within a small bounded pool (e.g., 16 workers per server process). One slow webhook destination cannot block other deliveries. + +### Test delivery isolation + +`POST /webhooks/{id}/test` sends synchronously, does not write to `webhook_delivery_attempts`, does not affect `consecutive_failures`, and returns the HTTP result inline. + +## Privacy Requirements + +- Webhook URLs are encrypted at rest where the local encryption facility supports it. +- Webhook URLs are never returned in API responses (only `url_host`). +- Generic signing secrets are returned only at create / rotate time; never readable after. +- Logs redact full URLs (host only) and full signing secrets. +- Logs do not persist webhook payload bodies. Failure logs may include HTTP status text. +- Webhook payloads must not include the Silo server URL or absolute artwork URLs that include the server origin. +- An admin must not be able to read another profile's webhook URLs or signing secrets. + +## Security Requirements + +- HTTPS-only destinations. +- Private-destination guard on URL submission and on each delivery (re-resolve to catch DNS rebinding to private IPs). +- HMAC-SHA256 with per-webhook 32-byte signing secrets for `generic`. +- Signing secret returned only at create / rotate; rotation immediately invalidates the previous secret. +- Per-profile cap on webhook count (default 10). +- Per-profile rate limit on webhook delivery (e.g., 60 deliveries / 60 seconds / profile across all webhooks). Notifications that exceed the limit are still durably stored in the inbox; webhooks just don't fire for them. Logged so admins can spot runaway scenarios. +- TLS verification is enforced and not user-overridable. +- HTTP redirects are followed (max 3 hops) but the final URL must still pass the private-destination guard at each hop. + +## DNS Rebinding Mitigation + +A naive implementation that resolves the URL once at registration time is vulnerable to DNS rebinding: the host could resolve to a public IP at registration and a private IP at delivery time. Mitigation: + +- At delivery time, resolve the host yourself, validate the resolved IPs against the private deny set, then connect using the validated IP (or a controlled `net.Dialer` that re-validates the address it's about to connect to). +- Do not let the standard library transparently re-resolve on each request. +- The `internal/notifications/webhook_http.go` HTTP client should use a custom `Dialer.Control` callback that inspects the resolved address and refuses connections to private ranges. + +## Settings And UX + +Profile-level settings: + +```text +Webhooks + +Add webhook + Send Silo notifications to a webhook URL. + Discord URLs render as native Discord embeds. + Other URLs receive signed JSON. + +[Family Discord] ✓ enabled + discord.com + ✓ Favorites ✓ Watchlist ☐ Continue watching ☐ Next up + Last success: 2 hours ago + [ Test ] [ Edit ] [ Delete ] + +[Slack: #media] ✓ enabled + hooks.slack.com + ✓ all reasons + Last failure: 3 minutes ago — 404 Not Found + ⚠ This webhook is failing. Check the destination URL. + [ Test ] [ Edit ] [ Delete ] + ++ Add webhook +``` + +Admin-level settings (server-wide guards, default values): + +- `notifications.webhooks.max_per_profile` (default 10) +- `notifications.webhooks.allow_private_destinations` (default false; for dev environments) +- `notifications.webhooks.deliveries_per_minute_per_profile` (default 60) + +## Implementation Plan + +### Task 1: Schema + +Files: + +- new migration under `migrations/` adding `notification_webhooks` and `webhook_delivery_attempts` tables. + +### Task 2: Webhook Repository And Service + +Files: + +- `internal/notifications/webhook_repo.go` +- `internal/notifications/webhook_service.go` (CRUD, validation, signing-secret handling) + +Includes: + +- URL validation (https, private-destination guard) +- Discord URL detection +- signing secret generation +- per-profile count enforcement + +### Task 3: HTTP Client With Address Guard + +Files: + +- `internal/notifications/webhook_http.go` + +Custom HTTP client with: + +- 10-second timeout +- TLS verification non-overridable +- `Dialer.Control` callback that re-validates the resolved IP at connect time +- bounded redirect handling +- structured error reporting (DNS error class, TCP, TLS, HTTP status) for `failure_message` + +### Task 4: Payload Builders + +Files: + +- `internal/notifications/webhook_payload_discord.go` +- `internal/notifications/webhook_payload_generic.go` + +Each pure function `func Build(delivery NotificationDelivery, hook Webhook) ([]byte, error)`. No side effects, easy to unit test. + +### Task 5: Dispatcher Integration + +Files: + +- `internal/notifications/webhook_dispatcher.go` + +Implements the channel `Dispatcher` interface from `00-architecture-overview.md`. Loads enabled webhooks for the delivery's profile, applies reason filters, posts payloads, records attempts. + +### Task 6: Retry Worker + +Files: + +- `internal/notifications/webhook_retry_worker.go` + +Polls `webhook_delivery_attempts WHERE outcome = 'retrying' AND next_retry_at <= now()` with `FOR UPDATE SKIP LOCKED`. Re-dispatches and updates state. Handles auto-disable. + +### Task 7: API Handlers And Routes + +Files: + +- `internal/api/handlers/notifications_webhooks.go` +- `internal/api/router.go` + +Routes: + +- `GET /api/v1/notifications/webhooks` +- `POST /api/v1/notifications/webhooks` +- `PUT /api/v1/notifications/webhooks/{id}` +- `DELETE /api/v1/notifications/webhooks/{id}` +- `POST /api/v1/notifications/webhooks/{id}/test` +- `POST /api/v1/notifications/webhooks/{id}/rotate-secret` + +All require `RequireProfile` middleware. + +### Task 8: Frontend + +Files: + +- `web/src/pages/settings/NotificationWebhooksSettings.tsx` +- `web/src/hooks/queries/notification-webhooks.ts` +- form components, status pills, test-result display + +Includes: + +- "Show signing secret once" UX with explicit "I've saved it" confirmation. +- Show secret-rotation flow. +- Last-failure status with admin-level message ("404 Not Found"), not stack traces. +- Test button. +- Per-reason filter checkboxes. + +### Task 9: Auto-Disable Notification + +When auto-disable fires, create a `notification_deliveries` row with `type = "webhook.auto_disabled"`, including the webhook name and last failure code. The profile sees this in their inbox so a broken webhook doesn't fail silently. + +### Task 10: Optional CDN Proxy For Discord Images + +Files: + +- separate repo (e.g., `silo-discord-cdn-proxy`) +- `internal/notifications/webhook_image_signer.go` for issuing short-lived signed URL tokens + +Out of scope for v1 unless Discord image rendering is wanted in the first release. v1 ships embeds without images; v1.5 adds the proxy and image URLs. + +## Validation Plan + +- Unit tests for URL validators (HTTPS, private-IP guard, DNS resolution). +- Unit tests for payload builders (Discord shape, generic canonicalization, HMAC determinism). +- Integration test: round-trip a generic webhook against a stubbed receiver that verifies the signature. +- Integration test: Discord embed against `https://discord.com/api/webhooks/.../test-only` (or recorded fixture) — manual verification only since Discord doesn't offer a sandbox. +- Concurrency test: 100 deliveries in parallel; verify retry rows aren't double-counted. +- Auto-disable test: simulate 3 consecutive deliveries returning 404 and verify the webhook is disabled with the correct in-app notification; verify 1-2 isolated 4xx blips do **not** disable it; verify persistent timeouts exhaust the 10-attempt schedule before disabling. +- Outbox recovery test: commit deliveries with `pending` attempt rows, skip the inline dispatcher, and verify the retry worker sends them after the 60s recovery window. +- Manual verification: + - Create a Discord webhook URL. + - Add it to a profile. + - Trigger a release event. + - Confirm the embed renders correctly in Discord. + - Disable the Discord webhook on Discord's side. + - Trigger another event. + - Verify the webhook auto-disables after retries. + - Verify the profile receives an in-app "webhook auto-disabled" notification. + +## Open Questions + +- **Discord image proxy scope.** Build it in v1, or ship v1 without images and add v1.5? Recommendation: ship without images in v1 — text-only embeds are useful and require no new infra. Add the proxy in v1.5 if user feedback wants images. +- **Slack and Microsoft Teams native types.** Should we add `slack` and `teams` types alongside `discord` and `generic`? Both Slack incoming webhooks and Teams Adaptive Cards have their own JSON shapes. Recommendation: `generic` works for Slack (Slack incoming webhooks accept arbitrary JSON; users can build their own message format with a Workflow). Teams requires Adaptive Card JSON specifically. Defer both to v2 unless concrete user demand emerges. +- **Per-event-type templating.** Some users will want to customize the embed text. Recommendation: defer to v2; v1 fixed shapes ship faster and most users will accept them. +- **Webhook "channels" beyond episode.available.** Should webhook payloads cover other event types (release-aggregated, server announcements)? V1 only fires on `notification.created` for `episode.available`. Other types are forward-compatible via the `event` and `type` fields. +- **Outgoing IP.** Webhook deliveries originate from the user's Silo server IP. Do we need an option to route through a proxy? Recommendation: not v1; document that deliveries come from the server's egress IP. +- **Backpressure.** What happens when a profile's webhook is slow but reachable, blocking delivery for an hour at a time? The 10s timeout caps individual deliveries; bounded worker pool caps fanout concurrency. A single bad webhook shouldn't degrade other webhooks because each delivery runs independently. diff --git a/docs/superpowers/plans/notifications/README.md b/docs/superpowers/plans/notifications/README.md new file mode 100644 index 00000000..9e8c0cc9 --- /dev/null +++ b/docs/superpowers/plans/notifications/README.md @@ -0,0 +1,88 @@ +# Silo Notifications — Design Index + +> **Provenance:** Imported 2026-06-11 from `ContinuumApp/continuum` (`docs/superpowers/plans/notifications/` at `04aa9266`), written 2026-04-28 before the project was renamed to Silo. Reviewed and amended 2026-06-11: product naming and all wire contracts (payload keys, headers, settings enums, localization keys, provisional domains/repo names) are normalized to Silo. Bundle topics, Android package names, and Firebase project IDs intentionally keep their current official client-build values (`com.continuum.app.*`, `continuum-prod-android`) — they are allowlist *config*, not part of this design's contract, and the shipped client builds still use them. Relay/proxy hostnames (`relay.silo.app`, `media.discord-cdn-proxy.silo.app`) are provisional. See "2026-06-11 review amendments" below for the design-level changes made during review. + +This folder collects the design work for Silo's notification system, covering durable in-app inbox + websocket realtime, Apple Push (APNs), Android Push (FCM), and outbound webhooks (Discord-native + generic). + +A self-contained visual summary of the design decisions (who/what/where/why per decision, pipeline, trust model) lives at [`design-decisions.html`](./design-decisions.html) — open it in any browser; no build or network access required. + +## Reading order + +1. **[`00-architecture-overview.md`](./00-architecture-overview.md)** — Cross-cutting overview. Read this first. Explains the channel model, fanout pipeline, preference shape, threat model, and addressing model (why a relay is required for mobile push). Links into the per-channel specs. +2. **[`01-release-events-and-inbox.md`](./01-release-events-and-inbox.md)** — Foundation. Durable `release_events` -> per-profile `notification_deliveries` fanout, the in-app inbox API, the websocket channel, and per-profile preferences. Vendor-agnostic; nothing else works without this. +3. **[`02-apns-relay.md`](./02-apns-relay.md)** — Apple push. Privacy-preserving design with three provider modes (`off` / `silo_relay` / `custom_apns`). +4. **[`03-fcm-relay.md`](./03-fcm-relay.md)** — Android push. Mirrors the APNs spec with `off` / `silo_relay` / `custom_fcm` modes using FCM v1 (OAuth2 service-account JSON). +5. **[`04-outbound-webhooks.md`](./04-outbound-webhooks.md)** — Profile-scoped outbound webhooks. Native Discord embed type and generic JSON+HMAC type. Content included by default because the profile chose the destination. + +## Status + +| Doc | Status | +|---|---| +| 00-architecture-overview | Draft (amended 2026-06-11) | +| 01-release-events-and-inbox | Draft (refined, amended 2026-06-11) | +| 02-apns-relay | Draft (refined, amended 2026-06-11) | +| 03-fcm-relay | Draft (amended 2026-06-11) | +| 04-outbound-webhooks | Draft (amended 2026-06-11) | + +Nothing in this folder has been implemented. The codebase has zero notification-related code beyond the existing realtime events hub (`internal/events/`, with the operational publisher wrapper at `internal/notifications/hub.go`), which publishes catalog/jobs events and is unrelated to the user-facing notification system designed here. + +## 2026-06-11 review amendments + +A scaling and privacy review (self-hosted servers with hundreds of users) resolved four design-level gaps. The detailed designs live in the per-doc sections; this is the index: + +1. **Back-catalog seeding + burst suppression** (`01`). First scan of a new library and the feature-enable backfill seed `episode_availability` *without* creating release events; bulk additions to existing libraries are bounded by a per-series fanout burst cap. Without this, importing a 200-episode back-catalog of a popular series would generate tens of thousands of deliveries and pushes in one scan. +2. **Durable dispatch outbox** (`01`, referenced by `02`/`03`/`04`). The fanout transaction that inserts `notification_deliveries` also inserts `pending` per-target attempt rows for the push and webhook channels. A crash between delivery commit and dispatch no longer silently loses pushes/webhooks; recovery workers drain stale pending rows. +3. **Cross-library episode dedupe** (`01`). Media items in Silo are catalog-level (`media_item_libraries` junction), so the same episode landing in "TV" and "TV 4K" shares one `episode_id`. A partial unique index on `(profile_id, episode_id)` guarantees at most one `episode.available` delivery per profile per episode across libraries. +4. **Mobile wake-fetch endpoints defined in the foundation** (`01`). `GET /api/v1/notifications/sync` (forward cursor) and `GET /api/v1/notifications/{id}` — previously referenced by `02`/`03` but defined nowhere. + +Smaller amendments: relay threat models now list the server's egress IP as residual leakage (`00`/`02`/`03`); `collapse_id`/`collapse_key` derivation is specified as per-server-keyed HMAC so the relay can't read series identity out of it (`02`/`03`); the websocket handshake uses a short-lived single-use ticket instead of a long-lived profile token in the query string (`00`/`01`); client-side relay pacing (`02`/`03`); webhook auto-disable requires 3 consecutive non-retryable 4xx instead of 1 (`04`); SSRF deny list gains `198.18.0.0/15` and `192.88.99.0/24` (`04`); interest recompute fires on watch-state transitions, not every progress tick (`01`); profile-deletion cascades and `release_events.dedupe_key` composition are specified (`01`); migration tasks now follow the repo's timestamped Goose convention (`01`). + +## Scope summary + +**In scope (v1):** + +- Episode-availability notifications (a new episode lands in a library and is relevant to a profile via favorites / watchlist / continue-watching / next-up). +- Durable per-profile inbox with read/unread state. +- Realtime websocket delivery for connected clients. +- Apple Push via opt-in Silo-operated relay or admin-supplied custom APNs credentials. +- Android Push via opt-in Silo-operated relay or admin-supplied custom FCM credentials. +- Outbound webhooks per profile, with native Discord embed rendering and generic JSON+HMAC. + +**Out of scope (v1):** + +- Movie release notifications (the trigger surface — favoriting a movie that doesn't yet exist — isn't a flow Silo supports). +- Email, SMS, or other delivery channels. +- Browser Web Push (PWA) notifications. +- Cross-profile administrator dashboards. +- Marketing or promotional notifications. +- Third-party ingest events (Plex/Sonarr/Radarr) as notification sources. + +## Cross-cutting decisions + +These hold across all channels and were resolved during spec review: + +- **Episode-only triggers in v1.** Confirmed: there's no UX path for a user to favorite/watchlist a movie before it exists in their library. +- **Per-profile webhooks include content by default.** The profile chose the destination; trust is implicit. Discord-native embeds show series and episode details; generic webhooks include the same. +- **Mobile push payloads stay opaque by default.** APNs and FCM relay paths never see notification content. The app fetches metadata from the user's own server after wake. No badge counts in v1 (deferred to user-opt-in in v2). +- **Preference shape is flat:** per-profile reason flags (favorites / watchlist / continue-watching / next-up) plus a master enable toggle on each push device and each webhook. Not a per-channel × per-reason matrix. +- **Per-platform APNs topics.** iOS, tvOS, and macOS each get their own bundle topic. Single-build collapse is a v2 concern. +- **Stateless relay v1.** No stored token aliases. Add only if abuse/rate-limit pressure justifies it. +- **Relay services live in their own repos.** This folder defines the contracts the user-server side implements; the actual relay deployment (APNs / FCM) is a separate operational repo. +- **Back-catalog never floods.** Initial library scans and the feature-enable backfill seed availability silently; per-series burst caps bound fanout for bulk additions (see `01`). +- **One delivery per episode per profile, across libraries.** Dual-quality library setups (e.g., "TV" + "TV 4K") do not double-notify. +- **Dispatch enqueue is durable (outbox).** Push and webhook sends survive a crash between delivery commit and dispatch; the websocket channel stays best-effort because the inbox snapshot covers reconnect. + +## Phasing + +Implementation should land in this order, gated by feature flags so partial deploys are safe: + +1. Schema (Phase 1 of `01-release-events-and-inbox.md`). +2. `profile_series_interest` updaters and backfill task (no fanout yet). +3. Availability seeding backfill, then release event creation (still no fanout). Seeding **must** complete before release events are enabled, or the first scan after enablement emits an event per back-catalog episode. +4. In-app fanout worker, websocket channel, inbox APIs. +5. Frontend inbox + badge + preference UI. +6. Outbound webhooks (no new infra; can ship before mobile push). +7. APNs relay + custom APNs. +8. FCM relay + custom FCM. + +The relay services (steps 7-8) require Silo to provision Apple Developer + Firebase developer accounts and host the relay services. Steps 1-6 require no new external infrastructure. diff --git a/docs/superpowers/plans/notifications/design-decisions.html b/docs/superpowers/plans/notifications/design-decisions.html new file mode 100644 index 00000000..35da199e --- /dev/null +++ b/docs/superpowers/plans/notifications/design-decisions.html @@ -0,0 +1,281 @@ + + + + + +Silo notifications — design decisions + + + +
+ +

Silo notifications — design decisions

+

Episode-availability notifications for a self-hosted, privacy-focused media server, designed to hold up with hundreds of users on one install.

+

Companion to the specs in docs/superpowers/plans/notifications/ · drafts amended 2026-06-11

+ +

How a notification flows

+

Ingest records availability, a worker fans out to interested profiles, and a durable inbox row is written before any channel fires. Every channel below the row is a transport, never a source of truth.

+
+ + + + + + + +Scan and ingest +new episode lands + +release_events +one row per episode + +Fanout worker +burst caps + eligibility + + + + +notification_deliveries +durable row, dispatched after commit + + + + + +Websocket +full content + +Apple push +opaque wake + +Android push +opaque wake + +Webhooks +your chosen URL + + + +Silo push relay +sees token + opaque IDs + + +custom credentials +skip the relay + +APNs / FCM +deliver the generic wake + + +Device wakes +fetches details from home + +
+content stays in Silo +opaque wake only, device fetches from home +full content to a destination the profile chose +
+
+ +

Trust model

+ + + + + + + +
ChannelWho sees contentWhy that's acceptable
Inbox + websocketOnly the user's own server and signed-in clientsSame auth boundary as the rest of Silo
Apple / Android pushNobody — relay and Apple/Google see a device token, timestamps, and opaque IDsThe device fetches real content from the user's server after waking
WebhooksThe destination operator (e.g., Discord)The profile pasted that URL; consent is explicit
+ +

Foundation

+
+
+

Durable inbox is the source of truth

+
+
Who
Every profile on the server
+
What
One delivery row per event; every channel is just a transport on top
+
Where
The server's own PostgreSQL
+
Why
Transports fail — the row survives, so reconnect or refresh always recovers
+
+
+
+

Preferences are flat, not a matrix

+
+
Who
Profiles tuning what they hear about
+
What
Four reason toggles plus a master switch; per-device and per-webhook filters can only narrow further
+
Where
Hard gate at fanout, extra filters at dispatch
+
Why
A per-channel by per-reason grid is unexplainable in UI and impossible to debug
+
+
+
+ +

Privacy boundaries

+
+
+

Opaque push through a hosted relay

+
+
Who
Admins who opt in; official store app builds
+
What
The relay holds the Apple and Google signing credentials and forwards a device token plus opaque IDs only
+
Where
Silo-operated service in its own repo, stateless on the request path
+
Why
Only the publisher's credentials can wake store builds; content never leaves the server — the device fetches details from home after waking
+
+
+
+

Custom credentials escape hatch

+
+
Who
Power users and forks shipping their own signed app builds
+
What
custom_apns / custom_fcm send straight to Apple or Google with the exact same minimal payload
+
Where
Entirely on the user's server
+
Why
Zero Silo infrastructure in the path, without loosening the privacy contract
+
+
+
+

Profiles, not accounts, are the unit

+
+
Who
Households sharing one login across several profiles
+
What
Preferences, devices, webhooks, inbox rows, and websocket bindings are all profile-keyed
+
Where
Every table, API route, and event envelope
+
Why
A kid's profile and a parent's profile must not see each other's activity
+
+
+
+

Webhooks carry full content, by choice

+
+
Who
Each profile, up to 10 destinations
+
What
Discord embeds or HMAC-signed JSON with series and episode details
+
Where
Direct HTTPS from the server to the URL the profile pasted
+
Why
Choosing the destination is the consent; HTTPS-only, the SSRF deny list, and no-server-origin rules stay mandatory
+
+
+
+ +

Scale and reliability

+
+
+

Fanout reads a compact interest index

+
+
Who
Servers with hundreds of users, roughly 1,000 profiles
+
What
profile_series_interest keyed by library and series, updated on favorite, watchlist, and watch transitions
+
Where
Written at interaction time, read once per release event
+
Why
Cost scales with profiles interested in one series, not with a million-episode catalog
+
+
+
+

Back-catalog imports never flood

+
+
Who
Anyone adding a new library or a full season pack
+
What
First scans seed availability silently; bulk additions cap at 3 notifications per series per scan
+
Where
Scanner seeding plus the fanout worker burst cap
+
Why
One 200-episode import with 300 interested profiles would otherwise mean about 60,000 pushes in a single scan
+
+
+
+

Dispatch enqueue is durable (outbox)

+
+
Who
Operators who care what a crash costs
+
What
Pending push and webhook attempt rows commit in the same transaction as the inbox row
+
Where
The fanout transaction; recovery workers sweep stale rows
+
Why
A crash between commit and dispatch delays sends instead of silently losing them
+
+
+
+

One delivery per episode, across libraries

+
+
Who
Dual-quality setups like TV plus TV 4K
+
What
A partial unique index on profile and episode spans libraries; the first event processed wins
+
Where
The notification_deliveries schema
+
Why
Media items are catalog-level in Silo, so the same episode landing twice must not notify twice
+
+
+
+ +

Source documents

+
    +
  • 00-architecture-overview.md — channel model, fanout pipeline, threat model, relay addressing
    • +
  • 01-release-events-and-inbox.md — durable inbox, fanout worker, seeding and burst suppression, sync API
    • +
  • 02-apns-relay.md — Apple push: hosted relay and custom APNs
    • +
  • 03-fcm-relay.md — Android push: hosted relay and custom FCM
    • +
  • 04-outbound-webhooks.md — Discord and generic JSON+HMAC webhooks
    • +
  • README.md — reading order, status, and the 2026-06-11 amendment index
    • +
+ + + +
+ +