codex/bound-transcode-segments
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
866392fecd |
feat(notifications): announce new audiobooks and ebooks on server channels (#260)
Audiobook and ebook libraries previously never entered the Recently Added pipeline: availability detection only ran for TV/movie/mixed libraries and release_events only knew episode/movie kinds, so server channels (Discord/generic webhooks) could not announce new audiobooks or ebooks. Generalize the movie path into a flat-item-kind registry (internal/notifications/item_kind.go) driving availability detection, recording, channel toggles, payload rendering, test fixtures, and the admin backfill seeder. New kinds share a kind-discriminated item_availability table; movie_availability stays as-is. Channels gain notify_new_audiobooks/notify_new_ebooks toggles (default on, additive API fields) and embeds carry the author from item_people. Flood-safe by construction: existing libraries seed silently on their first post-upgrade full scan. Extract internal/librarykind to replace the is*LibraryType helper copies that had drifted across scanner, libraryingest, and metadata (metadata's movie check silently included mixed; now spelled explicitly). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
cf0db385f3 |
Add Apple push notifications support (#255)
* Add push notifications support * fix(notifications): address push notification review findings - Gate the capability endpoint's apple_push availability on the admin delivery toggle, matching web push: Available now means setup will actually deliver. - Reject direct admin writes to push_relay_deployment_id/api_key; the relay issues them as a pair during registration and a lone write desyncs them (and poisons the next rotation request). - Purge a device's registrations under other profiles when it re-registers, so a profile switch on a shared device stops the old profile's pushes (attempts cascade); adds a DB-backed test. - Extract the shared channelDispatcher core + retry sweep and rebuild the webhook/web push/Apple push dispatchers on it instead of keeping three copies of the worker-pool/retry loop. - Deduplicate relay URL validation (admin setting + register flow) and the push outbox attempt-building loops behind shared helpers. - Cap free-text decline reasons in notification display bodies. - Fix TestHandleApplePushDisplayDB expectations to match the shared display copy (test previously failed under SILO_TEST_DATABASE_URL). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(notifications): route push relay URL writes through registration only Direct writes to notifications.push_relay_url via the admin settings endpoint bypassed the relay registration flow, letting the stored URL drift out of sync with the deployment id / API key pair the relay minted for it. Reject the URL alongside the deployment id and API key in the settings handler; POST /admin/notifications/push/relay/register remains the only path that persists all three together. The admin UI's Relay URL field now edits local draft state and is applied by the Register/Rotate action instead of the settings save. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bcf0253c09 |
feat(notifications): notify requesters of request status changes (#143)
Requests previously only notified the community server channels for submitted/approved/declined and the requester personally for fulfilled. This closes the gap and makes request posts addressable: - New request.approved / request.declined delivery types ride the operational dispatch path to the requesting profile: inbox, websocket toast, email, Discord DM, personal webhooks (gated by the existing notify_requests flag), and web push. Submitted stays broadcast-only (the requester performed the action themselves). Title/year/decline reason travel in reason_flags since no catalog item exists yet. - Request status notices are transactional: digest-mode recipients get an off-schedule early send (watermark-durable, last_digest_at left alone) instead of waiting for the digest hour. Per-episode recipients were already immediate via the dispatch nudge. - At-most-once per (profile, request, type) via a partial unique index (migration 20260612100000), mirroring the fulfilled dedupe. - Server-channel Discord request posts can @mention the requester via their OAuth-linked identity (notifications.server_channels. mention_requesters, default off). Resolved lazily in the sweep worker only when a Discord destination is about to receive the event; the ping uses content-level mention with pinned allowed_mentions, and the Discord identity never leaks into generic webhook payloads. Android/Apple clients render the new inbox types with their generic fallback until they add them. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
1e3780d4fb |
fix(notifications): address code review findings
- pin the four new sensitive setting keys (SMTP password, Discord secret/bot token, VAPID keypair) in the encryption audit test so a future drop from SensitiveSettingKeys fails CI - bound account-channel digest drains strictly before the stamped digest time so consecutive digest windows partition rows exactly, instead of recapping rows created at or after the previous stamp - keep the events websocket open when an event-frame snapshot fails, matching the writeSnapshotFrame degrade-gracefully contract - rename the seed task to Seed Content Availability to match its episode+movie seeding behavior - carry poster_source_path into realtime dispatch rows per the DeliveryRow contract Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
799754dde7 |
feat(notifications): rich Discord embeds with posters, provider links, and ratings
Upgrade all outbound Discord surfaces (personal webhooks, bot DMs, server channels, request events) from bare title/description embeds to rich ones: poster thumbnail, overview teaser, TMDB/IMDb/TVDB links, rating and genre fields, content-rating footer, and a clickable title URL. Artwork respects the v1 privacy contract via a new admin poster mode (notifications.discord.poster_mode): "provider" (default) only emits public provider-CDN URLs, "server" additionally presigns locally cached posters from this server's image storage, "off" drops images entirely. Builders never derive artwork URLs themselves; the sender layer resolves PosterURL through System.discordPosterURL. To keep provider-CDN URLs derivable after image caching rewrites poster_path to a local storage key, media_items gains poster_source_path, captured during cacheItemImages, preserved across refreshes that keep the cached poster, and cleared on explicit poster overrides. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
3d2368aed7 |
feat(notifications): admin server channels broadcasting new content and request activity
Add admin-owned broadcast destinations ("community channels"): Discord or
generic webhooks fed straight from release_events by a per-channel watermark
sweep, announcing newly added movies/episodes as grouped digest posts plus
configurable media request lifecycle events (submitted/approved/declined/
fulfilled).
- Extend release_events with a kind discriminator and add a movie
availability spine (movie_availability + kind-keyed
notification_content_seed_state; first full scan seeds silently so
upgrades never flood the movie back catalog)
- Sweep worker reads events by (created_at, id) cursor with batch-window
grouping, per-channel backoff, and auto-disable; request events post
best-effort via new requests.LifecycleNotifier hooks
- Reuse the webhook stack throughout: URL encryption (new AAD namespace),
SSRF guard, embed limits, HMAC signing; shared type/name validation
extracted for both services
- Admin CRUD API under /admin/notifications/server-channels and a Server
Channels section in the notifications admin settings UI
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
QuickandClaude Fable 5
|
ebf3352bda |
feat(notifications): per-profile email channel with verified addresses
Re-key the email notification channel from login accounts to profiles. Each profile owns its mode, dispatch watermark, and destination address; there is deliberately no fallback to the account email, so the account holder no longer receives mail for every household profile. A profile receives nothing until its own address is verified. - Genericize the watermark-sweep engine over a recipient key (accountChannel[K]): email keys by profile_id, Discord stays on user_id. Delivery reads move into the channel adapters. - Custom addresses verify via single-use SHA-256-hashed token links served by a public endpoint; enabling the channel requires a verified address, and clearing the address switches the channel off. - Addresses are globally unique (case-insensitive): rejected when verified for another profile or matching another account's email or username. Checked at request time, re-checked at verify time (first-to-verify wins), backstopped by a partial unique index. - Every email carries an RFC 8058 one-click unsubscribe link backed by a per-profile capability token, minted lazily under the claim tx. - Child profiles cannot set addresses (and so receive no email in v1). - Verification sends are rate limited (1/min, 10/day per profile); mail.Message gains custom header support for List-Unsubscribe. - Migration drops the account-level prefs table without carrying opt-ins over, so nobody gets surprise emails post-upgrade. Android/Apple notification settings need follow-up for the new profile-scoped response shape and address-management endpoints. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
5f05374a1d |
feat(notifications): Discord bot DM channel with account linking
Adds Discord direct messages as a notification channel. Users link their Discord account via OAuth2 (identify scope only, one-time server-side state rows); a bot delivers their inbox notifications as DMs. - Extract the email channel's watermark sweep into a generic account-channel engine; email and Discord are now thin adapters, so the SKIP LOCKED claim / watermark-after-send durability logic exists once. - New internal/discord REST client (token exchange, identity, open DM, send message) — no Gateway connection, no new dependencies. - Opt-in master switch (notifications.discord_enabled, default off) gates delivery, linking, capability, and the admin settings reveal. - Admin UI: credentials (secret + bot token encrypted at rest), dev portal setup checklist, bot invite link buttons, and a test button that bypasses the settings read cache and is disabled while credential edits are unsaved. - DM failures from missing shared guild (Discord 50007) surface as link health in user settings and self-heal via capped backoff. - New combined mode (per_episode_and_digest) for email and Discord: instant sends all day plus a daily digest recapping the whole window since the previous digest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
df95e3cb95 |
feat(notifications): email notification channel
Adds email as a notification channel built on the shared SMTP core (mail.Sender). Email mode is a per-account preference (off, daily digest, or per-episode) stored in notification_email_prefs; delivery is an account-watermark sweep over notification_deliveries that dedupes cross-profile duplicates, advancing the watermark only after a successful send. Admin controls cover the channel kill switch, the per-episode allowance (off coerces those accounts to the digest), digest hour, and an external URL for deep links inside emails. Availability is advertised through /notifications/capability and the user settings page gains an Email section for opt-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
e5b210589d |
fix(notifications): address PR #136 review findings
Codex + CodeRabbit review fixes, all verified against current behavior: - Web Push: single-writer VAPID provisioning via a new conditional SetIfAbsent settings write (no split-brain identity across nodes), and read/decode failures now surface instead of silently rotating the keypair; the eager-provisioning goroutine joins the shutdown WaitGroup - Web Push: endpoint reassignment purges the previous owner's pending attempts inside the upsert transaction, with an ownership re-check at send time - Webhooks: per-profile cap enforced atomically (advisory-locked count+insert), typed pgconn unique-violation mapping, create-time type/URL mismatch rejection, send-time HTTPS re-check, and Retry-After HTTP-date support (shared, clamped parser also used by web push) - Delivery workers: transient delivery-row lookup errors leave the claim to lease expiry instead of permanently failing the attempt - Interest: history-only imports now feed the index (userstore history hooks + completed-history folding in recompute/rebuild), rebuild also recomputes existing interest rows so removed sources get cleaned up, and failed flush mutations requeue (bounded) instead of dropping - Retention: read notifications age from read_at, not created_at - Startup: scan queue workers start only after the availability detector is wired, so resumed scans cannot skip availability recording - mail: settings-store read failures propagate instead of reading as "not configured" - DB: new migration adds episode ordinal/key CHECK constraints - Web: service worker restricts notification clicks to same-origin URLs, preferences popover gets an error+retry state, and the realtime profile-rebind backoff grows to 5 minutes to keep shared channels stable through notifications-only outages Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
d9e27da59e |
feat(notifications): request-fulfilled notifications across all channels
Notify the requesting profile once its media request is actually present in the catalog (roadmap 06, item 2). Completion transitions stay notification-agnostic; a presence-gated pass at the end of each reconcile run fires the notice, so it means "watchable in Silo", not "download finished". - New System.DispatchOperational: delivery insert + webhook/web-push outbox enqueue in one transaction, post-commit multi-dispatch. The webhook auto-disable notice now rides the same path (replacing its hand-rolled hub publish and the now-removed InsertOperational), which also delivers auto-disable notices over web push. - At-most-once delivery: partial unique index on (profile_id, reason_flags->>'request_id') plus a fulfilled_notified_at marker on media_requests, backfilled for pre-existing completed requests so deploys never flood. - Per-webhook notify_requests toggle (default on) through repo, service, API, and settings UI; gated independently of the episode reason flags. - request.fulfilled rendering in web inbox, realtime toast, web push payload, and Discord/generic webhook payloads, deep-linking to the matched catalog item. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
b091f0c6c1 |
feat(notifications): in-app inbox, realtime, webhooks, web push + shared SMTP core
Implements the notification system foundation and all v1 delivery channels that need no external infrastructure (specs 00/01/04/05 in docs/superpowers/plans/notifications/): Foundation (spec 01): - episode_availability seeding + per-library seed markers: "newly available" means newly released to this server, so back-catalog imports and first scans never flood (verified on dev: 1.13M episodes seeded silently) - release_events -> profile_series_interest fanout worker with settling delay, per-series burst caps, FOR UPDATE SKIP LOCKED multi-node claims, and a guarded last-notified cursor - interest index maintained via a userstore provider decorator so every favorites/watchlist/progress mutation path (REST, jellycompat, imports, playback) feeds it; progress writes only recompute on state transitions - durable per-profile inbox + read state, forward-sync cursor API, websocket channel with short-lived single-use handshake tickets - web UI: sidebar badge, inbox page, toasts, per-profile preferences - startup/daily tasks: availability seeding, interest rebuild, retention Outbound webhooks (spec 04): - Discord embeds (text-only per the v1 privacy contract) and generic JSON signed Stripe-style with per-webhook secrets - HTTPS-only + private-destination guard enforced at registration and at connect time (DNS-rebinding mitigation); URLs/secrets encrypted at rest - durable per-target outbox enqueued in the fanout transaction, lease-based claims, 24h exponential retry, 3x-consecutive-4xx auto-disable with an in-app notice (loop-guarded) Web push (spec 05): - VAPID keypair self-provisioned at startup (single atomic JSON setting, private half encrypted at rest) — no third-party accounts needed - payloads E2E-encrypted (RFC 8291); 404/410 treated as unsubscribe - service worker + subscribe flow in Settings -> Notifications Shared SMTP core (internal/mail): - feature-agnostic mail.Sender over live email.* settings, STARTTLS or implicit TLS, encrypted password, admin Email settings page with synchronous test send; no consumer yet by design (digest is v1.5) APNs/FCM (specs 02/03) are deferred to v2; the capability endpoint reports them unavailable so clients render truthfully. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |