Commit Graph
459 Commits
Author SHA1 Message Date
QuickandGitHub 5d7eec7a62 Merge pull request #457 from Silo-Server/t3code/review-discord-issue
fix(autoscan): match rewrite rules against UNC webhook paths
2026-07-23 11:42:33 -04:00
Quick104andClaude Fable 5 65d94459f4 fix(autoscan): preserve trailing separator through path rewrites
Review follow-up: normalizePath strips a trailing slash, but the slash is
semantic for legacy-scope changes — filepath.Dir("/x/Show/") is the
directory itself while filepath.Dir("/x/Show") is its parent, so dropping
it widened targeted directory notifications into parent/library scans.
applyRewrites now records whether the incoming path ended with a separator
(either form) and restores it on the returned path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:40:38 -04:00
QuickandGitHub c0b2e18173 Merge pull request #433 from RXWatcher/fix/ebook-enrichment-architecture
feat(ebooks): decouple metadata enrichment from scans via a durable queue
2026-07-23 11:18:42 -04:00
Quick104andClaude Fable 5 89b7d25084 fix(autoscan): match rewrite rules against UNC webhook paths
Incoming webhook paths were only separator-swapped (normalizeSeparators),
while rewrite From values went through normalizePath, which also collapses
duplicate slashes. A Windows UNC root from a Windows-hosted arr
(\\NAS\Media\TV -> //NAS/Media/TV) therefore never prefix-matched its
rewrite rule (/NAS/Media/TV), so every import logged "webhook paths
matched no library folder" and nothing scanned.

applyRewrites now normalizes the incoming path with the same normalizePath
used for the stored From, and normalizes the joined result so a
trailing-slash To cannot produce a doubled separator.

Reported via internal Discord thread (Sonarr on Windows with a UNC TV
root posting to the autoscan webhook).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:14:45 -04:00
Quick104andClaude Fable 5 84cd0acb0c fix(ebooks): keep legacy-lane rows in their lane until terminal outcomes
Scanner re-enqueues (priority 100) could silently promote pending legacy
backlog rows (priority -100) into the incremental lane via the enqueue
upsert's GREATEST, and the fail/release requeue branches hardcoded 100
regardless of the row's lane. A mass mtime shift or group-key-version
bump would have moved the entire legacy backlog out from under the
backfill task's pacing controls into the scheduled sync task.

Lane changes now happen only through terminal outcomes (complete or
discard); enqueue, fail, and release preserve a negative priority.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:10:31 -04:00
Quick104 f6615ef8d7 fix(jellycompat): confirm authoritative watch stops 2026-07-23 09:15:19 -04:00
Quick104 ac1dd99c72 fix(jellycompat): address scrobble lifecycle review 2026-07-22 23:22:09 -04:00
Quick104 487dc84829 fix(jellycompat): deduplicate playback negotiations
- Replace unstarted negotiations for the same device and item
- Apply deduplication atomically across durable store instances
2026-07-22 22:01:02 -04:00
Quick104 166c5ef32f Add reliable Jellycompat watch scrobbling
- Forward start, pause, resume, and stop events with stable media identities
- Persist and retry terminal scrobbles across teardown and restart paths
- Reject ambiguous playback-report route matches
2026-07-22 21:41:05 -04:00
QuickandGitHub a0507c78eb Merge pull request #447 from Silo-Server/codex/jellycompat-color-range
fix(playback): preserve and expose video color range
2026-07-22 13:38:16 -04:00
rxwatcherandClaude Fable 5 bcc78e7332 fix(server): return 404 for missing content-hashed assets
A /assets/ chunk from a previous build no longer exists after a deploy;
serving the SPA shell at that URL makes the browser fail dynamic imports
on a text/html module. Exclude /assets/ from the SPA fallback so the
miss surfaces as a 404 the client can react to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcherandClaude Fable 5 845d969af1 feat(ebooks): run legacy backfill automatically
Give the backfill task a default 15-minute interval trigger. With the
rate-limit cooldown floor each run meets a fresh ready-set, a saturated
batch trips the zero-progress breaker, and an empty lane exits in
milliseconds, so the backlog drains at provider speed unattended. The
canary claim cap and batch delay keep their semantics, and operators can
retune or disable the trigger through the admin task UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcherandClaude Fable 5 dfce4972e4 fix(ebooks): floor rate-limited requeue delays
The ebook-metadata plugin attaches ~1s RetryInfo to ResourceExhausted
errors as request-pacing advice for its internal token bucket. Adopting
that hint verbatim as the queue horizon made rate-limited rows claimable
again immediately, so every backfill run re-claimed the same saturated
tail. Clamp rate-limited requeues to a 15m floor (SILO_EBOOK_RATE_LIMIT_COOLDOWN
to tune); hints above the floor are honored up to the existing 24h cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcherandClaude Fable 5 a2e6cd8d5f docs(scan): document vanished-path rejection branch
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcherandClaude Fable 5 d6cdf5c9e4 fix(ebooks): account canceled and vanished claims accurately
A claim whose enrichment surfaced context.Canceled while the sweep was
still live was released uncounted, leaving it immediately reclaimable and
invisible to the no-progress circuit breaker. Let it flow through the
failure path as a transient error so it backs off and is counted; genuine
sweep shutdown still releases via the existing ctx check.

Claims discarded because the item vanished are terminal, not retried, so
report them in a new discarded counter instead of inflating deferred, and
surface the count in task progress output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcherandClaude Fable 5 fcd7eb697d fix(ebooks): harden OPF sidecar ingestion
Close the symlink-swap window in the sidecar reader: os.Open follows
symlinks, so a leaf swapped between the Lstat gate and the open could pull
metadata from outside the library root. Reject unless the opened handle is
the exact file Lstat inspected, matching the image cache processor guard.

Replace a stale ISBN provider id on rescan instead of silently dropping it,
so sidecar ISBN corrections actually take effect; tolerate the ISBN already
belonging to another item so duplicate copies keep scanning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:43:57 +02:00
rxwatcher 44db85457c feat(ebooks): ingest external OPF metadata sidecars 2026-07-22 13:43:57 +02:00
rxwatcher ab29bc8c8e fix(ebooks): skip remote lookup for complete local metadata 2026-07-22 13:43:57 +02:00
rxwatcher 37adec320e perf(ebooks): feed configured backfill workers 2026-07-22 13:43:57 +02:00
rxwatcher 537b3f6476 perf(ebooks): allow large backfill worker pools 2026-07-22 13:43:57 +02:00
rxwatcher 60ce7ed1a4 fix(scan): reconcile vanished media follow-up state 2026-07-22 13:43:57 +02:00
rxwatcher b3e963f54e fix(scan): handle typed media file edge cases 2026-07-22 13:43:57 +02:00
rxwatcher f094d70996 fix(ebooks): model targeted scans as file-only 2026-07-22 13:43:57 +02:00
rxwatcher 7f3b93d1b4 fix(ebooks): route targeted files through ebook scanner 2026-07-22 13:43:57 +02:00
rxwatcher b1545ae9f9 fix(scan): accept ebook and audio file targets 2026-07-22 13:43:57 +02:00
rxwatcher bb53bd0652 fix(ebooks): batch-load enrichment identifiers 2026-07-22 13:43:57 +02:00
rxwatcher bf3c5645f4 fix(ebooks): repair recent enrichment gaps 2026-07-22 13:43:57 +02:00
rxwatcher 794e2d786a fix(ebooks): use planner-safe queue cursors 2026-07-22 13:43:57 +02:00
rxwatcher fcaee9bfce fix(ebooks): persist bounded reconciliation cursor 2026-07-22 13:43:57 +02:00
rxwatcher 9136803130 fix(ebooks): harden enrichment rollout controls 2026-07-22 13:43:57 +02:00
rxwatcher 3a5d318714 feat(ebooks): add backfill canary controls 2026-07-22 13:43:57 +02:00
rxwatcher 194e5ff527 fix(ebooks): stop stalled enrichment drains 2026-07-22 13:43:57 +02:00
rxwatcher c786551ba2 fix(ebooks): use lane-specific claim indexes 2026-07-22 13:43:57 +02:00
rxwatcher 68b104fb53 fix(ebooks): isolate scans and bound queue claims 2026-07-22 13:43:57 +02:00
rxwatcher 8d40138bdb feat(ebooks): drain enrichment backlog with progress 2026-07-22 13:43:57 +02:00
rxwatcher 1d5f19d794 fix(ebooks): keep scans resilient to queue errors 2026-07-22 13:43:57 +02:00
rxwatcher 1c2d422548 fix(ebooks): bound enrichment lease work 2026-07-22 13:43:57 +02:00
rxwatcher eb6b968ed1 fix(ebooks): harden enrichment queue semantics 2026-07-22 13:43:57 +02:00
rxwatcher ef7eedf3fc feat(ebooks): add durable metadata enrichment queue 2026-07-22 13:43:57 +02:00
rxwatcher 1ab85d18ea fix(ebooks): decouple enrichment from library scans 2026-07-22 13:43:57 +02:00
Quick104andClaude Fable 5 31e6f1ef41 fix(diagnostics): address PR #450 review findings
- service.go: reject trailing data after the decoded manifest object.
  Decoder.More() only reports array/object iteration, so a stray closing
  delimiter (e.g. {...}}) slipped through where json.Unmarshal used to
  reject it. Require the stream to reach io.EOF after decoding on both
  the received and embedded sides; add a regression test.
- repo.go: split the list projection from cleanup. reportListSelectSQL
  keeps the app_build JSONB extraction for the admin list; new
  reportCleanupSelectSQL omits it so retention/stale batches don't touch
  each candidate's manifest JSONB just to delete a row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 14:53:47 -04:00
Quick104andClaude Fable 5 9fad08a6fa fix(diagnostics): address round-6 review findings on PR #445
- AdminDiagnostics list: fix regression where rows dereferenced the
  now-omitted manifest for app_build. Project app_build server-side out
  of manifest JSONB into both list and detail responses (cheap
  COALESCE(manifest->'report'->>'app_build','')), split the TS type into
  DiagnosticReportSummary (list, no manifest) and DiagnosticReport
  (detail, with manifest), and read report.app_build in the row/detail.
- embeddedManifestMatches: decode with json.Decoder + UseNumber so large
  integers above 2^53 (e.g. log_summary.lines) can't collapse to the same
  float and falsely match; re-assert no-trailing-data strictness.
- Quota reservation (SKIP): reserving the client-claimed archive.bytes is
  sound because archiveMatches requires claimed==actual before MarkReady,
  so no stored report exceeds its reservation; documented in a code comment.
- Multipart parts: reject a wrong-name/wrong-content-type part without
  calling part.Close(), which would drain up to the bundle limit while
  holding the in-flight slot; abandon it so malformed uploads fail promptly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 14:23:59 -04:00
Quick104andClaude Fable 5 dee46f9398 fix(diagnostics): address round-5 review findings on PR #445
- service: reject supplied child-profile attribution with a distinct
  ErrChildProfileForbidden (403 child_profile_forbidden) instead of
  silently dropping it as if the profile were not found; a profile that
  is simply not the user's still drops attribution unchanged
- repo: add a manifest-free list projection (reportListSelectSQL /
  scanReportSummary) for admin list and retention/stale cleanup queries
  so they no longer drag the full manifest JSONB per row; keep the full
  projection for GetByID/DeleteByID and mark Manifest omitempty
- cleanup: delete/mark the DB row before the blob in retention and stale
  loops so a mid-run DB failure can't leave a ready report pointing at a
  missing bundle; blob-delete failures are logged with bucket/keys for
  orphan cleanup to reap rather than aborting the run (shared helper with
  the admin DeleteReport path)
- admin: reject diagnostics settings where max_bytes_per_user would fall
  below max_bundle_bytes (and the reciprocal), which would make every
  max-size upload fail quota
- router/demo: route POST /diagnostics/reports through DemoGuard and block
  the reports prefix in demo mode while keeping GET /diagnostics/status
  available

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 13:36:08 -04:00
Quick104 b822bb7181 test(catalogseed): cover unknown color range 2026-07-21 13:03:53 -04:00
Quick104andClaude Fable 5 1f9bd99990 fix(diagnostics): address round-4 review findings on PR #445
- schema: add crash/report.type conditionals (allOf if/then) so a
  crash/anr/native_crash/hang/abnormal_exit manifest requires `crash`
  and a `manual` manifest forbids it, matching ValidateManifest.
- service: reject uploads where X-Profile-Id and manifest.report.profile_id
  are both present but differ (new ErrProfileMismatch, mapped to 400
  profile_mismatch) instead of silently preferring the header; single-source
  and matching cases unchanged. Adds service tests for mismatch, match, and
  header-only attribution.
- schema: require manifest.json as the first archive.entries element via
  prefixItems (contains retained for validators without prefixItems support).
- schema: document that maxLength is a character-count bound while the server
  enforces UTF-8 byte length, via a top-level note and per-field notes on the
  free-text device_summary and crash fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 12:58:28 -04:00
Quick104 4b3362d32e fix(playback): expose source color range to native clients 2026-07-21 12:57:54 -04:00
Quick104andClaude Fable 5 93b851fe81 fix(diagnostics): address round-3 review findings on PR #445
- Extend the upload write deadline alongside the read deadline so a slow
  upload finishing after the integrated server's 120s WriteTimeout can still
  return its success response instead of timing out a report that succeeded.
- Reject child-profile attribution for diagnostics: wire the attribution
  validator through a shared profile lookup that reports IsChild and drop
  attribution for child profiles, which must not perform diagnostics actions.
- Assert the download test captures the clicked anchor and checks its blob:
  href and silo-diagnostics-<short_id>.tar.gz filename, not just cleanup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 12:25:11 -04:00
Quick104andClaude Fable 5 2d5d4980de fix(diagnostics): address round-2 review findings on PR #445
- settings.go: cap the parsed cleanup interval at 7 days before converting to
  time.Duration so a huge configured value can't overflow int64 nanoseconds and
  wrap into a tiny/negative interval; add boundary tests.
- settings.go: propagate genuine settings read failures from LoadSettings
  (missing/empty -> default, error -> fail) so a transient DB error surfaces
  retryably instead of silently reporting uploads disabled or wrong quotas.
- bundle.go: validate non-manifest bundle entries while streaming with bounded
  memory -- device.json and crash/*.json must be a single JSON object,
  logs.jsonl/breadcrumbs.jsonl must be newline-delimited JSON objects with a
  per-line byte cap (new contract.MaxLogLineBytes); binary members stay opaque.
- diagnostics upload handler: extend the read deadline per-route via
  http.ResponseController.SetReadDeadline (10m) so slow mobile uploads of large
  bundles aren't cut off by the shared 30s server ReadTimeout.
- web admin download: request the ?proxy=1 streaming path directly so downloads
  work when S3Private is only server-reachable and errors can surface in-page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 11:29:02 -04:00
Quick104 827843fd21 fix(jellycompat): omit unspecified color range 2026-07-21 11:03:08 -04:00
Quick104andClaude Fable 5 a6348b3dc5 fix(diagnostics): address PR #445 review findings
- bundle: reject tar entry names that differ from their trimmed form instead
  of normalizing padded names into the allowlist
- repo: reserve expected bytes on receiving rows and count receiving+ready in
  the per-user byte quota so concurrent/multi-node uploads can't overshoot
- contract: require the crash object for event report types and keep it absent
  for manual; add contract tests
- settings/service: seed diagnostics.server_instance_id atomically via
  insert-if-absent and adopt the winning value across nodes
- bundle/service: capture the embedded manifest.json during ValidateBundle and
  reject reports whose embedded manifest disagrees with the part-1 manifest
  (minus archive); add tests
- admin: delete the DB row before the blob on DeleteReport; log bucket/key when
  the blob delete fails instead of leaving a visible report with a missing bundle
- bundle: reject PAX/GNU tar formats and extension records that smuggle bytes
  past validation; add a PAX-archive rejection test
- migration: add CHECK constraints for state, report_type, and platform
- docs: add text/jsonc language identifiers to the two unfenced code blocks
- cleanup: log-and-continue per report and aggregate errors so one poisoned
  report no longer blocks the whole run; update tests
- tasks: give diagnostics its own cleanup interval key instead of reusing the
  opslog key, and bound the startup settings lookup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
2026-07-21 10:03:11 -04:00