* feat(metadata): reconcile artwork cache after public S3 provider changes
Changing the public S3 provider previously broke every cached image
permanently: the DB keeps bucket-relative keys, the image cache pipeline
treats a cached path as its durable dedup marker and never re-enqueues,
and clients eat the 404s straight from S3 so the server never notices.
Add a storage identity fingerprint (s3.public_storage_identity, seeded
via SetIfAbsent at boot) and a reconcile_artwork_cache task whose
startup trigger only fires when the identity changed; manual runs
always sweep, doubling as bucket-data-loss recovery. The task probes a
random sample of cached objects, then either bulk-resets (near-total
miss) or per-row verifies. Missing provider-sourced artwork is reset to
its *_source_path so the existing enqueue loop re-caches it; surfaces
without a re-downloadable source (chapter thumbnails, collection
artwork, library posters, branding refs, embedded book covers) are
cleared so their owning pipelines refill them. Small upload-holding
tables are always per-row verified so bulk mode cannot blind-clear an
upload that survived migration, and transport errors never reset rows.
Users never see broken images during the transition: reset rows serve
the provider's original URL via the existing absolute-URL pass-through
and thumbhashes are preserved. The storage settings page now warns that
uploads cannot be re-downloaded when the identity fields are edited.
Part of #348
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): harden artwork reconcile per code review
Address the confirmed findings from the PR review:
- Fingerprint the key prefix case-sensitively and slash-trimmed exactly
as s3client applies it (new exported NormalizeKeyPrefix): a case-only
prefix edit is a real storage move and must reconcile; a slash-only
edit is not and must not.
- Certify the storage fingerprint immediately after the artwork sweep
succeeds and make the 4-object branding check non-fatal (reported in
the task message), so a transient branding error cannot discard a
completed catalog sweep and force it to repeat every boot.
- Fail closed on conditional-task preflight errors in the task manager
(previously fail-open ran the task), and retry transient settings
reads in ShouldRun since the startup trigger fires once per process.
- Track probe HEAD errors against a separate baseline so a flaky probe
cannot consume the sweep's error budget.
- Probe before counting: bulk mode skips the per-surface count(*)
full scans entirely, and probe sampling drops ORDER BY random()
(plain LIMIT answers "is the cache in this bucket" just as well).
- Verify chapter thumbnails across a whole 500-file batch in one HEAD
fan-out instead of per file, keeping the worker pool saturated.
- Replace the 10 inline non-provider-scheme ARRAY literals in the
enqueue query with the shared nonProviderImageSchemesSQL constant.
Part of #348
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): guard bulk reset against degraded probes, certify only clean sweeps
Address bot review feedback on the reconcile hardening:
- A probe where more than half the HEAD requests error aborts the run:
errored requests are excluded from the sample, so a partial outage
could otherwise present a handful of surviving 404s as a ~100% miss
rate and bulk-reset the catalog. Bulk mode additionally requires a
minimum number of successful samples; thinned probes and tiny
catalogs take the safe per-row verify path.
- Track sweep errors separately from probe/branding errors
(stats.sweep_errors) and certify the storage fingerprint only when
the sweep completed with zero of them — skipped rows were never
verified, so the next startup retries. Applied resets stay durable.
- Give each ObjectExists attempt its own timeout so a stalled HEAD
fails that attempt instead of pinning the retry loop to the run
context.
- Report branding assets checked (not just cleared) in stats.Checked.
- Drop the dead settingsRepo/brandingSvc nil guards in cmd/silo and
sync spec numbers with the implementation constants.
Part of #348
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
