Compare commits

..
Author SHA1 Message Date
Quick104andClaude Fable 5 6cd09c96cc fix(downloads): pin swept-artifact status in the delete predicate
Codex: the sweep's status-unrestricted delete could remove a failed row
that Ensure had just requeued and the drain had claimed as running —
before any download links it, the active-link fence and the FK both pass,
and the live job's row (and then its bytes) would be removed. The sweep
now deletes only rows still in the status the candidate list saw
(DeleteFailedIfEvictable / DeleteReadyIfEvictable), so a requeued or
running job survives regardless of links.

Plus one straggler misspell finding from CI's changed-lines lint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:58:26 -04:00
Quick104andClaude Fable 5 611c9ac594 fix(downloads): fail migration-healed dangling rows and satisfy lint
Bugbot: nulling a dangling artifact_id on a ready/preparing remux row
left it advertised — ready rows would silently fall back to serving the
source file and preparing rows hung forever (reconciliation only visits
artifact-linked rows) while consuming quota. The backfill now fails those
rows with a retryable message instead; clients re-request and get a
fresh artifact.

Also: misspell (cancelled->canceled in comments/locals) and sloglint
(literal log messages) findings from CI's changed-lines lint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:46:10 -04:00
Quick104andClaude Fable 5 80397a5eb3 fix(downloads): enforce artifact links with an FK and make eviction leak-proof
Remediates review findings on the eviction fence:

- The EXISTS/NOT EXISTS fences read independent snapshots under READ
  COMMITTED and take no conflicting locks, so a concurrent evict/link
  pair can both commit (reproduced empirically in both interleavings),
  leaving a ready download pointing at a deleted artifact. New
  downloads_artifact_id_fkey (ON DELETE RESTRICT) is the enforcement
  layer: RI triggers take FOR KEY SHARE and re-check on a fresh snapshot.
  The fences stay as fast-path filters; 23503 maps to ErrArtifactEvicted
  on insert/replace and reads as 'not evictable' on delete. Terminal
  transitions null artifact_id so dead rows cannot pin artifacts; the
  migration heals legacy dangling/terminal links before the constraint.
- Row-first eviction could leak bytes when the unlink failed after the
  row was gone (no local orphan queue, no reaper; remote leaked when the
  node delete and orphan enqueue both failed). Cleanup intents are now
  recorded durably BEFORE the row delete — a new
  download_artifact_local_orphans queue mirrors the remote one — and a
  failed enqueue skips the candidate so the row survives for retry.
- Output paths now include the artifact id, so a replacement row can
  never claim an evicted row's path; the OutputPathInUse probe (an
  unindexed seq scan with a TOCTOU window) is gone.
- The failed-artifact hygiene sweep is fenced (DeleteIfEvictable):
  Ensure requeues failed rows, so the unfenced delete could remove an
  artifact a fresh download had just linked, stranding it in preparing.
- ReplaceManagedEntry drops the non-atomic artifact probe; zero rows now
  unambiguously means revision/identity conflict and self-heals via
  GetManagedEntry. The replace retry refreshes its row snapshot so the
  second attempt's revision fence can succeed.
- Reuse/cleanups: shared withEnsuredArtifact retry helper, downloadColumns
  interpolated into all insert sites, execInsertDownload shared by
  Create/CreateBatch, EnsureDevice hoisted out of the retry loop,
  HasActiveLink and unconditional DeleteArtifact deleted, distinct log
  messages for the two sweep failure paths.

DB-backed tests cover the FK both directions, the requeued-artifact sweep
fence, local orphan round-trip, terminal unpinning, revision-conflict
self-healing, and a 100-round concurrent evict-vs-link race asserting no
dangling links.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:32:11 -04:00
Cursor AgentandQuick 1ae35c244f fix(downloads): map artifact eviction to a retryable conflict
A Create that loses two eviction races should not surface as a generic
500. Return 409 artifact_evicted so the client can retry.

Co-authored-by: Quick <Quick104@users.noreply.github.com>
2026-08-13 14:06:01 +00:00
Cursor AgentandQuick b22d86fdd1 fix(downloads): fence LRU eviction against concurrent artifact links
Cleanup deleted ready artifacts after a separate HasActiveLink check, so a
cancel-then-redownload during remote byte deletion could persist a ready
row whose artifact id was then removed. ReconcileLinkedDownloads cannot
heal a missing artifact, so the download stayed advertised and 404ed.

Delete the ready row only when no active link exists, refuse Create and
ReplaceManagedEntry against a vanished id, and retry Ensure once so the
new download attaches to a fresh artifact instead.

Co-authored-by: Quick <Quick104@users.noreply.github.com>
2026-08-13 14:03:22 +00:00
140 changed files with 1574 additions and 7373 deletions
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 62 KiB

+1 -1
View File
@@ -730,7 +730,7 @@ func main() {
handler = srv.Handler()
} else {
srv := transcodenode.NewServer(watcher, tracker)
srv.SetInputPathAuthorizer(transcodenode.NewCatalogPathAuthorizer(scanner.NewFileRepository(pool)))
srv.SetInputPathAuthorizer(transcodenode.NewMediaRootAuthorizer(catalog.NewFolderRepository(pool)))
srv.SetFFmpegLogSink(playback.NewSlogFFmpegLogSink(slog.Default(), nodeID))
// Read jellycompat reconstruction recipes central wrote at transcode
// start, so this node can rebuild a Jellyfin transcode after its own
-2
View File
@@ -25,8 +25,6 @@ func buildLiveSessionSync(s *playback.Session, reportingNode string) worker.Sess
ClientIP: s.ClientIP,
ClientName: s.ClientName,
ClientVersion: s.ClientVersion,
ClientBuild: s.ClientBuild,
ClientChannel: s.ClientChannel,
ClientUserAgent: s.ClientUserAgent,
AudioTrackIndex: s.AudioTrackIndex,
TranscodeAudio: s.TranscodeAudio,
@@ -39,22 +39,6 @@
"audio_underruns": {
"type": "integer",
"description": "Audio underrun count."
},
"session_id": {
"type": "string",
"description": "Playback session identifier for correlation with operational logs."
},
"play_method": {
"type": "string",
"description": "Selected playback delivery method such as direct, remux, or transcode."
},
"reason": {
"type": "string",
"description": "Playback route, interruption, or lifecycle reason."
},
"position_ms": {
"type": "integer",
"description": "Current playback position in milliseconds."
}
},
"focus": {
@@ -83,44 +67,12 @@
"duration_ms": {
"type": "integer",
"description": "Request duration in milliseconds."
},
"outcome": {
"type": "string",
"description": "Request outcome such as success, http_error, transport_error, or cancelled."
},
"error_code": {
"type": "string",
"description": "Stable error classification for a failed request."
},
"attempt": {
"type": "integer",
"description": "Attempt ordinal for a retried request."
}
},
"lifecycle": {
"state": {
"type": "string",
"description": "App lifecycle state."
},
"phase": {
"type": "string",
"description": "Startup or lifecycle phase identifier."
},
"duration_ms": {
"type": "integer",
"description": "Elapsed duration for the phase in milliseconds."
},
"outcome": {
"type": "string",
"description": "Result of the phase, such as success, failure, or skipped."
},
"reason": {
"type": "string",
"description": "Classification for the phase outcome."
},
"launch_type": {
"type": "string",
"description": "Launch classification, such as cold or warm."
}
},
"crash": {
@@ -10,7 +10,6 @@
"playback_route_diagnostics",
"device_quirks_v1",
"seek_reanchor_v1",
"output_change_v1",
"direct_stream_resume_v1",
"plan_source_duration_v1"
],
@@ -7,7 +7,6 @@
"playback_route_diagnostics",
"device_quirks_v1",
"seek_reanchor_v1",
"output_change_v1",
"direct_stream_resume_v1",
"plan_source_duration_v1"
],
@@ -580,14 +580,6 @@
"type": "string",
"maxLength": 64
},
"app_build": {
"type": "string",
"maxLength": 64
},
"app_channel": {
"type": "string",
"maxLength": 32
},
"device": {
"type": "object",
"description": "Manufacturer and model stay first-class because the device-quirk registry matches on them. Everything platform-specific travels in platform_details as opaque bounded strings.",
@@ -531,14 +531,6 @@
"type": "string",
"maxLength": 64
},
"app_build": {
"type": "string",
"maxLength": 64
},
"app_channel": {
"type": "string",
"maxLength": 32
},
"device": {
"type": "object",
"description": "Manufacturer and model stay first-class because the device-quirk registry matches on them. Everything platform-specific travels in platform_details as opaque bounded strings.",
-32
View File
@@ -48,38 +48,6 @@ the other. Send the exact lower-case family:
| macOS | `desktop` |
| Browser | `web` |
### App identity headers
Separately from the family, every first-party client should send its own app
identity on playback requests. These are server-wide contextual headers, not
settings-specific: the playback session stores them, the admin Activity page
renders them ("Silo Android TV 1.0.0 (build 5)"), and playback decision logs
carry them so a report can be tied to an exact build.
| Header | Clamp | Meaning |
| ----------------------- | ----- | ------------------------------------------------------------------------------------------------------------- |
| `X-Silo-Client` | 128 | Product name, e.g. `Silo Android TV`, `Silo iOS`. |
| `X-Silo-Client-Version` | 64 | Marketing version, e.g. `1.0.0`. Sent verbatim and displayed verbatim — do not pre-shorten it. |
| `X-Silo-Client-Build` | 64 | Opaque per-platform build identifier (Android `versionCode`, Apple `CFBundleVersion`). Never parsed or compared. |
| `X-Silo-Client-Channel` | 32 | Opaque distribution channel: `release`, `beta`, `sideload`, `dev`. Stored verbatim; `release` is not displayed. |
Values are trimmed and truncated to the clamp above — never rejected, on either
route, because an identity label must not be able to fail a playback start. The
clamp counts characters, not bytes, matching `maxLength` in the v3 request
schemas, and is applied where the request is read rather than where the session
is created so the decision logs and `playback_route_events` observe it too.
Nothing is validated against an enum either, so a client may introduce a new
channel without a server change.
Protocol-v3 `POST /playback/start` accepts `client_playback_context.app_version`,
`.app_build`, and `.app_channel` as a body-level fallback for clients that cannot
set the headers on every request. The headers win field by field when both are
present, and the fallback applies **only to a client that sent `X-Silo-Client`**:
`client_playback_context` carries no app name, so nothing in the body can
identify a client that did not name itself — such a session is labeled from its
user agent, and its `app_version` is a free-form platform string rather than the
marketing version `client_version` promises.
## Remote scopes
Every stored value has exactly one identity. Context fields not named by the
-9
View File
@@ -278,15 +278,6 @@ func (s stubStore) GetSeriesPlaybackPreference(context.Context, string, string)
func (s stubStore) DeleteSeriesPlaybackPreference(context.Context, string, string) error {
panic("unused")
}
func (s stubStore) SetCollectionSortPreference(context.Context, userstore.CollectionSortPreference) error {
panic("unused")
}
func (s stubStore) GetCollectionSortPreference(context.Context, string, string, string) (*userstore.CollectionSortPreference, error) {
panic("unused")
}
func (s stubStore) ClearCollectionSortPreference(context.Context, string, string, string) error {
panic("unused")
}
func (s stubStore) GetLibraryPlaybackPreference(context.Context, string, int) (*userstore.LibraryPlaybackPreference, error) {
panic("unused")
}
-16
View File
@@ -59,16 +59,6 @@ type catalogResponse struct {
Items []itemListResponse `json:"items"`
Snapshot string `json:"snapshot,omitempty"`
SearchDiagnostics *searchDiagnostics `json:"search_diagnostics,omitempty"`
// EffectiveSort reports the order a collection source actually resolved in
// once the viewer's saved override and the collection's configured default
// were applied. Omitted for non-collection sources and when the collection
// resolved in its own source order.
EffectiveSort *effectiveSortResponse `json:"effective_sort,omitempty"`
}
type effectiveSortResponse struct {
Field string `json:"field"`
Order string `json:"order"`
}
// searchDiagnostics is an additive, per-query observability object emitted on
@@ -254,11 +244,6 @@ func (h *CatalogHandler) writeCatalogResponse(w http.ResponseWriter, result *cat
}
}
var effectiveSort *effectiveSortResponse
if field := strings.TrimSpace(result.EffectiveSort.Field); field != "" {
effectiveSort = &effectiveSortResponse{Field: field, Order: result.EffectiveSort.Order}
}
writeJSON(w, http.StatusOK, catalogResponse{
Total: result.Total,
TotalExact: result.TotalExact && !groupedByWork,
@@ -266,7 +251,6 @@ func (h *CatalogHandler) writeCatalogResponse(w http.ResponseWriter, result *cat
Items: items,
Snapshot: snapshot,
SearchDiagnostics: diag,
EffectiveSort: effectiveSort,
})
}
@@ -1,195 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"strings"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type collectionPreferenceLibraryReader interface {
GetByID(ctx context.Context, id string) (*models.LibraryCollection, error)
}
// collectionSortPreferenceRequest saves the sort a viewer chose while browsing a
// collection. An empty Field is meaningful — it pins the viewer to the
// collection's own source order even if its creator later configures a default.
// To go back to following the creator's default, DELETE the preference instead.
type collectionSortPreferenceRequest struct {
CollectionKind string `json:"collection_kind"`
CollectionID string `json:"collection_id"`
Field string `json:"field"`
Order string `json:"order"`
}
type collectionSortPreferenceResponse struct {
CollectionKind string `json:"collection_kind"`
CollectionID string `json:"collection_id"`
Field string `json:"field"`
Order string `json:"order"`
}
// NormalizeCollectionSortConfig validates the default sort a collection's
// creator configured and returns its canonical sort_config JSON. Shared by the
// personal and library collection handlers so both id spaces accept exactly the
// same vocabulary.
//
// allowPersonalized must be false for library collection defaults because
// those defaults are shared across profiles. Viewer overrides are validated
// separately and may use profile-scoped fields for either collection kind.
func NormalizeCollectionSortConfig(raw json.RawMessage, allowPersonalized bool) (string, error) {
trimmed := strings.TrimSpace(string(raw))
if trimmed == "" || trimmed == jsonNullLiteral || trimmed == "{}" {
return "{}", nil
}
var cfg struct {
Field string `json:"field"`
Order string `json:"order"`
}
if err := json.Unmarshal(raw, &cfg); err != nil {
return "", errInvalidSortConfig
}
if strings.TrimSpace(cfg.Field) == "" {
// sort_config predates default sorting and may contain independent
// collection modes such as {"mode":"manual_pins"}. Preserve those
// configurations byte-for-byte instead of erasing them merely because
// they do not contain a sort field.
return string(raw), nil
}
qs, ok := catalog.NormalizeCollectionSort(cfg.Field, cfg.Order, allowPersonalized)
if !ok {
return "", errInvalidSortConfig
}
return catalog.EncodeCollectionDefaultSort(qs, true)
}
var errInvalidSortConfig = &sortConfigError{}
type sortConfigError struct{}
func (*sortConfigError) Error() string {
return "sort_config must be {} or {\"field\":\"<supported sort>\",\"order\":\"asc|desc\"}"
}
// HandleSetCollectionSortPreference handles PUT /collections/sort-preference.
func (h *CollectionHandler) HandleSetCollectionSortPreference(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
var req collectionSortPreferenceRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
kind, collectionID, ok := normalizeCollectionRef(req.CollectionKind, req.CollectionID)
if !ok {
writeError(w, http.StatusBadRequest, "bad_request", "collection_kind must be 'library' or 'user' and collection_id is required")
return
}
field := strings.ToLower(strings.TrimSpace(req.Field))
order := ""
if field != "" {
qs, valid := catalog.NormalizeCollectionSort(field, req.Order, true)
if !valid {
writeError(w, http.StatusBadRequest, "bad_request", "Unsupported sort field or order for this collection")
return
}
field, order = qs.Field, qs.Order
}
store, err := h.storeProvider.ForUser(r.Context(), userID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to access user store")
return
}
if !h.canSaveCollectionSortPreference(r, store, kind, collectionID) {
// Use one not-found response for absent and inaccessible collections so
// the preference endpoint cannot be used to enumerate hidden IDs.
writeError(w, http.StatusNotFound, "not_found", "Collection not found")
return
}
if err := store.SetCollectionSortPreference(r.Context(), userstore.CollectionSortPreference{
ProfileID: profileID,
CollectionKind: kind,
CollectionID: collectionID,
SortField: field,
SortOrder: order,
}); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save sort preference")
return
}
writeJSON(w, http.StatusOK, collectionSortPreferenceResponse{
CollectionKind: kind,
CollectionID: collectionID,
Field: field,
Order: order,
})
}
func (h *CollectionHandler) canSaveCollectionSortPreference(
r *http.Request,
store userstore.UserStore,
kind, collectionID string,
) bool {
switch kind {
case userstore.CollectionKindLibrary:
if h.LibraryCollections == nil {
return false
}
collection, err := h.LibraryCollections.GetByID(r.Context(), collectionID)
return err == nil && catalog.CanAccessLibraryCollection(collection, requestAccessFilter(r))
case userstore.CollectionKindUser:
collection, err := store.GetCollection(r.Context(), collectionID)
return err == nil && catalog.ProfileCanAccessCollection(collection, apimw.GetProfileID(r.Context()))
default:
return false
}
}
// HandleClearCollectionSortPreference handles DELETE /collections/sort-preference,
// returning the collection to whatever default its creator configured.
func (h *CollectionHandler) HandleClearCollectionSortPreference(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
values := r.URL.Query()
kind, collectionID, ok := normalizeCollectionRef(values.Get("collection_kind"), values.Get("collection_id"))
if !ok {
writeError(w, http.StatusBadRequest, "bad_request", "collection_kind must be 'library' or 'user' and collection_id is required")
return
}
store, err := h.storeProvider.ForUser(r.Context(), userID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to access user store")
return
}
if err := store.ClearCollectionSortPreference(r.Context(), profileID, kind, collectionID); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to clear sort preference")
return
}
w.WriteHeader(http.StatusNoContent)
}
func normalizeCollectionRef(rawKind, rawID string) (string, string, bool) {
kind := strings.ToLower(strings.TrimSpace(rawKind))
if kind != userstore.CollectionKindLibrary && kind != userstore.CollectionKindUser {
return "", "", false
}
collectionID := strings.TrimSpace(rawID)
if collectionID == "" {
return "", "", false
}
return kind, collectionID, true
}
@@ -1,302 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
"github.com/Silo-Server/silo-server/internal/userstore/pgstore"
)
type sortPrefLibraryReader struct {
collection *models.LibraryCollection
}
func (r sortPrefLibraryReader) GetByID(_ context.Context, id string) (*models.LibraryCollection, error) {
if r.collection == nil || r.collection.ID != id {
return nil, catalog.ErrLibraryCollectionNotFound
}
return r.collection, nil
}
// These exercise the sort-preference endpoints through real HTTP handlers and a
// real store, using middleware.SetClaims / SetProfileID to supply the auth
// context the middleware would normally attach. Set SILO_TEST_DATABASE_URL to a
// migrated database to run them.
func sortPrefHandlerPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
var tableName *string
if err := pool.QueryRow(context.Background(),
`SELECT to_regclass('public.user_collection_sort_preferences')::text`).Scan(&tableName); err != nil {
t.Fatalf("check preference table: %v", err)
}
if tableName == nil || *tableName == "" {
t.Skip("test database has not applied the collection sort preference migration")
}
return pool
}
func sortPrefFixtureUser(t *testing.T, pool *pgxpool.Pool) int {
t.Helper()
var userID int
if err := pool.QueryRow(context.Background(), `
INSERT INTO users (username, email, password_hash, role)
VALUES ('sort-pref-handler-fixture', 'sort-pref-handler@invalid.test', '', 'user')
RETURNING id
`).Scan(&userID); err != nil {
t.Fatalf("seed fixture user: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM users WHERE id = $1`, userID)
})
return userID
}
// authed attaches the user + profile scope the middleware chain would normally
// have put on the request.
func authed(r *http.Request, userID int, profileID string) *http.Request {
ctx := middleware.SetClaims(r.Context(), &auth.Claims{UserID: userID, Role: "user"})
ctx = middleware.SetProfileID(ctx, profileID)
return r.WithContext(ctx)
}
func TestCollectionSortPreferenceEndpoints(t *testing.T) {
pool := sortPrefHandlerPool(t)
userID := sortPrefFixtureUser(t, pool)
const profileID = "profile-http"
const collectionID = "http-collection-1"
provider := pgstore.NewPostgresProvider(pool)
handler := NewCollectionHandler(provider)
handler.LibraryCollections = sortPrefLibraryReader{collection: &models.LibraryCollection{
ID: collectionID, Visibility: "visible", LibraryIDs: []int{7},
}}
store, err := provider.ForUser(context.Background(), userID)
if err != nil {
t.Fatalf("user store: %v", err)
}
if err := store.CreateProfile(context.Background(), userstore.Profile{ID: profileID, Name: "HTTP sort preference"}); err != nil {
t.Fatalf("seed profile: %v", err)
}
personalCollection, err := store.CreateCollection(context.Background(), userstore.CreateCollectionInput{
CreatorProfileID: profileID,
Name: "HTTP personal sort preference",
CollectionType: "manual",
})
if err != nil {
t.Fatalf("seed personal collection: %v", err)
}
t.Cleanup(func() {
_ = store.DeleteProfile(context.Background(), profileID)
})
put := func(body string) *httptest.ResponseRecorder {
req := authed(httptest.NewRequest(http.MethodPut, "/collections/sort-preference", strings.NewReader(body)), userID, profileID)
rec := httptest.NewRecorder()
handler.HandleSetCollectionSortPreference(rec, req)
return rec
}
t.Run("saves a sort the viewer picked", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"` + collectionID + `","field":"title","order":"asc"}`)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var resp collectionSortPreferenceResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
if resp.Field != "title" || resp.Order != "asc" {
t.Fatalf("response = %+v", resp)
}
// Read it back through the store, not the response, so the assertion
// covers persistence rather than echo.
pref, err := store.GetCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindLibrary, collectionID)
if err != nil || pref == nil {
t.Fatalf("stored preference = %v, err = %v", pref, err)
}
if pref.SortField != "title" || pref.SortOrder != "asc" {
t.Fatalf("stored preference = %+v", pref)
}
})
t.Run("fills in the field's default order", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"` + collectionID + `","field":"year"}`)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
pref, _ := store.GetCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindLibrary, collectionID)
if pref == nil || pref.SortOrder != "desc" {
t.Fatalf("stored preference = %+v, want year/desc", pref)
}
})
t.Run("an empty field pins to source order", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"` + collectionID + `","field":""}`)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
pref, _ := store.GetCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindLibrary, collectionID)
if pref == nil {
t.Fatal("pinning to source order removed the row; it must persist as an explicit choice")
}
if pref.SortField != "" {
t.Fatalf("stored field = %q, want empty", pref.SortField)
}
})
t.Run("accepts a personalized viewer sort on a library collection", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"` + collectionID + `","field":"progress"}`)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
pref, _ := store.GetCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindLibrary, collectionID)
if pref == nil || pref.SortField != "progress" || pref.SortOrder != "desc" {
t.Fatalf("stored preference = %+v, want progress/desc", pref)
}
})
t.Run("rejects an invalid sort order", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"` + collectionID + `","field":"title","order":"sideways"}`)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400; body = %s", rec.Code, rec.Body.String())
}
})
t.Run("accepts a personalized sort on a personal collection", func(t *testing.T) {
rec := put(`{"collection_kind":"user","collection_id":"` + personalCollection.ID + `","field":"progress"}`)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
t.Cleanup(func() {
_ = store.ClearCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindUser, personalCollection.ID)
})
})
t.Run("rejects a nonexistent collection", func(t *testing.T) {
rec := put(`{"collection_kind":"library","collection_id":"missing","field":"title"}`)
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404; body = %s", rec.Code, rec.Body.String())
}
})
t.Run("rejects a library collection outside the viewer scope", func(t *testing.T) {
req := authed(httptest.NewRequest(http.MethodPut, "/collections/sort-preference",
strings.NewReader(`{"collection_kind":"library","collection_id":"`+collectionID+`","field":"title"}`)), userID, profileID)
req = req.WithContext(access.SetScope(req.Context(), access.Scope{AllowedLibraryIDs: []int{99}}))
rec := httptest.NewRecorder()
handler.HandleSetCollectionSortPreference(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404; body = %s", rec.Code, rec.Body.String())
}
})
t.Run("rejects an unknown collection kind", func(t *testing.T) {
rec := put(`{"collection_kind":"nonsense","collection_id":"x","field":"title"}`)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", rec.Code)
}
})
t.Run("clearing removes the override", func(t *testing.T) {
req := authed(httptest.NewRequest(http.MethodDelete,
"/collections/sort-preference?collection_kind=library&collection_id="+collectionID, nil), userID, profileID)
rec := httptest.NewRecorder()
handler.HandleClearCollectionSortPreference(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
pref, err := store.GetCollectionSortPreference(context.Background(), profileID, userstore.CollectionKindLibrary, collectionID)
if err != nil {
t.Fatalf("reading cleared preference: %v", err)
}
if pref != nil {
t.Fatalf("preference still present after clear: %+v", pref)
}
})
}
// TestNormalizeCollectionSortConfigForCreators covers the validation the
// collection create/update handlers apply to a creator-configured default.
func TestNormalizeCollectionSortConfigForCreators(t *testing.T) {
cases := []struct {
name string
raw string
allowPersonalized bool
want string
wantErr bool
}{
{name: "absent", raw: "", want: "{}"},
{name: "explicit null", raw: `null`, want: "{}"},
{name: "legacy empty object", raw: `{}`, want: "{}"},
{name: "empty field remains source order", raw: `{"field":""}`, want: `{"field":""}`},
{name: "preserves manual pins mode", raw: `{"mode":"manual_pins"}`, want: `{"mode":"manual_pins"}`},
{name: "canonicalizes", raw: `{"field":" Title ","order":"ASC"}`, want: `{"field":"title","order":"asc"}`},
{name: "fills the default order", raw: `{"field":"year"}`, want: `{"field":"year","order":"desc"}`},
{name: "rejects unknown fields", raw: `{"field":"not_a_sort"}`, wantErr: true},
{name: "rejects invalid orders", raw: `{"field":"year","order":"sideways"}`, wantErr: true},
{name: "rejects personalized for library", raw: `{"field":"progress"}`, wantErr: true},
{
name: "allows personalized for personal",
raw: `{"field":"progress"}`,
allowPersonalized: true,
want: `{"field":"progress","order":"desc"}`,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := NormalizeCollectionSortConfig(json.RawMessage(tc.raw), tc.allowPersonalized)
if tc.wantErr {
if err == nil {
t.Fatalf("expected an error, got %q", got)
}
return
}
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != tc.want {
t.Fatalf("got %s, want %s", got, tc.want)
}
})
}
}
func TestNormalizeCollectionSortConfigPreservesManualPinsAcrossUpdates(t *testing.T) {
const manualPins = `{"mode":"manual_pins"}`
created, err := NormalizeCollectionSortConfig(json.RawMessage(manualPins), false)
if err != nil {
t.Fatalf("normalize on create: %v", err)
}
updated, err := NormalizeCollectionSortConfig(json.RawMessage(created), false)
if err != nil {
t.Fatalf("normalize on update: %v", err)
}
if updated != manualPins {
t.Fatalf("manual pins config changed across create/read/update: got %s, want %s", updated, manualPins)
}
}
+9 -24
View File
@@ -22,12 +22,11 @@ import (
// CollectionHandler handles personal collection CRUD endpoints.
type CollectionHandler struct {
storeProvider userstore.UserStoreProvider
LibraryCollections collectionPreferenceLibraryReader
Executor *catalog.QueryExecutor
S3GP *s3client.Client
HTTPClient *http.Client
PresignTTL time.Duration
storeProvider userstore.UserStoreProvider
Executor *catalog.QueryExecutor
S3GP *s3client.Client
HTTPClient *http.Client
PresignTTL time.Duration
}
// NewCollectionHandler creates a new CollectionHandler.
@@ -107,11 +106,8 @@ type collectionCapabilitiesResponse struct {
// DisplayFilterFields are the catalog query fields a personal-collection
// display filter may use. Clients build a display_query_definition fragment
// from these rather than a bespoke enum.
DisplayFilterFields []string `json:"display_filter_fields"`
DisplayFilterPresets collectionDisplayFilterPresets `json:"display_filter_presets"`
CollectionDefaultSort bool `json:"collection_default_sort"`
CollectionSortPreferences bool `json:"collection_sort_preferences"`
EffectiveCollectionSort bool `json:"effective_collection_sort"`
DisplayFilterFields []string `json:"display_filter_fields"`
DisplayFilterPresets collectionDisplayFilterPresets `json:"display_filter_presets"`
}
type collectionDisplayFilterPresets struct {
@@ -237,9 +233,6 @@ func (h *CollectionHandler) HandleCapabilities(w http.ResponseWriter, r *http.Re
Watched: []string{"all", "watched", "unwatched"},
Media: []string{"all", "movie", "series"},
},
CollectionDefaultSort: true,
CollectionSortPreferences: true,
EffectiveCollectionSort: true,
})
}
@@ -281,11 +274,7 @@ func (h *CollectionHandler) HandleCreateCollection(w http.ResponseWriter, r *htt
}
}
queryDefinition := string(queryDefinitionJSON)
sortConfig, err := NormalizeCollectionSortConfig(req.SortConfig, true)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
sortConfig := string(defaultJSON(req.SortConfig))
displayQueryDefinition, err := catalog.NormalizeDisplayQueryFragment(req.DisplayQueryDefinition)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
@@ -369,11 +358,7 @@ func (h *CollectionHandler) HandleUpdateCollection(w http.ResponseWriter, r *htt
input.QueryDefinition = &value
}
if len(req.SortConfig) > 0 {
value, err := NormalizeCollectionSortConfig(req.SortConfig, true)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
value := string(req.SortConfig)
input.SortConfig = &value
}
input.IsShared = req.IsShared
@@ -1,25 +0,0 @@
package handlers
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
func TestCollectionCapabilitiesAdvertiseSortSupport(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/collections/capabilities", nil)
NewCollectionHandler(nil).HandleCapabilities(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
var got collectionCapabilitiesResponse
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode response: %v", err)
}
if !got.CollectionDefaultSort || !got.CollectionSortPreferences || !got.EffectiveCollectionSort {
t.Fatalf("sort capabilities not fully advertised: %+v", got)
}
}
+2
View File
@@ -834,6 +834,8 @@ func (h *DownloadHandler) writeDownloadError(w http.ResponseWriter, err error) {
writeError(w, http.StatusTooManyRequests, "download_quota_exceeded", "Download quota exceeded for this period")
case errors.Is(err, downloads.ErrNoDownloadableEpisodes):
writeError(w, http.StatusNotFound, "no_downloadable_episodes", "No downloadable episodes found")
case errors.Is(err, downloads.ErrArtifactEvicted):
writeError(w, http.StatusConflict, "artifact_evicted", "The prepared download was reclaimed; retry the request")
case errors.Is(err, catalog.ErrItemNotFound):
writeError(w, http.StatusNotFound, "not_found", "Media item not found")
default:
+1
View File
@@ -417,6 +417,7 @@ func TestHandleCreateDownloadErrorMapping(t *testing.T) {
{"profile required", downloads.ErrProfileRequired, http.StatusBadRequest},
{"concurrent limit", downloads.ErrConcurrentLimitReached, http.StatusTooManyRequests},
{"period limit", downloads.ErrPeriodLimitReached, http.StatusTooManyRequests},
{"artifact evicted", downloads.ErrArtifactEvicted, http.StatusConflict},
{"item not found", catalog.ErrItemNotFound, http.StatusNotFound},
}
for _, tc := range cases {
+12 -29
View File
@@ -2264,42 +2264,17 @@ func (h *ItemsHandler) resolveWatchedTargets(ctx context.Context, contentID stri
}}, nil
}
// episodeTargets resolves the watched leaf targets for a season or series.
// Durations come from one batched file lookup rather than a query per episode:
// marking a long series used to issue one GetByEpisodeID per episode before it
// wrote anything.
func (h *ItemsHandler) episodeTargets(ctx context.Context, episodes []*models.Episode) []watchedLeafTarget {
episodeIDs := make([]string, 0, len(episodes))
for _, episode := range episodes {
episodeIDs = append(episodeIDs, episode.ContentID)
}
filesByEpisode := h.listEpisodeFiles(ctx, episodeIDs)
targets := make([]watchedLeafTarget, 0, len(episodes))
for _, episode := range episodes {
targets = append(targets, watchedLeafTarget{
ContentID: episode.ContentID,
DurationSeconds: mediaFileDurationSeconds(filesByEpisode[episode.ContentID], episode.Runtime),
DurationSeconds: h.contentDurationSeconds(ctx, "", episode.ContentID, episode.Runtime),
})
}
return targets
}
// mediaFileDurationSeconds picks a playable duration from an item's files,
// falling back to the catalog runtime. Mirrors contentDurationSeconds's
// preference order so the batched and single-item paths agree.
func mediaFileDurationSeconds(files []*models.MediaFile, fallbackRuntimeMinutes int) float64 {
for _, file := range files {
if file != nil && file.Duration > 0 {
return float64(file.Duration)
}
}
if fallbackRuntimeMinutes > 0 {
return float64(fallbackRuntimeMinutes * 60)
}
return 0
}
func (h *ItemsHandler) contentDurationSeconds(ctx context.Context, contentID, episodeID string, fallbackRuntimeMinutes int) float64 {
if h.fileRepo == nil {
if fallbackRuntimeMinutes > 0 {
@@ -2319,10 +2294,18 @@ func (h *ItemsHandler) contentDurationSeconds(ctx context.Context, contentID, ep
case contentID != "":
files, err = h.fileRepo.GetByContentID(ctx, contentID)
}
if err != nil {
files = nil
if err == nil {
for _, file := range files {
if file != nil && file.Duration > 0 {
return float64(file.Duration)
}
}
}
return mediaFileDurationSeconds(files, fallbackRuntimeMinutes)
if fallbackRuntimeMinutes > 0 {
return float64(fallbackRuntimeMinutes * 60)
}
return 0
}
// isNotFound checks if an error is a "not found" sentinel.
+60 -106
View File
@@ -33,28 +33,26 @@ import (
"github.com/Silo-Server/silo-server/internal/s3client"
"github.com/Silo-Server/silo-server/internal/sections"
"github.com/Silo-Server/silo-server/internal/usercollections"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type LibraryCollectionHandler struct {
repo *catalog.LibraryCollectionRepository
service *catalog.LibraryCollectionService
itemRepo *catalog.ItemRepository
Executor *catalog.QueryExecutor
detailSvc *catalog.DetailService
presignTTL time.Duration
httpClient *http.Client
s3GP *s3client.Client
FrontendFS fs.FS
SectionRepo *sections.Repository
UserCollectionPool *pgxpool.Pool
GroupRepo *catalog.LibraryCollectionGroupRepository
FolderRepo *catalog.FolderRepository
TemplateRegistry *templates.Registry
SmartCountRefresher *catalog.SmartCountRefresher
JobRepo *adminjob.Repository
EventsHub *evt.Hub
SortPreferenceCleaner *userstore.CollectionSortPreferenceCleaner
repo *catalog.LibraryCollectionRepository
service *catalog.LibraryCollectionService
itemRepo *catalog.ItemRepository
Executor *catalog.QueryExecutor
detailSvc *catalog.DetailService
presignTTL time.Duration
httpClient *http.Client
s3GP *s3client.Client
FrontendFS fs.FS
SectionRepo *sections.Repository
UserCollectionPool *pgxpool.Pool
GroupRepo *catalog.LibraryCollectionGroupRepository
FolderRepo *catalog.FolderRepository
TemplateRegistry *templates.Registry
SmartCountRefresher *catalog.SmartCountRefresher
JobRepo *adminjob.Repository
EventsHub *evt.Hub
}
var errLibraryCollectionInUse = errors.New("collection is used by one or more sections")
@@ -379,22 +377,21 @@ type updateLibraryCollectionRequest struct {
}
type importMDBListRequest struct {
SortConfig json.RawMessage `json:"sort_config,omitempty"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
Limit *int `json:"limit,omitempty"`
Featured bool `json:"featured"`
SortOrder int `json:"sort_order,omitempty"`
PosterURL string `json:"poster_url"`
PosterSourceURL string `json:"poster_source_url"`
BackdropSourceURL string `json:"backdrop_source_url"`
SyncSchedule string `json:"sync_schedule"`
ManagementMode string `json:"management_mode,omitempty"`
ManagementSource string `json:"management_source,omitempty"`
ManagementKey string `json:"management_key,omitempty"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
Limit *int `json:"limit,omitempty"`
Featured bool `json:"featured"`
SortOrder int `json:"sort_order,omitempty"`
PosterURL string `json:"poster_url"`
PosterSourceURL string `json:"poster_source_url"`
BackdropSourceURL string `json:"backdrop_source_url"`
SyncSchedule string `json:"sync_schedule"`
ManagementMode string `json:"management_mode,omitempty"`
ManagementSource string `json:"management_source,omitempty"`
ManagementKey string `json:"management_key,omitempty"`
// PosterFromTemplate marks the poster as supplied by a template bundle so
// future template applies can refresh it. Set programmatically by
// createCollectionFromTemplate; not user-settable via JSON.
@@ -402,25 +399,24 @@ type importMDBListRequest struct {
}
type importTMDBRequest struct {
SortConfig json.RawMessage `json:"sort_config,omitempty"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
Preset string `json:"preset"`
TimeWindow string `json:"time_window"`
MediaType string `json:"media_type"`
Limit *int `json:"limit,omitempty"`
Featured bool `json:"featured"`
SortOrder int `json:"sort_order,omitempty"`
PosterURL string `json:"poster_url"`
PosterSourceURL string `json:"poster_source_url"`
BackdropSourceURL string `json:"backdrop_source_url"`
SyncSchedule string `json:"sync_schedule"`
ManagementMode string `json:"management_mode,omitempty"`
ManagementSource string `json:"management_source,omitempty"`
ManagementKey string `json:"management_key,omitempty"`
PosterFromTemplate bool `json:"-"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
Preset string `json:"preset"`
TimeWindow string `json:"time_window"`
MediaType string `json:"media_type"`
Limit *int `json:"limit,omitempty"`
Featured bool `json:"featured"`
SortOrder int `json:"sort_order,omitempty"`
PosterURL string `json:"poster_url"`
PosterSourceURL string `json:"poster_source_url"`
BackdropSourceURL string `json:"backdrop_source_url"`
SyncSchedule string `json:"sync_schedule"`
ManagementMode string `json:"management_mode,omitempty"`
ManagementSource string `json:"management_source,omitempty"`
ManagementKey string `json:"management_key,omitempty"`
PosterFromTemplate bool `json:"-"`
}
// importTMDBFranchiseRequest is the request body for creating a collection
@@ -491,14 +487,13 @@ type importTMDBDiscoverSpecBody struct {
}
type importTraktRequest struct {
SortConfig json.RawMessage `json:"sort_config,omitempty"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
Preset string `json:"preset"`
MediaType string `json:"media_type"`
ProfileID string `json:"profile_id,omitempty"`
LibraryID int `json:"library_id"`
LibraryIDs []int `json:"library_ids"`
Title string `json:"title"`
Description string `json:"description"`
Preset string `json:"preset"`
MediaType string `json:"media_type"`
ProfileID string `json:"profile_id,omitempty"`
// ListURL selects a user-authored Trakt list
// (https://trakt.tv/users/{user}/lists/{slug}) instead of a preset.
ListURL string `json:"list_url,omitempty"`
@@ -1142,14 +1137,6 @@ func (h *LibraryCollectionHandler) HandleCreateAdminCollection(w http.ResponseWr
}
}
// Library collections resolve with user scope stripped, so per-profile sort
// fields are rejected here rather than failing at browse time.
sortConfig, err := NormalizeCollectionSortConfig(req.SortConfig, false)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
var syncSchedule *string
if s := strings.TrimSpace(req.SyncSchedule); s != "" {
if err := catalog.ParseCronExpression(s); err != nil {
@@ -1179,7 +1166,7 @@ func (h *LibraryCollectionHandler) HandleCreateAdminCollection(w http.ResponseWr
BackdropURL: req.BackdropURL,
SourceURL: req.SourceURL,
QueryDefinition: queryDefinition,
SortConfig: json.RawMessage(sortConfig),
SortConfig: defaultJSON(req.SortConfig),
SourceConfig: defaultCollectionSourceConfig(req.SourceConfig),
ManagementMode: managementMode,
ManagementSource: managementSource,
@@ -1238,16 +1225,6 @@ func (h *LibraryCollectionHandler) HandleUpdateAdminCollection(w http.ResponseWr
}
}
sortConfig := req.SortConfig
if len(req.SortConfig) > 0 {
normalized, err := NormalizeCollectionSortConfig(req.SortConfig, false)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
sortConfig = json.RawMessage(normalized)
}
// Validate sync_schedule if provided.
if req.SyncSchedule != nil && *req.SyncSchedule != "" {
if err := catalog.ParseCronExpression(*req.SyncSchedule); err != nil {
@@ -1280,7 +1257,7 @@ func (h *LibraryCollectionHandler) HandleUpdateAdminCollection(w http.ResponseWr
BackdropURL: req.BackdropURL,
SourceURL: req.SourceURL,
QueryDefinition: queryDefinition,
SortConfig: sortConfig,
SortConfig: req.SortConfig,
SourceConfig: req.SourceConfig,
ManagementMode: normalizeOptionalCollectionManagementMode(req.ManagementMode),
ManagementSource: req.ManagementSource,
@@ -1454,13 +1431,7 @@ func (h *LibraryCollectionHandler) deleteServerCollection(ctx context.Context, c
}
}
if err := h.repo.Delete(ctx, collectionID); err != nil {
return err
}
if h.SortPreferenceCleaner != nil {
h.SortPreferenceCleaner.DeleteForCollection(ctx, userstore.CollectionKindLibrary, collectionID)
}
return nil
return h.repo.Delete(ctx, collectionID)
}
type adminReorderCollectionsRequest struct {
@@ -2665,10 +2636,6 @@ func (h *LibraryCollectionHandler) createMDBListCollection(
if err != nil {
return nil, requestValidationError{err: err}
}
sortConfig, err := NormalizeCollectionSortConfig(req.SortConfig, false)
if err != nil {
return nil, requestValidationError{err: err}
}
collection, err := h.repo.Create(ctx, catalog.CreateLibraryCollectionInput{
LibraryID: req.LibraryID,
LibraryIDs: req.LibraryIDs,
@@ -2676,7 +2643,6 @@ func (h *LibraryCollectionHandler) createMDBListCollection(
Title: req.Title,
Description: req.Description,
CollectionType: "mdblist",
SortConfig: json.RawMessage(sortConfig),
Visibility: "visible",
Featured: req.Featured,
SortOrder: req.SortOrder,
@@ -2730,10 +2696,6 @@ func (h *LibraryCollectionHandler) createTMDBCollection(
if err != nil {
return nil, requestValidationError{err: err}
}
tmdbSortConfig, err := NormalizeCollectionSortConfig(req.SortConfig, false)
if err != nil {
return nil, requestValidationError{err: err}
}
collection, err := h.repo.Create(ctx, catalog.CreateLibraryCollectionInput{
LibraryID: req.LibraryID,
LibraryIDs: req.LibraryIDs,
@@ -2741,7 +2703,6 @@ func (h *LibraryCollectionHandler) createTMDBCollection(
Title: req.Title,
Description: req.Description,
CollectionType: "tmdb",
SortConfig: json.RawMessage(tmdbSortConfig),
Visibility: "visible",
Featured: req.Featured,
SortOrder: req.SortOrder,
@@ -3087,12 +3048,6 @@ func (h *LibraryCollectionHandler) HandleImportTraktCollection(w http.ResponseWr
return
}
traktSortConfig, err := NormalizeCollectionSortConfig(req.SortConfig, false)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
collection, err := h.repo.Create(r.Context(), catalog.CreateLibraryCollectionInput{
LibraryID: req.LibraryID,
LibraryIDs: req.LibraryIDs,
@@ -3100,7 +3055,6 @@ func (h *LibraryCollectionHandler) HandleImportTraktCollection(w http.ResponseWr
Title: req.Title,
Description: req.Description,
CollectionType: "trakt",
SortConfig: json.RawMessage(traktSortConfig),
Visibility: "visible",
Featured: req.Featured,
PosterURL: req.PosterURL,
+10 -29
View File
@@ -43,18 +43,12 @@ import (
type SessionManagerInterface interface {
StartSession(userID int, profileID string, fileID int, method playback.PlayMethod, transcodeAudio bool) (*playback.Session, error)
StartSessionWithFiles(userID int, profileID string, effectiveFileID int, requestedFileID int, method playback.PlayMethod, transcodeAudio bool) (*playback.Session, error)
// StartSessionWithFilesContext is required rather than probed for at run
// time: the context is how the reporting client's identity reaches the new
// session, so an implementation without it would start sessions that
// silently carry no client name, version, build or channel.
StartSessionWithFilesContext(ctx context.Context, userID int, profileID string, effectiveFileID int, requestedFileID int, method playback.PlayMethod, transcodeAudio bool) (*playback.Session, error)
UpdateProgress(sessionID string, position float64, isPaused bool) error
UpdateAudioTrack(sessionID string, audioTrackIndex int, method playback.PlayMethod) error
UpdateStreamState(sessionID string, state playback.SessionStreamState) error
TouchActivity(sessionID string) error
BeginTransport(sessionID string) error
EndTransport(sessionID string) error
SetRemoteTransport(sessionID string, remote bool) error
SetEffectiveMediaFileID(sessionID string, fileID int) error
SetTranscodeNodeURL(sessionID, url string) error
SetTranscodeRoute(sessionID string, route playback.TranscodeRoute) error
@@ -68,6 +62,10 @@ type SessionManagerInterface interface {
GetSession(sessionID string) (*playback.Session, error)
}
type sessionStarterWithFilesContext interface {
StartSessionWithFilesContext(ctx context.Context, userID int, profileID string, effectiveFileID int, requestedFileID int, method playback.PlayMethod, transcodeAudio bool) (*playback.Session, error)
}
type transcodePermissionChecker interface {
CheckTranscodingAllowed(ctx context.Context, userID int, requiresVideoTranscode bool) error
}
@@ -398,20 +396,12 @@ const streamTokenParam = "st"
// reconstruction recipe. Returns "" when no signing secret is configured
// (reconstruct effectively disabled, e.g. in tests).
func (h *PlaybackHandler) signSessionToken(card playback.RecipeCard) string {
return h.signStreamClaims(card.ToClaims())
}
// signStreamClaims mints a stream token from claims that are already assembled.
// Callers serving a session from another node use it to add the claims a
// RecipeCard does not model (the file's Dolby Vision profile, audio-only flag),
// which a remote executor cannot look up for itself.
func (h *PlaybackHandler) signStreamClaims(claims streamtoken.Claims) string {
if h.JWTSecret == "" {
return ""
}
token, err := streamtoken.Sign(claims, h.JWTSecret, playback.MaxTokenTTL)
token, err := streamtoken.Sign(card.ToClaims(), h.JWTSecret, playback.MaxTokenTTL)
if err != nil {
slog.Warn("sign stream token failed", "error", err, "session", claims.SessionID, "playback_session_id", claims.SessionID)
slog.Warn("sign stream token failed", "error", err, "session", card.SessionID, "playback_session_id", card.SessionID)
return ""
}
return token
@@ -953,9 +943,7 @@ func (h *PlaybackHandler) handleExpiredSession(session *playback.Session) {
}
sessionCopy := *session
go func() {
slog.Info("expired inactive playback session", append([]any{
"session", sessionCopy.ID, "playback_session_id", sessionCopy.ID,
}, sessionCopy.ClientInfo().LogAttrs()...)...)
slog.Info("expired inactive playback session", "session", sessionCopy.ID, "playback_session_id", sessionCopy.ID)
// Expiry is a liveness reap, not a user stop — keep the recipe card so a
// resume reconstructs under the same id (the card's own TTL reaps it if
// the session is truly abandoned).
@@ -1079,18 +1067,11 @@ func playbackClientInfoFromRequest(r *http.Request) playback.ClientInfo {
if r == nil {
return playback.ClientInfo{}
}
// Clamped here, at the boundary, rather than only where the session stamps
// them: the decision logs and playback_route_events are written from this
// value directly, so a client sending a header-sized build would otherwise
// reach both despite the published bound. Values stay opaque — trimmed and
// length-clamped, never parsed or validated against an enum.
return playback.ClientInfo{
Name: r.Header.Get("X-Silo-Client"),
Version: r.Header.Get("X-Silo-Client-Version"),
Build: r.Header.Get("X-Silo-Client-Build"),
Channel: r.Header.Get("X-Silo-Client-Channel"),
Name: strings.TrimSpace(r.Header.Get("X-Silo-Client")),
Version: strings.TrimSpace(r.Header.Get("X-Silo-Client-Version")),
UserAgent: r.UserAgent(),
}.Normalized()
}
}
// HandleUpdateProgress handles POST /playback/{session_id}/progress.
+2 -58
View File
@@ -48,10 +48,7 @@ type playbackSessionRow struct {
ClientIP string `json:"client_ip,omitempty"`
ClientName string `json:"client_name,omitempty"`
ClientVersion string `json:"client_version,omitempty"`
ClientBuild string `json:"client_build,omitempty"`
ClientChannel string `json:"client_channel,omitempty"`
ClientLabel string `json:"client_label,omitempty"`
ClientLabelFull string `json:"client_label_full,omitempty"`
ClientUserAgent string `json:"client_user_agent,omitempty"`
AudioTrackIndex int `json:"audio_track_index"`
TranscodeAudio bool `json:"transcode_audio"`
@@ -92,11 +89,6 @@ type playbackSessionsCapabilitiesResponse struct {
EffectivePlayMethodValues []string `json:"effective_play_method_values"`
// IsJellyfinClient reports that rows carry is_jellyfin_client.
IsJellyfinClient bool `json:"is_jellyfin_client"`
// ClientBuild reports that rows carry client_build (and the exact-version
// client_label_full derived from it).
ClientBuild bool `json:"client_build"`
// ClientChannel reports that rows carry client_channel.
ClientChannel bool `json:"client_channel"`
}
// HandleGetSessionsCapabilities exposes additive feature support for the live
@@ -106,8 +98,6 @@ func (h *AdminHandler) HandleGetSessionsCapabilities(w http.ResponseWriter, _ *h
EffectivePlayMethod: true,
EffectivePlayMethodValues: []string{"direct", "remux", "transcode", "audio"},
IsJellyfinClient: true,
ClientBuild: true,
ClientChannel: true,
})
}
@@ -192,8 +182,6 @@ func (l *PlaybackSessionsLoader) Load(
COALESCE(HOST(s.client_ip), ''),
COALESCE(s.client_name, ''),
COALESCE(s.client_version, ''),
COALESCE(s.client_build, ''),
COALESCE(s.client_channel, ''),
COALESCE(s.client_user_agent, ''),
COALESCE(s.audio_track_index, 0),
COALESCE(s.transcode_audio, FALSE),
@@ -251,7 +239,6 @@ func (l *PlaybackSessionsLoader) Load(
&posterPath,
&s.PlayMethod, &s.ReportingNode, &s.NodeDisplayName, &s.FileDuration, &s.StartedAt, &s.UpdatedAt,
&s.PositionSeconds, &s.IsPaused, &s.HasPlaybackControl, &s.ClientIP, &s.ClientName, &s.ClientVersion,
&s.ClientBuild, &s.ClientChannel,
&s.ClientUserAgent, &s.AudioTrackIndex, &s.TranscodeAudio, &streamBitrateKbps,
&s.TranscodeNodeURL, &s.TargetResolution, &s.TargetVideoCodec, &s.TargetAudioCodec, &targetBitrateKbps,
&s.TranscodeHWAccel, &s.SourceContainer, &sourceBitrateKbps, &s.SourceVideoCodec, &s.SourceVideoResolution,
@@ -266,13 +253,6 @@ func (l *PlaybackSessionsLoader) Load(
s.SourceBitrateKbps = sourceBitrateKbps
s.SourceAudioChannels = sourceAudioChannels
s.ClientLabel = playbackClientDisplayName(s.ClientName, s.ClientVersion, s.ClientUserAgent)
// Only worth the bytes when it says more than the compact label — which
// it does not for any client without a build or a non-release channel,
// i.e. most of a 200-row page. Clients read client_label when
// client_label_full is absent, so omitting the duplicate costs nothing.
if full := playbackClientFullDisplayName(s.ClientName, s.ClientVersion, s.ClientBuild, s.ClientChannel, s.ClientUserAgent); full != s.ClientLabel {
s.ClientLabelFull = full
}
enrichPlaybackSessionRow(&s, audioTracksJSON)
sessions = append(sessions, s)
}
@@ -467,47 +447,11 @@ func firstNonEmptyValue(values ...string) string {
return ""
}
// playbackClientFullDisplayName renders the client's exact identity — version,
// opaque build, and non-default channel — for surfaces that can afford the
// width (expanded session details, tooltips). The name-and-version half is
// whatever the compact label resolved, so a client identified only by its user
// agent still gets its build named: a client that bothered to report a build
// has earned having it displayed, whether or not it also named itself.
func playbackClientFullDisplayName(name, version, build, channel, userAgent string) string {
label := playbackClientDisplayName(name, version, userAgent)
if label == "" {
return ""
}
build = strings.TrimSpace(build)
channel = strings.TrimSpace(channel)
// "release" is the assumed channel, so naming it adds width without adding
// information; every other channel is worth calling out.
if strings.EqualFold(channel, "release") {
channel = ""
}
var qualifiers []string
if build != "" {
qualifiers = append(qualifiers, "build "+build)
}
if channel != "" {
qualifiers = append(qualifiers, channel)
}
if len(qualifiers) > 0 {
label += " (" + strings.Join(qualifiers, ", ") + ")"
}
return label
}
// playbackClientDisplayName renders the compact label the session lists show.
// A client that reports its own name keeps its version verbatim — truncating
// "1.0.0" to "1" hid the only field that identifies the running build. Labels
// derived from a user agent stay shortened, where a full browser version
// ("120.0.6099.109") is noise.
func playbackClientDisplayName(name, version, userAgent string) string {
name = strings.TrimSpace(name)
version = shortPlaybackClientVersion(version)
if name != "" {
if version = strings.TrimSpace(version); version != "" {
if version != "" {
return name + " " + version
}
return name
+2 -274
View File
@@ -4,12 +4,7 @@ import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"unicode"
"unicode/utf8"
"github.com/Silo-Server/silo-server/internal/playback"
)
func TestSessionComponentDecisionLabelsCopiedAudioDuringHLSAsRemux(t *testing.T) {
@@ -70,8 +65,8 @@ func TestSessionsCapabilitiesAdvertisesActivityFields(t *testing.T) {
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode capabilities: %v", err)
}
if !resp.EffectivePlayMethod || !resp.IsJellyfinClient || !resp.ClientBuild || !resp.ClientChannel {
t.Fatalf("capabilities must advertise every additive field: %+v", resp)
if !resp.EffectivePlayMethod || !resp.IsJellyfinClient {
t.Fatalf("capabilities must advertise both fields: %+v", resp)
}
want := []string{"direct", "remux", "transcode", "audio"}
if len(resp.EffectivePlayMethodValues) != len(want) {
@@ -218,273 +213,6 @@ func TestPlaybackClientDisplayNameAndroidDevices(t *testing.T) {
}
}
func TestPlaybackClientDisplayNameKeepsNamedClientVersionExact(t *testing.T) {
cases := []struct {
name string
clientName string
clientVersion string
want string
}{
{name: "patch version survives", clientName: "Silo Android TV", clientVersion: "1.0.0", want: "Silo Android TV 1.0.0"},
{name: "prerelease suffix survives", clientName: "Silo iOS", clientVersion: "2.1.0-rc.3", want: "Silo iOS 2.1.0-rc.3"},
{name: "no version", clientName: "Silo Android TV", clientVersion: "", want: "Silo Android TV"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := playbackClientDisplayName(tc.clientName, tc.clientVersion, "")
if got != tc.want {
t.Fatalf("playbackClientDisplayName(%q, %q, \"\") = %q, want %q", tc.clientName, tc.clientVersion, got, tc.want)
}
})
}
}
func TestPlaybackClientFullDisplayName(t *testing.T) {
cases := []struct {
name string
client string
version string
build string
channel string
userAgent string
want string
}{
{
name: "version build and channel",
client: "Silo Android TV",
version: "1.0.0",
build: "5",
channel: "dev",
want: "Silo Android TV 1.0.0 (build 5, dev)",
},
{
name: "release channel omitted",
client: "Silo Android TV",
version: "1.0.0",
build: "5",
channel: "release",
want: "Silo Android TV 1.0.0 (build 5)",
},
{
name: "release channel omitted case insensitively",
client: "Silo Android TV",
version: "1.0.0",
build: "5",
channel: "Release",
want: "Silo Android TV 1.0.0 (build 5)",
},
{
name: "empty channel omitted",
client: "Silo Android TV",
version: "1.0.0",
build: "5",
want: "Silo Android TV 1.0.0 (build 5)",
},
{
name: "channel without build",
client: "Silo Android TV",
version: "1.0.0",
channel: "dev",
want: "Silo Android TV 1.0.0 (dev)",
},
{
name: "build and channel empty",
client: "Silo Android TV",
version: "1.0.0",
want: "Silo Android TV 1.0.0",
},
{
name: "version empty",
client: "Silo Android TV",
want: "Silo Android TV",
},
{
name: "opaque build is not parsed",
client: "Silo tvOS",
version: "1.0.0",
build: "2026.08.13-abcdef",
want: "Silo tvOS 1.0.0 (build 2026.08.13-abcdef)",
},
{
// A client that reports a build but not a name is still named by its
// build: the user-agent label supplies the product half, and dropping
// the qualifiers would hide the exact field this surface exists for.
name: "no client name keeps build and channel on the user agent label",
userAgent: "Mozilla/5.0 (X11; Linux x86_64) Chrome/120.0.0.0 Safari/537.36",
build: "5",
channel: "dev",
want: "Chrome 120 (build 5, dev)",
},
{
name: "no client name and no qualifiers keeps the plain user agent label",
userAgent: "Mozilla/5.0 (X11; Linux x86_64) Chrome/120.0.0.0 Safari/537.36",
want: "Chrome 120",
},
{
name: "no client name and no user agent",
want: "",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := playbackClientFullDisplayName(tc.client, tc.version, tc.build, tc.channel, tc.userAgent)
if got != tc.want {
t.Fatalf("playbackClientFullDisplayName() = %q, want %q", got, tc.want)
}
})
}
}
func TestPlaybackClientInfoForStartV3(t *testing.T) {
cases := []struct {
name string
headers map[string]string
context playback.ClientPlaybackContextV3
want playback.ClientInfo
}{
{
name: "headers win over the body",
headers: map[string]string{"X-Silo-Client": "Silo iOS", "X-Silo-Client-Version": "2.1.0", "X-Silo-Client-Build": "9", "X-Silo-Client-Channel": "beta"},
context: playback.ClientPlaybackContextV3{AppVersion: "1.0.0", AppBuild: "1", AppChannel: "release"},
want: playback.ClientInfo{Name: "Silo iOS", Version: "2.1.0", Build: "9", Channel: "beta"},
},
{
// The fallback is per field, not per struct: a client may set the
// name and version headers on every request and still report the
// build it only knows at start time in the body.
name: "body fills only the fields the headers omit",
headers: map[string]string{"X-Silo-Client": "Silo tvOS", "X-Silo-Client-Version": "2.1.0"},
context: playback.ClientPlaybackContextV3{AppVersion: "1.0.0", AppBuild: "77", AppChannel: "sideload"},
want: playback.ClientInfo{Name: "Silo tvOS", Version: "2.1.0", Build: "77", Channel: "sideload"},
},
{
name: "body supplies everything but the name",
headers: map[string]string{"X-Silo-Client": "Silo Android TV"},
context: playback.ClientPlaybackContextV3{AppVersion: "1.0.0", AppBuild: "5", AppChannel: "dev"},
want: playback.ClientInfo{Name: "Silo Android TV", Version: "1.0.0", Build: "5", Channel: "dev"},
},
{
// The regression this guards: the web player reports the literal
// "web" as its app_version and sends no X-Silo-Client. Taking the
// body anyway would stamp "web" onto client_version — the one field
// the contract promises is a marketing version — for every browser
// session, and onto every route event and decision log with it.
name: "nameless client keeps the body out of client_version",
context: playback.ClientPlaybackContextV3{AppVersion: "web"},
want: playback.ClientInfo{},
},
{
name: "nameless client takes no build or channel either",
context: playback.ClientPlaybackContextV3{AppVersion: "web", AppBuild: "5", AppChannel: "dev"},
want: playback.ClientInfo{},
},
{
name: "whitespace-only header falls through to the body",
headers: map[string]string{"X-Silo-Client": "Silo iOS", "X-Silo-Client-Build": " "},
context: playback.ClientPlaybackContextV3{AppBuild: " 42 "},
want: playback.ClientInfo{Name: "Silo iOS", Build: "42"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/api/v1/playback/start", nil)
for name, value := range tc.headers {
req.Header.Set(name, value)
}
got := playbackClientInfoForStartV3(req, tc.context)
got.UserAgent = ""
if got != tc.want {
t.Fatalf("playbackClientInfoForStartV3() = %+v, want %+v", got, tc.want)
}
})
}
}
// TestPlaybackClientInfoFromRequestClampsHeaders pins the bound at the request
// boundary rather than at session creation. The resolved identity is written
// straight to the plan-decision log and to playback_route_events, so a client
// sending a header-sized build would reach both if the clamp only happened
// where the session stamps its fields.
func TestPlaybackClientInfoFromRequestClampsHeaders(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/api/v1/playback/start", nil)
req.Header.Set("X-Silo-Client", " "+strings.Repeat("N", 200)+" ")
req.Header.Set("X-Silo-Client-Version", strings.Repeat("v", 100))
req.Header.Set("X-Silo-Client-Build", strings.Repeat("b", 100))
req.Header.Set("X-Silo-Client-Channel", strings.Repeat("c", 100))
got := playbackClientInfoFromRequest(req)
for _, tc := range []struct {
field string
value string
want int
}{
{"Name", got.Name, 128},
{"Version", got.Version, 64},
{"Build", got.Build, 64},
{"Channel", got.Channel, 32},
} {
if len(tc.value) != tc.want {
t.Errorf("%s length = %d, want %d", tc.field, len(tc.value), tc.want)
}
}
}
// TestNormalizeClientMetadataCountsRunes guards the two ways a clamp can go
// wrong on a client-supplied header: the published bound is JSON Schema
// maxLength, which counts characters, and a byte-wise cut through a multi-byte
// rune yields invalid UTF-8 that Postgres refuses — failing the whole per-node
// session-sync transaction, not just the offending row.
func TestNormalizeClientMetadataCountsRunes(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/api/v1/playback/start", nil)
// 40 three-byte runes: within the 64-character channel bound as the schema
// counts it, well past it as bytes.
req.Header.Set("X-Silo-Client-Channel", strings.Repeat("δ", 40))
got := playbackClientInfoFromRequest(req)
if runes := utf8.RuneCountInString(got.Channel); runes != 32 {
t.Errorf("Channel runes = %d, want the 32-character bound", runes)
}
if !utf8.ValidString(got.Channel) {
t.Errorf("Channel = %q is not valid UTF-8; a text column would reject it", got.Channel)
}
}
// TestPlaybackClientInfoStripsControlCharacters guards the one malformed value
// that reaches a text column looking perfectly well-formed. A JSON NUL escape in a
// v3 start body decodes to a real NUL, which is valid UTF-8 — so the UTF-8
// repair above leaves it and TrimSpace does not consider it whitespace — but
// Postgres refuses NUL in text. The per-node session upserts share one
// transaction, so one such start would stop every live session on that node
// from reconciling. Headers cannot carry it (net/http rejects bytes below
// 0x20), which is exactly why the body path needs its own guard.
func TestPlaybackClientInfoStripsControlCharacters(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/api/v1/playback/start", nil)
req.Header.Set("X-Silo-Client", "Silo Android TV")
got := playbackClientInfoForStartV3(req, playback.ClientPlaybackContextV3{
AppVersion: "1.0\x00.0",
AppBuild: "5\x00",
AppChannel: "de\nv",
})
if want := "1.0.0"; got.Version != want {
t.Errorf("Version = %q, want %q", got.Version, want)
}
if want := "5"; got.Build != want {
t.Errorf("Build = %q, want %q", got.Build, want)
}
if want := "dev"; got.Channel != want {
t.Errorf("Channel = %q, want %q", got.Channel, want)
}
for _, field := range []string{got.Name, got.Version, got.Build, got.Channel} {
if strings.ContainsFunc(field, unicode.IsControl) {
t.Errorf("%q still carries a control character; a text column would reject it", field)
}
}
}
func TestEnrichPlaybackSessionRowUsesCompatOrigin(t *testing.T) {
row := playbackSessionRow{
ClientName: "Unrecognized Client",
-6
View File
@@ -167,10 +167,6 @@ func (failingSessionManager) StartSessionWithFiles(int, string, int, int, playba
return nil, errors.New("boom")
}
func (failingSessionManager) StartSessionWithFilesContext(context.Context, int, string, int, int, playback.PlayMethod, bool) (*playback.Session, error) {
return nil, errors.New("boom")
}
func (failingSessionManager) UpdateProgress(string, float64, bool) error { return nil }
func (failingSessionManager) UpdateAudioTrack(string, int, playback.PlayMethod) error { return nil }
@@ -185,8 +181,6 @@ func (failingSessionManager) BeginTransport(string) error { return nil }
func (failingSessionManager) EndTransport(string) error { return nil }
func (failingSessionManager) SetRemoteTransport(string, bool) error { return nil }
func (failingSessionManager) SetEffectiveMediaFileID(string, int) error { return nil }
func (failingSessionManager) SetTranscodeNodeURL(string, string) error { return nil }
+25 -287
View File
@@ -198,12 +198,6 @@ type transcodeNodeEnumeratorV3 interface {
TranscodeNodeURLs() []string
}
// proxyNodeEnumeratorV3 is the proxy-pool counterpart, letting identity
// planning narrow selection to proxies that can execute the plan's recipe.
type proxyNodeEnumeratorV3 interface {
ProxyNodeURLs() []string
}
// hlsPlanningRegistryV3 returns the registry HLS deliveries plan against: the
// local probe plus every pooled transcode node's advertised transformations.
// Only availability of locally-defined specs widens (name and recipe version
@@ -457,25 +451,20 @@ func (h *PlaybackHandler) handleStartPlaybackV3(w http.ResponseWriter, r *http.R
}
}
h.clarifyOriginalQuality4KTerminalV3(r.Context(), result.Terminal, requestedFile, !shouldTryAlternateFileV3(req.QualityPreference))
// The exact app identity is logged with every decision so a route or
// terminal reported against one build is attributable without asking the
// user which version they are running.
clientInfo := playbackClientInfoForStartV3(r, req.ClientPlaybackContext)
if result.Terminal != nil {
slog.InfoContext(r.Context(), "playback plan decided", append([]any{
logComponentKey, "playback",
slog.InfoContext(r.Context(), "playback plan decided", "component", "playback",
"outcome", "terminal",
"reason", result.Terminal.Reason,
"file_id", effectiveFile.ID,
"quality_preference", req.QualityPreference,
}, clientInfo.LogAttrs()...)...)
)
response, persistErr := h.persistTerminalStartDecisionV3(r.Context(), userID, profileID, req, requestDigests, requestedFile.ID, effectiveFile.ID, playback.NewTerminalResponseV3(result.Terminal.Reason, result.Terminal.Message, result.Terminal.Retryable))
if persistErr != nil {
writeStartAttemptPersistenceErrorV3(w, persistErr)
return
}
if response.Terminal != nil {
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, Event: playback.RouteEventTerminalV3, FallbackReason: response.Terminal.Reason, OutputContextID: req.ClientPlaybackContext.Output.OutputContextID}, UserID: userID, ProfileID: profileID, ClientName: clientInfo.Name, ClientVersion: clientInfo.Version, ClientBuild: clientInfo.Build, ClientChannel: clientInfo.Channel, ClientModel: req.ClientPlaybackContext.Device.Model})
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, Event: playback.RouteEventTerminalV3, FallbackReason: response.Terminal.Reason, OutputContextID: req.ClientPlaybackContext.Output.OutputContextID}, UserID: userID, ProfileID: profileID, ClientName: playbackClientInfoFromRequest(r).Name, ClientVersion: playbackClientInfoFromRequest(r).Version, ClientModel: req.ClientPlaybackContext.Device.Model})
}
writeJSON(w, http.StatusCreated, response)
return
@@ -483,8 +472,7 @@ func (h *PlaybackHandler) handleStartPlaybackV3(w http.ResponseWriter, r *http.R
// One line per plan decision so route selection is reconstructible from
// server logs alone (finding a mis-planned route previously required
// correlating client logcat, ffmpeg commands, and session rows).
slog.InfoContext(r.Context(), "playback plan decided", append([]any{
logComponentKey, "playback",
slog.InfoContext(r.Context(), "playback plan decided", "component", "playback",
"outcome", "plan",
"decision_reason", result.Plan.DecisionReason,
"delivery", result.Plan.Delivery,
@@ -497,9 +485,9 @@ func (h *PlaybackHandler) handleStartPlaybackV3(w http.ResponseWriter, r *http.R
"target_bitrate_kbps", result.TargetBitrateKbps,
"quality_preference", req.QualityPreference,
"bandwidth_estimate_kbps", intOrZeroHandlerV3(req.BandwidthEstimateKbps),
}, clientInfo.LogAttrs()...)...)
)
result.Plan.DegradationWarnings = append(result.Plan.DegradationWarnings, warnings...)
response, statusErr := h.startPlannedPlaybackV3(r, userID, profileID, req, requestDigests, requestedFile, effectiveFile, audioIndex, result, clientInfo)
response, statusErr := h.startPlannedPlaybackV3(r, userID, profileID, req, requestDigests, requestedFile, effectiveFile, audioIndex, result)
if statusErr != nil {
if statusErr.reason == "playback_attempt_reused" {
writeError(w, http.StatusConflict, "playback_attempt_reused", statusErr.message)
@@ -524,70 +512,6 @@ type playbackStartRequestDigestsV3 struct {
legacy string
}
// playbackClientInfoForStartV3 resolves the client's app identity for a v3
// start request. The X-Silo-Client-* headers win because they are present on
// every request; the start body's client_playback_context is the fallback for
// clients that report their app identity only there. All values stay opaque —
// they are trimmed and length-clamped when the session stamps them.
//
// The fallback applies only to a client that named itself. client_playback_context
// carries no app name, so nothing in the body can identify a nameless client
// anyway — it is labeled from its user agent, and its app_version is a
// free-form platform string rather than the marketing version client_version
// promises. The web player, for one, reports the literal "web" there; taking it
// unconditionally would write "web" into the one field that is contractually
// semver, on every browser session.
func playbackClientInfoForStartV3(r *http.Request, clientContext playback.ClientPlaybackContextV3) playback.ClientInfo {
info := playbackClientInfoFromRequest(r)
if info.Name == "" {
return info
}
if info.Version == "" {
info.Version = strings.TrimSpace(clientContext.AppVersion)
}
if info.Build == "" {
info.Build = strings.TrimSpace(clientContext.AppBuild)
}
if info.Channel == "" {
info.Channel = strings.TrimSpace(clientContext.AppChannel)
}
// The header half is already normalized; body-sourced values have to be
// clamped too before they reach the decision log and the route event.
return info.Normalized()
}
// playbackClientInfoWithSessionFallbackV3 completes a header-derived identity
// from the session the event belongs to. Route events posted out of band carry
// no client_playback_context, so without this a client that reports its build
// only in the start body would attribute its plan_selected event to a build and
// every later event of the same attempt to none.
func (h *PlaybackHandler) playbackClientInfoWithSessionFallbackV3(sessionID string, info playback.ClientInfo) playback.ClientInfo {
if sessionID == "" || h.sessionMgr == nil {
return info
}
if info.Name != "" && info.Version != "" && info.Build != "" && info.Channel != "" {
return info
}
session, err := h.sessionMgr.GetSession(sessionID)
if err != nil || session == nil {
return info
}
stamped := session.ClientInfo()
if info.Name == "" {
info.Name = stamped.Name
}
if info.Version == "" {
info.Version = stamped.Version
}
if info.Build == "" {
info.Build = stamped.Build
}
if info.Channel == "" {
info.Channel = stamped.Channel
}
return info
}
// newPlaybackStartRequestDigestsV3 fingerprints both the body and normalized
// device identity because either can change the selected playback plan. It
// also retains the pre-device digest while attempts written by an older
@@ -608,7 +532,7 @@ func (d playbackStartRequestDigestsV3) matches(stored string) bool {
return stored == "" || stored == d.current || stored == d.legacy
}
func (h *PlaybackHandler) startPlannedPlaybackV3(r *http.Request, userID int, profileID string, req playback.StartRequestV3, requestDigests playbackStartRequestDigestsV3, requestedFile, effectiveFile *models.MediaFile, audioIndex int, result playback.PlannerResultV3, clientInfo playback.ClientInfo) (playback.DecisionResponseV3, *transportErrorV3) {
func (h *PlaybackHandler) startPlannedPlaybackV3(r *http.Request, userID int, profileID string, req playback.StartRequestV3, requestDigests playbackStartRequestDigestsV3, requestedFile, effectiveFile *models.MediaFile, audioIndex int, result playback.PlannerResultV3) (playback.DecisionResponseV3, *transportErrorV3) {
if result.Plan == nil {
return playback.DecisionResponseV3{}, &transportErrorV3{reason: "internal_error", message: "The server produced no playback plan."}
}
@@ -621,8 +545,15 @@ func (h *PlaybackHandler) startPlannedPlaybackV3(r *http.Request, userID int, pr
return playback.DecisionResponseV3{}, &transportErrorV3{reason: reason, message: "The selected server adaptation is disabled for this user."}
}
}
clientInfo := playbackClientInfoFromRequest(r)
ctx := playback.WithClientInfo(r.Context(), clientInfo)
session, err := h.sessionMgr.StartSessionWithFilesContext(ctx, userID, profileID, effectiveFile.ID, requestedFile.ID, result.PlayMethod, result.TranscodeAudio)
var session *playback.Session
var err error
if starter, ok := h.sessionMgr.(sessionStarterWithFilesContext); ok {
session, err = starter.StartSessionWithFilesContext(ctx, userID, profileID, effectiveFile.ID, requestedFile.ID, result.PlayMethod, result.TranscodeAudio)
} else {
session, err = h.sessionMgr.StartSessionWithFiles(userID, profileID, effectiveFile.ID, requestedFile.ID, result.PlayMethod, result.TranscodeAudio)
}
if err != nil {
return playback.DecisionResponseV3{}, sessionStartErrorV3(err)
}
@@ -721,7 +652,7 @@ func (h *PlaybackHandler) startPlannedPlaybackV3(r *http.Request, userID int, pr
h.maybeQueueLazyPlaybackMarkers(r.Context(), session, effectiveFile)
h.persistSeriesSelectionsV3(r.Context(), userID, profileID, effectiveFile, plannedAudioTrackIndexV3(result, audioIndex))
h.syncSessionsNow(r.Context(), "v3_start")
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, SessionID: session.ID, PlanID: result.Plan.PlanID, Event: playback.RouteEventPlanSelectedV3, AppliedQuirkIDs: appliedQuirkIDsV3(result.Plan), QuirkRegistryRevision: appliedQuirkRevisionV3(result.Plan), OutputContextID: req.ClientPlaybackContext.Output.OutputContextID}, UserID: userID, ProfileID: profileID, ClientName: clientInfo.Name, ClientVersion: clientInfo.Version, ClientBuild: clientInfo.Build, ClientChannel: clientInfo.Channel, ClientModel: req.ClientPlaybackContext.Device.Model})
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, SessionID: session.ID, PlanID: result.Plan.PlanID, Event: playback.RouteEventPlanSelectedV3, AppliedQuirkIDs: appliedQuirkIDsV3(result.Plan), QuirkRegistryRevision: appliedQuirkRevisionV3(result.Plan), OutputContextID: req.ClientPlaybackContext.Output.OutputContextID}, UserID: userID, ProfileID: profileID, ClientName: clientInfo.Name, ClientVersion: clientInfo.Version, ClientModel: req.ClientPlaybackContext.Device.Model})
return response, nil
}
@@ -741,7 +672,7 @@ func (h *PlaybackHandler) prepareTransportV3(r *http.Request, session *playback.
return preparedTransportV3{}, timelineErr
}
if result.Plan.Delivery != playback.DeliveryTranscodeHLSV3 && result.Plan.Delivery != playback.DeliveryRemuxHLSV3 {
return h.prepareIdentityTransportV3(r, session, file, result, timeline)
return h.prepareIdentityTransportV3(session, result, timeline), nil
}
if h.NodePlanner != nil {
plan := h.planNodeSessionV3(r.Context(), session, result)
@@ -846,7 +777,7 @@ func planRequiresServerTransformationsV3(plan *playback.PlanV3) bool {
return false
}
func (h *PlaybackHandler) prepareIdentityTransportV3(r *http.Request, session *playback.Session, file *models.MediaFile, result playback.PlannerResultV3, timeline preparedTimelineV3) (preparedTransportV3, *transportErrorV3) {
func (h *PlaybackHandler) prepareIdentityTransportV3(session *playback.Session, result playback.PlannerResultV3, timeline preparedTimelineV3) preparedTransportV3 {
routeSession := *session
routeSession.PlayMethod = result.PlayMethod
routeSession.BasePlayMethod = result.PlayMethod
@@ -857,31 +788,11 @@ func (h *PlaybackHandler) prepareIdentityTransportV3(r *http.Request, session *p
routeSession.TargetAudioChannels = result.TargetAudioChannels
routeSession.TargetAudioBitrateKbps = result.TargetAudioBitrateKbps
routeSession.RemuxDVMode = remuxDVModeForPlanV3(result.Plan)
proxyNode, proxyErr := h.planIdentityProxyV3(r, session.ID, result)
if proxyErr != nil {
return preparedTransportV3{}, proxyErr
}
streamURL, servedByProxy := h.identityStreamURLV3(&routeSession, file, proxyNode)
releaseProxyReservation := func() {
if releaser, ok := h.NodePlanner.(sessionReservationReleaserV3); ok {
releaser.ReleaseSession(session.ID)
}
}
if proxyNode != nil && !servedByProxy {
// A planned proxy that could not be addressed (no signable token, no
// file record) falls back to the local path, so its reservation must be
// dropped now rather than pinning that node's budget until it ages out.
releaseProxyReservation()
if err := h.refuseLocalIdentityWorkV3(r, result); err != nil {
return preparedTransportV3{}, err
}
}
previousNodeURL := session.TranscodeNodeURL
previousTransportID := remoteTransportID(session)
unlock := h.tm.LockSessionLifecycle(session.ID)
committed := false
streamURL := h.playbackStreamURL(&routeSession)
if result.Plan != nil && result.Plan.Delivery == playback.DeliveryRemuxProgressiveV3 {
if seek := timeline.seekSeconds; seek > 0 {
streamURL = appendPlaybackQueryV3(streamURL, "seek", strconv.FormatFloat(seek, 'f', -1, 64))
@@ -898,7 +809,6 @@ func (h *PlaybackHandler) prepareIdentityTransportV3(r *http.Request, session *p
if previousNodeURL != "" {
h.tm.StopRemoteTranscode(previousTransportID, previousNodeURL)
}
h.applyRemoteTransportMarkV3(r.Context(), session.ID, servedByProxy)
unlock()
},
rollback: func() {
@@ -906,158 +816,9 @@ func (h *PlaybackHandler) prepareIdentityTransportV3(r *http.Request, session *p
return
}
committed = true
// The session never reached the client, so a proxy admitted for it
// must not keep consuming that node's job/bandwidth budget until the
// reservation ages out.
if servedByProxy {
releaseProxyReservation()
}
unlock()
},
}, nil
}
// planIdentityProxyV3 selects the proxy node that will serve a direct-play or
// progressive-remux session. These deliveries need no transcode node — the
// bytes are either the source file or a single remux pipe — so the planner is
// asked for a proxy alone, exactly as the Jellyfin-compat transport does.
//
// A nil node (no planner, no signing secret, or no eligible proxy) means the
// API server serves the stream itself, which is both the single-node case and
// the correct degradation when every proxy is unhealthy or at capacity. The one
// exception is a remux that must run ffmpeg: that is transcode work, so it
// honors the same local-fallback gate as the HLS routes rather than quietly
// spawning an encoder on an API-only node.
func (h *PlaybackHandler) planIdentityProxyV3(r *http.Request, sessionID string, result playback.PlannerResultV3) (*nodepool.Node, *transportErrorV3) {
if h.NodePlanner == nil || h.JWTSecret == "" {
return nil, h.refuseLocalIdentityWorkV3(r, result)
}
// Reserve against the session id the rest of the transport uses, so a
// re-plan replaces its own reservation instead of double-counting, and the
// rollback path can release it.
plan := h.planIdentityProxySessionV3(r.Context(), sessionID, result)
if plan.ProxyNode == nil {
return nil, h.refuseLocalIdentityWorkV3(r, result)
}
return plan.ProxyNode, nil
}
// applyRemoteTransportMarkV3 records whether the committed route's media bytes
// leave this server or another node.
//
// Every committed transition calls it, including the ones that serve locally:
// a session replanned from a proxy onto the integrated transcoder would
// otherwise keep a stale mark, and the widened idle grace it grants would hold
// that session's stream and transcode slots for five minutes after the local
// stream disconnected without an explicit stop.
func (h *PlaybackHandler) applyRemoteTransportMarkV3(ctx context.Context, sessionID string, remote bool) {
if err := h.sessionMgr.SetRemoteTransport(sessionID, remote); err != nil &&
!errors.Is(err, playback.ErrSessionNotFound) {
slog.WarnContext(ctx, "failed to record transport locality",
"component", "api", "playback_session_id", sessionID, "remote", remote, "error", err)
}
}
// planIdentityProxySessionV3 asks the planner for a proxy, narrowing selection
// to proxies that can execute the plan's frozen recipe when one is required.
//
// The narrowing happens before selection rather than after, mirroring how HLS
// filters transcode nodes: rejecting a single round-robin pick would abandon
// the whole pool, so a capable sibling with free capacity would sit unused
// while playback fell back to the API — or was refused outright when local
// fallback is disabled.
//
// Direct play copies bytes and needs no recipe, so it skips the probe entirely
// and any healthy proxy serves it.
func (h *PlaybackHandler) planIdentityProxySessionV3(ctx context.Context, sessionID string, result playback.PlannerResultV3) nodepool.Plan {
estBitrate := identityStreamBitrateKbpsV3(result)
selector, selectable := h.NodePlanner.(capabilitySessionPlannerV3)
enumerator, enumerable := h.NodePlanner.(proxyNodeEnumeratorV3)
if !selectable || !enumerable || !planRequiresServerTransformationsV3(result.Plan) {
return h.NodePlanner.PlanSession(sessionID, "", false, estBitrate)
}
capable := make(map[string]struct{})
for nodeURL, advertised := range h.pooledNodeTransformationsV3(ctx, enumerator.ProxyNodeURLs()) {
if validateAdvertisedTransformationsV3(result.Plan, advertised) == nil {
capable[nodeURL] = struct{}{}
}
}
if len(capable) == 0 {
slog.WarnContext(ctx, "protocol v3 no proxy advertises the planned recipe",
"component", "api", "delivery", result.Plan.Delivery)
}
// The predicate runs under the planner lock: a set lookup only.
return selector.PlanSessionWith(sessionID, "", false, estBitrate, func(node *nodepool.Node) bool {
if node == nil {
return false
}
_, ok := capable[node.URL]
return ok
})
}
// refuseLocalIdentityWorkV3 enforces playback.local_transcode_fallback for the
// identity deliveries. Direct play moves bytes and always falls back locally;
// a progressive remux that must convert audio is ffmpeg work, so on an API-only
// node it is refused for the same reason the HLS routes refuse it, instead of
// silently spawning an encoder the operator disabled.
func (h *PlaybackHandler) refuseLocalIdentityWorkV3(r *http.Request, result playback.PlannerResultV3) *transportErrorV3 {
if result.Plan == nil || result.Plan.Delivery != playback.DeliveryRemuxProgressiveV3 ||
!planRequiresServerTransformationsV3(result.Plan) ||
nodepool.LocalTranscodeFallbackAllowed(r.Context(), h.SettingsRepo) {
return nil
}
return &transportErrorV3{reason: "capacity_unavailable", message: "No proxy node is available and local fallback is disabled.", retryable: true}
}
// identityStreamBitrateKbpsV3 estimates the bitrate a proxy will egress for an
// identity delivery, so bandwidth-capped proxies admit it accurately. The plan's
// effective recipe is authoritative (a remux that downmixes audio egresses less
// than the source); the source descriptor is the fallback.
func identityStreamBitrateKbpsV3(result playback.PlannerResultV3) int {
if result.Plan == nil {
return 0
}
if effective := result.Plan.EffectiveRecipe.BitrateKbps; effective != nil && *effective > 0 {
return *effective
}
if source := result.Plan.Source.BitrateKbps; source > 0 {
return source
}
return 0
}
// identityStreamURLV3 builds the stream URL for a direct-play or
// progressive-remux session: an absolute proxy URL when one was planned,
// otherwise the API-local path.
//
// The proxy serves from the token alone, so the token has to carry everything
// the API-local path would have read from the session and the file record: the
// media path it opens, and the Dolby Vision profile its remux needs to strip a
// dangling Profile 7 RPU. Omitting either would not fail loudly — the proxy
// would serve a subtly different stream than the plan promised.
//
// The bool reports whether the returned URL is actually a proxy URL, so the
// caller can release the planner reservation when it is not.
func (h *PlaybackHandler) identityStreamURLV3(s *playback.Session, file *models.MediaFile, proxyNode *nodepool.Node) (string, bool) {
if proxyNode == nil || file == nil {
return h.playbackStreamURL(s), false
}
card := identityRecipeCard(s)
card.InputPath = file.FilePath
claims := card.ToClaims()
claims.DVProfile = file.PrimaryDVProfile()
claims.AudioOnly = file.IsAudioOnly()
token := h.signStreamClaims(claims)
if token == "" {
return h.playbackStreamURL(s), false
}
base := strings.TrimRight(proxyNode.URL, "/")
if s.PlayMethod == playback.PlayRemux {
return base + "/stream/remux/" + token, true
}
return base + "/stream/direct/" + token, true
}
// sessionOwnsResumeTimelineV3 reports whether the session's own position is a
@@ -1268,7 +1029,6 @@ func (h *PlaybackHandler) prepareLocalTransportV3(r *http.Request, session *play
}
committed = true
previous := h.tm.SwapTranscodeSession(session.ID, ts)
h.applyRemoteTransportMarkV3(r.Context(), session.ID, false)
unlock()
if previous != nil && previous != ts {
_ = previous.Close()
@@ -1327,10 +1087,6 @@ func (h *PlaybackHandler) prepareRemoteTransportV3(r *http.Request, session *pla
hw := firstNonEmptyHandlerV3(strings.TrimSpace(nodeResp.HWAccel), strings.TrimSpace(req.HWAccel))
card := playback.NewRecipeCard(session.UserID, session.ProfileID, file.ID, node.URL, playback.TranscodeOpts{InputPath: req.InputPath, SessionID: session.ID, TranscodeTransportID: transportID, SourceVideoCodec: req.SourceVideoCodec, SourceVideoProfile: req.SourceVideoProfile, SourceVideoBitDepth: req.SourceVideoBitDepth, SoftwareVideoDecode: req.SoftwareVideoDecode, VideoBitstreamFilter: req.VideoBitstreamFilter, SeekSeconds: req.SeekSeconds, StreamOriginSeconds: req.StreamOriginSeconds, CopySeekAnchorResolved: req.CopySeekAnchorResolved, StartSegmentNumber: req.StartSegmentNumber, TargetResolution: req.TargetResolution, TargetCodecVideo: req.TargetCodecVideo, TargetCodecAudio: req.TargetCodecAudio, TargetAudioChannels: req.TargetAudioChannels, TargetAudioBitrateKbps: req.TargetAudioBitrateKbps, TargetBitrateKbps: req.TargetBitrateKbps, SegmentDuration: req.SegmentDuration, HWAccel: hw, AudioTrackIndex: req.AudioTrackIndex, SubtitleTrackIndex: req.SubtitleTrackIndex, SubtitleBurnIn: req.SubtitleBurnIn, SubtitleCodec: req.SubtitleCodec, TotalDuration: req.TotalDuration})
url := h.buildProxyManifestURL(card, nodePlan.ProxyNode)
// buildProxyManifestURL only returns an absolute proxy URL when a proxy was
// planned and the token could be signed; otherwise the client fetches the
// manifest from this server and the local liveness path applies.
servedByProxy := nodePlan.ProxyNode != nil && strings.HasPrefix(url, "http")
committed := false
previousNodeURL := session.TranscodeNodeURL
previousTransportID := remoteTransportID(session)
@@ -1344,7 +1100,6 @@ func (h *PlaybackHandler) prepareRemoteTransportV3(r *http.Request, session *pla
if previousNodeURL != "" {
h.tm.StopRemoteTranscode(previousTransportID, previousNodeURL)
}
h.applyRemoteTransportMarkV3(r.Context(), session.ID, servedByProxy)
unlock()
}, rollback: func() {
if committed {
@@ -2159,7 +1914,7 @@ func (h *PlaybackHandler) executeReplanV3(r *http.Request, record *playback.Atte
event = playback.RouteEventRuntimeCorrectionSucceededV3
clientModel = start.ClientPlaybackContext.Device.Model
}
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, SessionID: session.ID, PlanID: result.Plan.PlanID, PlanAttemptID: req.PlanAttemptID, PlanAttemptKey: playback.PlanAttemptKeyV3(*result.Plan, start.ClientPlaybackContext.Output.OutputContextID, nil), Event: event, FallbackReason: req.Failure.Classification, AppliedQuirkIDs: appliedQuirkIDsV3(result.Plan), QuirkRegistryRevision: appliedQuirkRevisionV3(result.Plan), OutputContextID: start.ClientPlaybackContext.Output.OutputContextID}, UserID: session.UserID, ProfileID: session.ProfileID, ClientName: session.ClientName, ClientVersion: session.ClientVersion, ClientBuild: session.ClientBuild, ClientChannel: session.ClientChannel, ClientModel: clientModel})
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: playback.RouteEventV3{ProtocolVersion: playback.ProtocolV3, PlaybackAttemptID: req.PlaybackAttemptID, SessionID: session.ID, PlanID: result.Plan.PlanID, PlanAttemptID: req.PlanAttemptID, PlanAttemptKey: playback.PlanAttemptKeyV3(*result.Plan, start.ClientPlaybackContext.Output.OutputContextID, nil), Event: event, FallbackReason: req.Failure.Classification, AppliedQuirkIDs: appliedQuirkIDsV3(result.Plan), QuirkRegistryRevision: appliedQuirkRevisionV3(result.Plan), OutputContextID: start.ClientPlaybackContext.Output.OutputContextID}, UserID: session.UserID, ProfileID: session.ProfileID, ClientName: session.ClientName, ClientVersion: session.ClientVersion, ClientModel: clientModel})
}
transport.rollback = func() {
originalRollback()
@@ -2605,32 +2360,15 @@ func shouldTryAlternateFileV3(qualityPreference string) bool {
}
const (
terminalNoAlternateVersionV3 = "no_alternate_version"
terminalHDRTranscodeUnsupportedV3 = "hdr_transcode_unsupported"
terminalSubtitleConversionUnsupportedV3 = "subtitle_conversion_unsupported"
terminalNoAlternateVersionV3 = "no_alternate_version"
terminalHDRTranscodeUnsupportedV3 = "hdr_transcode_unsupported"
)
// terminalAllowsAlternateFileV3 reports whether a refusal is the kind another
// version of the same item could satisfy.
//
// subtitle_conversion_unsupported belongs here because it is not only a
// subtitle-format refusal: when a burn-in requirement is the sole trigger of an
// adaptation the source cannot take, the planner reports the blocker in terms
// of the subtitle rather than the HDR pipeline or the 4K policy. Those cases
// used to surface as hdr_transcode_unsupported / no_alternate_version and were
// the exact reason this gate exists — a bitmap subtitle needing burn-in that an
// HDR source cannot support while an SDR alternate can. Leaving the new reason
// out silently retired that fallback and refused playback outright.
func terminalAllowsAlternateFileV3(terminal *playback.TerminalV3) bool {
if terminal == nil {
return false
}
switch terminal.Reason {
case terminalNoAlternateVersionV3, terminalHDRTranscodeUnsupportedV3, terminalSubtitleConversionUnsupportedV3:
return true
default:
return false
}
return terminal.Reason == terminalNoAlternateVersionV3 || terminal.Reason == terminalHDRTranscodeUnsupportedV3
}
func replanAllowsAlternateFileV3(operation playback.ReplanOperationV3, qualityPreference string) bool {
@@ -2770,8 +2508,8 @@ func (h *PlaybackHandler) HandlePlaybackRouteEventV3(w http.ResponseWriter, r *h
return
}
event.Diagnostics = sanitizeDiagnosticsV3(event.Diagnostics)
client := h.playbackClientInfoWithSessionFallbackV3(firstNonEmptyValue(event.SessionID, identity.SessionID), playbackClientInfoFromRequest(r))
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: event, UserID: userID, ProfileID: profileID, ClientName: client.Name, ClientVersion: client.Version, ClientBuild: client.Build, ClientChannel: client.Channel, ClientModel: event.Diagnostics["device_model"]})
client := playbackClientInfoFromRequest(r)
h.enqueueRouteEventV3(playback.RouteEventRecordV3{RouteEventV3: event, UserID: userID, ProfileID: profileID, ClientName: client.Name, ClientVersion: client.Version, ClientModel: event.Diagnostics["device_model"]})
w.WriteHeader(http.StatusAccepted)
}
-516
View File
@@ -23,7 +23,6 @@ import (
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/nodepool"
"github.com/Silo-Server/silo-server/internal/playback"
"github.com/Silo-Server/silo-server/internal/streamtoken"
"github.com/Silo-Server/silo-server/internal/subtitles"
"github.com/Silo-Server/silo-server/internal/transcodenode"
"github.com/Silo-Server/silo-server/internal/userstore"
@@ -3635,518 +3634,3 @@ func postPlaybackReplanV3(t *testing.T, handler *PlaybackHandler, sessionID stri
}
return response
}
// recordingNodePlannerV3 is a planner double that records what the transport
// asked for and whether the reservation was released.
type recordingNodePlannerV3 struct {
plan nodepool.Plan
needsTranscode bool
estBitrateKbps int
plannedSessionID string
released []string
}
func (p *recordingNodePlannerV3) PlanSession(sessionID, _ string, needsTranscode bool, estBitrateKbps int) nodepool.Plan {
p.plannedSessionID = sessionID
p.needsTranscode = needsTranscode
p.estBitrateKbps = estBitrateKbps
return p.plan
}
func (p *recordingNodePlannerV3) ReleaseSession(sessionID string) {
p.released = append(p.released, sessionID)
}
// PlanSessionWith mirrors the real planner: the eligibility predicate narrows
// the pool before selection, so a proxy that cannot execute the recipe is
// skipped rather than picked and then rejected.
func (p *recordingNodePlannerV3) PlanSessionWith(sessionID, currentTranscodeURL string, needsTranscode bool, estBitrateKbps int, eligible func(*nodepool.Node) bool) nodepool.Plan {
plan := p.PlanSession(sessionID, currentTranscodeURL, needsTranscode, estBitrateKbps)
if eligible == nil {
return plan
}
if !needsTranscode && plan.ProxyNode != nil && !eligible(plan.ProxyNode) {
return nodepool.Plan{}
}
if needsTranscode && plan.TranscodeNode != nil && !eligible(plan.TranscodeNode) {
return nodepool.Plan{}
}
return plan
}
func (p *recordingNodePlannerV3) ProxyNodeURLs() []string {
if p.plan.ProxyNode == nil {
return nil
}
return []string{p.plan.ProxyNode.URL}
}
func (p *recordingNodePlannerV3) TranscodeNodeURLs() []string {
if p.plan.TranscodeNode == nil {
return nil
}
return []string{p.plan.TranscodeNode.URL}
}
func identityProxyPlanV3(delivery playback.DeliveryV3, transformations ...playback.TransformationV3) *playback.PlanV3 {
return &playback.PlanV3{
PlanID: "plan:identity-proxy",
Delivery: delivery,
Transformations: transformations,
Source: playback.SourceDescriptorV3{BitrateKbps: 5_673},
}
}
func TestPrepareTransportV3RoutesDirectPlayThroughProxyNode(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
planner := &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: "http://proxy-1"}}}
handler.NodePlanner = planner
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-direct-proxy", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
defer transport.rollback()
if !strings.HasPrefix(transport.url, "http://proxy-1/stream/direct/") {
t.Fatalf("direct-play stream url = %q, want a proxy /stream/direct URL", transport.url)
}
if planner.needsTranscode {
t.Fatal("direct play must not request a transcode node")
}
if planner.plannedSessionID != "session-direct-proxy" {
t.Fatalf("reservation session id = %q, want the playback session id", planner.plannedSessionID)
}
if planner.estBitrateKbps != 5_673 {
t.Fatalf("estimated bitrate = %d, want the source bitrate for cap admission", planner.estBitrateKbps)
}
token := strings.TrimPrefix(transport.url, "http://proxy-1/stream/direct/")
claims, err := streamtoken.Verify(token, handler.JWTSecret)
if err != nil {
t.Fatalf("verify proxy stream token: %v", err)
}
if claims.MediaPath == "" {
t.Fatal("proxy token carries no media path; the proxy cannot open the source")
}
if claims.PlayMethod != string(playback.PlayDirect) {
t.Fatalf("token play method = %q, want direct", claims.PlayMethod)
}
}
func TestPrepareTransportV3RoutesProgressiveRemuxThroughProxyNodeWithSeekAndDV(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
stubCopySeekAnchorV3(handler)
proxy := capableProxyStubV3(t)
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: proxy.URL + "/"}}}
file := v3HandlerFixtureFile(t)
file.VideoTracks[0].DVProfile = 7
plan := identityProxyPlanV3(playback.DeliveryRemuxProgressiveV3, playback.TransformationV3{Name: playback.TransformationAudioToAACV3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"})
plan.Timeline = playback.TimelineV3{SourceStartSeconds: 39.5}
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-remux-proxy", UserID: 7, ProfileID: "profile-1"},
file,
playback.PlannerResultV3{Plan: plan, PlayMethod: playback.PlayRemux, TranscodeAudio: true, TargetAudioCodec: "aac"},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
defer transport.rollback()
remuxPrefix := proxy.URL + "/stream/remux/"
if !strings.HasPrefix(transport.url, remuxPrefix) {
t.Fatalf("remux stream url = %q, want a proxy /stream/remux URL", transport.url)
}
if !strings.Contains(transport.url, "seek=39.5") {
t.Fatalf("remux stream url = %q, want the resolved seek preserved", transport.url)
}
token := strings.TrimSuffix(strings.TrimPrefix(transport.url, remuxPrefix), "?seek=39.5")
claims, err := streamtoken.Verify(token, handler.JWTSecret)
if err != nil {
t.Fatalf("verify proxy stream token: %v", err)
}
if claims.DVProfile != 7 {
t.Fatalf("token DV profile = %d, want 7 so the proxy strips the dangling RPU", claims.DVProfile)
}
if !claims.TranscodeAudio {
t.Fatal("token must tell the proxy to convert audio")
}
}
func TestPrepareTransportV3FallsBackLocallyWithoutEligibleProxy(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{}}
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-direct-local", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
defer transport.rollback()
if !strings.HasPrefix(transport.url, "/stream/session-direct-local") {
t.Fatalf("stream url = %q, want the API-local path when no proxy is eligible", transport.url)
}
}
func TestPrepareTransportV3RefusesLocalRemuxWhenFallbackDisabled(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
stubCopySeekAnchorV3(handler)
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{}}
handler.SettingsRepo = &mutablePlaybackSettingsV3{values: map[string]string{"playback.local_transcode_fallback": "false"}}
_, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-remux-refused", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{
Plan: identityProxyPlanV3(playback.DeliveryRemuxProgressiveV3, playback.TransformationV3{Name: playback.TransformationAudioToAACV3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"}),
PlayMethod: playback.PlayRemux,
TranscodeAudio: true,
},
)
if transportErr == nil || transportErr.reason != "capacity_unavailable" {
t.Fatalf("transport error = %#v, want capacity_unavailable when local remux work is disabled", transportErr)
}
}
func TestPrepareTransportV3AllowsLocalDirectPlayWhenFallbackDisabled(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{}}
handler.SettingsRepo = &mutablePlaybackSettingsV3{values: map[string]string{"playback.local_transcode_fallback": "false"}}
// Direct play moves bytes rather than running ffmpeg, so the transcode
// fallback gate must not strand a single-node deployment.
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-direct-allowed", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("direct play refused with local fallback disabled: %#v", transportErr)
}
transport.rollback()
}
func TestPrepareTransportV3ReleasesProxyReservationOnRollback(t *testing.T) {
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
planner := &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: "http://proxy-1"}}}
handler.NodePlanner = planner
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-rollback", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
transport.rollback()
if len(planner.released) != 1 || planner.released[0] != "session-rollback" {
t.Fatalf("released reservations = %v, want the session released once", planner.released)
}
}
// capableProxyStubV3 is a proxy that advertises the audio_to_aac and DV strip
// recipes, mirroring the /hw-capabilities endpoint a real proxy now serves.
func capableProxyStubV3(t *testing.T) *httptest.Server {
t.Helper()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/hw-capabilities" {
w.WriteHeader(http.StatusNotFound)
return
}
writeJSON(w, http.StatusOK, playback.HWAccelInfo{Transformations: []playback.TransformationV3{
{Name: playback.TransformationAudioToAACV3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"},
{Name: playback.TransformationServerDV7HDR10V3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"},
}})
}))
t.Cleanup(server.Close)
return server
}
func TestPrepareTransportV3KeepsRemuxLocalWhenProxyLacksTheRecipe(t *testing.T) {
// A proxy on an older ffmpeg build advertises no aac encoder. Sending the
// remux there would 500 at stream time, so selection must reject it.
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, playback.HWAccelInfo{Transformations: []playback.TransformationV3{
{Name: playback.TransformationVideoToH264V3, Executor: playback.ExecutorServerV3, RecipeVersion: "2"},
}})
}))
defer proxy.Close()
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
stubCopySeekAnchorV3(handler)
planner := &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: proxy.URL}}}
handler.NodePlanner = planner
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-incapable-proxy", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{
Plan: identityProxyPlanV3(playback.DeliveryRemuxProgressiveV3, playback.TransformationV3{Name: playback.TransformationAudioToAACV3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"}),
PlayMethod: playback.PlayRemux,
TranscodeAudio: true,
},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
defer transport.rollback()
if strings.HasPrefix(transport.url, proxy.URL) {
t.Fatalf("stream url = %q, want local fallback when the proxy lacks the recipe", transport.url)
}
// Narrowing happens before selection, so no reservation is made against an
// incapable proxy in the first place and none needs releasing.
if len(planner.released) != 0 {
t.Fatalf("released = %v, want no reservation taken on an incapable proxy", planner.released)
}
}
func TestPrepareTransportV3DirectPlaySkipsProxyCapabilityProbe(t *testing.T) {
// Direct play copies bytes and needs no recipe, so it must not pay a
// capability fetch nor be rejected by one.
probes := 0
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
probes++
w.WriteHeader(http.StatusNotFound)
}))
defer proxy.Close()
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: proxy.URL}}}
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-direct-noprobe", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
defer transport.rollback()
if !strings.HasPrefix(transport.url, proxy.URL+"/stream/direct/") {
t.Fatalf("stream url = %q, want the proxy direct URL", transport.url)
}
if probes != 0 {
t.Fatalf("capability probes = %d, want none for direct play", probes)
}
}
func TestIdentityStreamBitrateKbpsV3PrefersEffectiveRecipe(t *testing.T) {
// A downmixing remux egresses less than the source, so the effective
// recipe — not the source descriptor — is what a capped proxy admits on.
effective := 1_200
plan := identityProxyPlanV3(playback.DeliveryRemuxProgressiveV3)
plan.EffectiveRecipe.BitrateKbps = &effective
if got := identityStreamBitrateKbpsV3(playback.PlannerResultV3{Plan: plan}); got != effective {
t.Fatalf("bitrate estimate = %d, want the effective recipe value %d", got, effective)
}
plan.EffectiveRecipe.BitrateKbps = nil
if got := identityStreamBitrateKbpsV3(playback.PlannerResultV3{Plan: plan}); got != 5_673 {
t.Fatalf("bitrate estimate = %d, want the source fallback 5673", got)
}
}
func TestPrepareTransportV3MarksProxySessionsAsRemotelyTransported(t *testing.T) {
manager := playback.NewSessionManager(0, 0)
handler := NewPlaybackHandler(manager)
handler.JWTSecret = "test-secret"
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: "http://proxy-1"}}}
session, err := manager.StartSession(7, "profile-1", 42, playback.PlayDirect, false)
if err != nil {
t.Fatalf("start session: %v", err)
}
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
session,
v3HandlerFixtureFile(t),
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %v", transportErr)
}
transport.commit()
// The client talks to the proxy, so this server sees no transport request.
// Without the remote-transport mark the idle sweep would reap a healthy
// stream as soon as heartbeats lapsed.
if expired := manager.CleanInactive(time.Nanosecond, time.Nanosecond); len(expired) != 0 {
t.Fatalf("expired %d proxy-served sessions, want them preserved", len(expired))
}
if _, err := manager.GetSession(session.ID); err != nil {
t.Fatalf("proxy-served session was reaped: %v", err)
}
}
// poolNodePlannerV3 models a multi-proxy pool so capability narrowing can be
// observed end to end rather than only at the planner unit level.
type poolNodePlannerV3 struct {
proxies []*nodepool.Node
released []string
}
func (p *poolNodePlannerV3) PlanSession(string, string, bool, int) nodepool.Plan {
if len(p.proxies) == 0 {
return nodepool.Plan{}
}
return nodepool.Plan{ProxyNode: p.proxies[0]}
}
func (p *poolNodePlannerV3) PlanSessionWith(sessionID, currentTranscodeURL string, needsTranscode bool, estBitrateKbps int, eligible func(*nodepool.Node) bool) nodepool.Plan {
for _, node := range p.proxies {
if eligible == nil || eligible(node) {
return nodepool.Plan{ProxyNode: node}
}
}
return nodepool.Plan{}
}
func (p *poolNodePlannerV3) ProxyNodeURLs() []string {
urls := make([]string, 0, len(p.proxies))
for _, node := range p.proxies {
urls = append(urls, node.URL)
}
return urls
}
func (p *poolNodePlannerV3) TranscodeNodeURLs() []string { return nil }
func (p *poolNodePlannerV3) ReleaseSession(sessionID string) {
p.released = append(p.released, sessionID)
}
func TestPrepareTransportV3PrefersACapableSiblingProxy(t *testing.T) {
// A rolling upgrade leaves one proxy on an ffmpeg without an aac encoder.
// The remux must land on its capable sibling rather than falling back to
// the API node, which would be refused outright with local fallback off.
stale := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, playback.HWAccelInfo{Transformations: []playback.TransformationV3{
{Name: playback.TransformationVideoToH264V3, Executor: playback.ExecutorServerV3, RecipeVersion: "2"},
}})
}))
defer stale.Close()
upgraded := capableProxyStubV3(t)
handler := NewPlaybackHandler(playback.NewSessionManager(0, 0))
handler.JWTSecret = "test-secret"
stubCopySeekAnchorV3(handler)
handler.SettingsRepo = &mutablePlaybackSettingsV3{values: map[string]string{"playback.local_transcode_fallback": "false"}}
handler.NodePlanner = &poolNodePlannerV3{proxies: []*nodepool.Node{
{ID: 1, URL: stale.URL},
{ID: 2, URL: upgraded.URL},
}}
transport, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil),
&playback.Session{ID: "session-sibling", UserID: 7, ProfileID: "profile-1"},
v3HandlerFixtureFile(t),
playback.PlannerResultV3{
Plan: identityProxyPlanV3(playback.DeliveryRemuxProgressiveV3, playback.TransformationV3{Name: playback.TransformationAudioToAACV3, Executor: playback.ExecutorServerV3, RecipeVersion: "1"}),
PlayMethod: playback.PlayRemux,
TranscodeAudio: true,
},
)
if transportErr != nil {
t.Fatalf("prepare identity transport: %#v", transportErr)
}
defer transport.rollback()
if !strings.HasPrefix(transport.url, upgraded.URL+"/stream/remux/") {
t.Fatalf("stream url = %q, want the capable sibling proxy %q", transport.url, upgraded.URL)
}
}
func TestPrepareTransportV3ClearsRemoteTransportMarkWhenServingLocally(t *testing.T) {
// A session replanned from a proxy onto this server must lose the widened
// idle grace, or its stream and transcode slots stay held for minutes after
// a local stream disconnects without an explicit stop.
manager := playback.NewSessionManager(0, 0)
handler := NewPlaybackHandler(manager)
handler.JWTSecret = "test-secret"
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{ProxyNode: &nodepool.Node{URL: "http://proxy-1"}}}
session, err := manager.StartSession(7, "profile-1", 42, playback.PlayDirect, false)
if err != nil {
t.Fatalf("start session: %v", err)
}
file := v3HandlerFixtureFile(t)
proxied, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil), session, file,
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare proxy transport: %v", transportErr)
}
proxied.commit()
// Re-plan with no eligible proxy: the same delivery now serves locally.
handler.NodePlanner = &recordingNodePlannerV3{plan: nodepool.Plan{}}
local, transportErr := handler.prepareTransportV3(
httptest.NewRequest(http.MethodPost, "/", nil), session, file,
playback.PlannerResultV3{Plan: identityProxyPlanV3(playback.DeliveryOriginalHTTPV3), PlayMethod: playback.PlayDirect},
)
if transportErr != nil {
t.Fatalf("prepare local transport: %v", transportErr)
}
local.commit()
if expired := manager.CleanInactive(time.Nanosecond, time.Nanosecond); len(expired) != 1 {
t.Fatalf("expired %d sessions, want the locally-served session to lose its widened grace", len(expired))
}
}
// A burn-in requirement that only an SDR alternate can satisfy must still reach
// the alternate-version fallback. The planner reports that refusal in terms of
// the subtitle rather than the HDR pipeline, so a gate listing only the HDR and
// version reasons silently retires the fallback and refuses playback outright.
func TestTerminalAllowsAlternateFileV3CoversSubtitleForcedRefusals(t *testing.T) {
for _, reason := range []string{
terminalNoAlternateVersionV3,
terminalHDRTranscodeUnsupportedV3,
terminalSubtitleConversionUnsupportedV3,
} {
if !terminalAllowsAlternateFileV3(&playback.TerminalV3{Reason: reason}) {
t.Fatalf("terminal %q must allow an alternate-version retry", reason)
}
}
if terminalAllowsAlternateFileV3(&playback.TerminalV3{Reason: "client_hls_unsupported"}) {
t.Fatal("a client-route refusal must not trigger an alternate-version retry")
}
if terminalAllowsAlternateFileV3(nil) {
t.Fatal("a nil terminal must not trigger an alternate-version retry")
}
}
+12 -15
View File
@@ -23,19 +23,18 @@ import (
// SectionHandler handles section management and batch section endpoints.
type SectionHandler struct {
repo *sections.Repository
fetcher *sections.Fetcher
previewFetcher sectionPreviewFetcher // set to fetcher at construction; separate for test injection
episodeFetcher sectionEpisodeFetcher
FolderRepo *catalog.FolderRepository
EpisodeRepo *catalog.EpisodeRepository
StoreProvider userstore.UserStoreProvider
UserRepo *auth.UserRepository
DetailSvc *catalog.DetailService
Settings catalog.SettingsStore
CollectionRepo *catalog.LibraryCollectionRepository
SortPreferenceCleaner *userstore.CollectionSortPreferenceCleaner
EbookProgress EbookReaderProgressLister
repo *sections.Repository
fetcher *sections.Fetcher
previewFetcher sectionPreviewFetcher // set to fetcher at construction; separate for test injection
episodeFetcher sectionEpisodeFetcher
FolderRepo *catalog.FolderRepository
EpisodeRepo *catalog.EpisodeRepository
StoreProvider userstore.UserStoreProvider
UserRepo *auth.UserRepository
DetailSvc *catalog.DetailService
Settings catalog.SettingsStore
CollectionRepo *catalog.LibraryCollectionRepository
EbookProgress EbookReaderProgressLister
}
// NewSectionHandler creates a new SectionHandler.
@@ -406,8 +405,6 @@ func (h *SectionHandler) deleteUnreferencedSectionManagedCollection(ctx context.
}
if err := h.CollectionRepo.Delete(ctx, collectionID); err != nil && !errors.Is(err, catalog.ErrLibraryCollectionNotFound) {
slog.WarnContext(ctx, "failed to delete unreferenced section-managed collection", "component", "api", "collection_id", collectionID, "error", err)
} else if err == nil && h.SortPreferenceCleaner != nil {
h.SortPreferenceCleaner.DeleteForCollection(ctx, userstore.CollectionKindLibrary, collectionID)
}
}
@@ -75,7 +75,6 @@ type userImportSharedFields struct {
PosterURL string `json:"poster_url"`
LibraryIDs []int `json:"library_ids,omitempty"`
DisplayQueryDefinition json.RawMessage `json:"display_query_definition,omitempty"`
SortConfig json.RawMessage `json:"sort_config,omitempty"`
}
type userImportMDBListRequest struct {
@@ -218,11 +217,6 @@ func (h *UserCollectionImportHandler) createImportedCollection(
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
sortConfig, err := NormalizeCollectionSortConfig(shared.SortConfig, true)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
collection, err := store.CreateCollection(r.Context(), userstore.CreateCollectionInput{
CreatorProfileID: profileID,
@@ -231,7 +225,7 @@ func (h *UserCollectionImportHandler) createImportedCollection(
CollectionType: collectionType,
IsShared: shared.IsShared,
QueryDefinition: "{}",
SortConfig: sortConfig,
SortConfig: "{}",
SourceURL: cfg.DisplayURL(),
SourceConfig: sourceConfigJSON,
SyncSchedule: schedule,
-11
View File
@@ -1438,14 +1438,6 @@ func NewRouter(deps Dependencies) chi.Router {
}
libraryCollectionRepo := catalog.NewLibraryCollectionRepository(deps.DB)
var collectionSortCleaner *userstore.CollectionSortPreferenceCleaner
if userRepo != nil && deps.UserStoreProvider != nil {
collectionSortCleaner = userstore.NewCollectionSortPreferenceCleaner(userRepo, deps.UserStoreProvider)
}
if collectionHandler != nil {
collectionHandler.LibraryCollections = libraryCollectionRepo
}
sectionHandler.SortPreferenceCleaner = collectionSortCleaner
if libraryCollectionService == nil {
libraryCollectionService = catalog.NewLibraryCollectionService(
libraryCollectionRepo,
@@ -1545,7 +1537,6 @@ func NewRouter(deps Dependencies) chi.Router {
libraryCollectionHandler.SectionRepo = sectionRepo
libraryCollectionHandler.UserCollectionPool = deps.DB
libraryCollectionHandler.EventsHub = deps.EventsHub
libraryCollectionHandler.SortPreferenceCleaner = collectionSortCleaner
if deps.FolderRepo != nil {
libraryCollectionHandler.FolderRepo = deps.FolderRepo
} else {
@@ -2300,8 +2291,6 @@ func NewRouter(deps Dependencies) chi.Router {
r.Use(apimw.RequireProfile)
r.Get("/", collectionHandler.HandleListCollections)
r.Get("/capabilities", collectionHandler.HandleCapabilities)
r.Put("/sort-preference", collectionHandler.HandleSetCollectionSortPreference)
r.Delete("/sort-preference", collectionHandler.HandleClearCollectionSortPreference)
if libraryCollectionHandler != nil {
// Aggregated server (admin-curated) collections across
// every accessible library. Separate from "/" (personal,
-25
View File
@@ -8,8 +8,6 @@ import (
"github.com/Silo-Server/silo-server/internal/models"
)
const LibraryCollectionVisibilityVisible = "visible"
// AccessFilter captures effective viewer access constraints for catalog reads.
type AccessFilter struct {
AllowedLibraryIDs []int
@@ -47,29 +45,6 @@ type AccessFilter struct {
ExcludedMediaTypes []string
}
// CanAccessLibraryCollection reports whether a visible server collection is
// reachable through at least one library in the viewer's effective scope.
// Collections without explicit library scope retain their legacy unrestricted
// visibility, but restricted viewers cannot address them by ID.
func CanAccessLibraryCollection(collection *models.LibraryCollection, filter AccessFilter) bool {
if collection == nil || collection.Visibility != LibraryCollectionVisibilityVisible {
return false
}
if len(collection.LibraryIDs) == 0 {
return filter.AllowedLibraryIDs == nil && len(filter.DisabledLibraryIDs) == 0
}
for _, libraryID := range collection.LibraryIDs {
if filter.AllowedLibraryIDs != nil && !intInSlice(libraryID, filter.AllowedLibraryIDs) {
continue
}
if intInSlice(libraryID, filter.DisabledLibraryIDs) {
continue
}
return true
}
return false
}
func applyAccessFilter(alias string, filter AccessFilter, conditions *[]string, args *[]any, argIdx *int) {
if filter.MaxContentRating != "" {
allowedRatings := access.AllowedRatingsUpTo(filter.MaxContentRating)
+5 -74
View File
@@ -37,12 +37,6 @@ type CatalogResult struct {
SemanticUsed bool
FallbackReason string
IndexPendingEvents int
// EffectiveSort is the order collection sources actually resolved in, after
// applying the request's own sort, the viewer's saved override, and the
// collection's configured default in that order. An empty Field means the
// collection's source order. Only collection sources populate it; clients
// use it to show which sort is active when the request carried none.
EffectiveSort QuerySort
}
type CatalogFiltersResult struct {
@@ -493,25 +487,10 @@ func (r *CatalogResolver) resolveSectionBrowseSource(ctx context.Context, req Ca
func (r *CatalogResolver) resolveLibraryCollectionSource(ctx context.Context, req CatalogRequest, access AccessFilter) (*CatalogResult, error) {
collectionRepo := NewLibraryCollectionRepository(r.itemRepo.pool)
collection, err := collectionRepo.GetByID(ctx, req.CollectionID)
if err != nil || !CanAccessLibraryCollection(collection, access) {
if err != nil || collection.Visibility != "visible" {
return nil, ErrCatalogSourceNotFound
}
return r.resolveCollectionWithEffectiveSort(
ctx, req, access, userstore.CollectionKindLibrary, collection.ID, collection.SortConfig,
func(req CatalogRequest) (*CatalogResult, error) {
return r.resolveLibraryCollectionItems(ctx, req, access, collection, collectionRepo)
},
)
}
func (r *CatalogResolver) resolveLibraryCollectionItems(
ctx context.Context,
req CatalogRequest,
access AccessFilter,
collection *models.LibraryCollection,
collectionRepo *LibraryCollectionRepository,
) (*CatalogResult, error) {
if IsLiveQueryType(collection.CollectionType) {
return r.resolveLiveLibraryCollectionSource(ctx, req, access, collection)
}
@@ -565,55 +544,10 @@ func (r *CatalogResolver) resolveUserCollectionSource(ctx context.Context, req C
}
collection, err := store.GetCollection(ctx, req.CollectionID)
if err != nil || !ProfileCanAccessCollection(collection, access.ProfileID) {
if err != nil || !catalogProfileCanAccessCollection(collection, access.ProfileID) {
return nil, ErrCatalogSourceNotFound
}
return r.resolveCollectionWithEffectiveSort(
ctx, req, access, userstore.CollectionKindUser, collection.ID, []byte(collection.SortConfig),
func(req CatalogRequest) (*CatalogResult, error) {
return r.resolveUserCollectionItems(ctx, req, access, store, collection)
},
)
}
// resolveCollectionWithEffectiveSort applies the shared collection-sort
// precedence, delegates item resolution, and stamps the resolved order on the
// response. Keeping this sequence in one place prevents the library and
// personal collection paths from drifting.
func (r *CatalogResolver) resolveCollectionWithEffectiveSort(
ctx context.Context,
req CatalogRequest,
access AccessFilter,
kind, collectionID string,
sortConfig []byte,
resolve func(CatalogRequest) (*CatalogResult, error),
) (*CatalogResult, error) {
// A sort in the request is the viewer's live choice and always wins; only
// when there is none do the saved override and the collection's configured
// default come into play.
if req.UseSourceOrder {
if qs, ok := r.EffectiveCollectionSort(ctx, access, kind, collectionID, sortConfig); ok {
req.Query.Sort = qs
req.UseSourceOrder = false
}
}
result, err := resolve(req)
if err != nil {
return nil, err
}
result.EffectiveSort = req.Query.Sort
return result, nil
}
func (r *CatalogResolver) resolveUserCollectionItems(
ctx context.Context,
req CatalogRequest,
access AccessFilter,
store userstore.UserStore,
collection *userstore.Collection,
) (*CatalogResult, error) {
if IsLiveQueryType(collection.CollectionType) {
def, err := parseCatalogCollectionQueryDefinition([]byte(collection.QueryDefinition))
if err != nil {
@@ -1752,10 +1686,7 @@ func (r *CatalogResolver) catalogStoreForAccess(ctx context.Context, access Acce
return store, nil
}
// ProfileCanAccessCollection reports whether profileID may view a personal
// collection. Creator access is represented in AllowedProfileIDs at write time,
// so this is the canonical check for both browse and preference endpoints.
func ProfileCanAccessCollection(collection *userstore.Collection, profileID string) bool {
func catalogProfileCanAccessCollection(collection *userstore.Collection, profileID string) bool {
if collection == nil || strings.TrimSpace(profileID) == "" {
return false
}
@@ -1875,7 +1806,7 @@ func (r *CatalogResolver) loadCollectionSourceBaseItems(ctx context.Context, req
case CatalogSourceLibraryCollection:
collectionRepo := NewLibraryCollectionRepository(r.itemRepo.pool)
collection, err := collectionRepo.GetByID(ctx, req.CollectionID)
if err != nil || collection.Visibility != LibraryCollectionVisibilityVisible {
if err != nil || collection.Visibility != "visible" {
return nil, ErrCatalogSourceNotFound
}
if IsLiveQueryType(collection.CollectionType) || catalogCollectionUsesLiveQuery(collection.QueryDefinition) {
@@ -1906,7 +1837,7 @@ func (r *CatalogResolver) loadCollectionSourceBaseItems(ctx context.Context, req
return nil, err
}
collection, err := store.GetCollection(ctx, req.CollectionID)
if err != nil || !ProfileCanAccessCollection(collection, access.ProfileID) {
if err != nil || !catalogProfileCanAccessCollection(collection, access.ProfileID) {
return nil, ErrCatalogSourceNotFound
}
var items []*models.MediaItem
-185
View File
@@ -1,185 +0,0 @@
package catalog
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"strings"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// QueryCollectionItemsBySort loads collection members through the catalog
// query executor, applying an optional personal-collection display filter and
// the resolved sort before the database limit. This keeps rail queries bounded
// without changing the full-match total returned to callers.
func QueryCollectionItemsBySort(
ctx context.Context,
pool *pgxpool.Pool,
contentIDs []string,
qs QuerySort,
access AccessFilter,
limit int,
displayQueryDefinition string,
) ([]*models.MediaItem, int, error) {
if len(contentIDs) == 0 {
return []*models.MediaItem{}, 0, nil
}
def := QueryDefinition{Sort: qs}
if trimmed := strings.TrimSpace(displayQueryDefinition); trimmed != "" {
if err := json.Unmarshal([]byte(trimmed), &def); err != nil {
return nil, 0, fmt.Errorf("parsing collection display_query_definition: %w", err)
}
// Display fragments are filter-only. Never let stale or malformed stored
// execution fields expand scope, replace the selected sort, or set a
// different limit.
def.MediaScope = ""
def.LibraryIDs = nil
def.Sort = qs
def.Limit = nil
}
queryAccess := access
if access.AllowedContentIDs != nil {
queryAccess.AllowedContentIDs = intersectContentIDs(contentIDs, access.AllowedContentIDs)
} else {
queryAccess.AllowedContentIDs = contentIDs
}
if limit <= 0 || limit > len(contentIDs) {
limit = len(contentIDs)
}
executor := &QueryExecutor{Pool: pool}
items, total, err := executor.Preview(ctx, def.Normalize(), queryAccess, limit)
if err != nil {
return nil, 0, fmt.Errorf("querying collection items by %q: %w", qs.Field, err)
}
return items, total, nil
}
// EffectiveCollectionSort resolves the order a collection's items are shown in
// when the request itself carries no sort. Precedence, highest first:
//
// 1. the viewing profile's saved override — set by changing the sort while
// browsing the collection, and sticky from then on;
// 2. the default sort the collection's creator configured (sort_config);
// 3. the collection's own source order, signaled by ok == false.
//
// An override row with an empty sort field is a real choice ("keep source
// order") and short-circuits the creator's default, which is why it returns
// false rather than falling through.
//
// Creator defaults for library collections stay user-agnostic because they are
// shared. A viewer override is profile-scoped, however, so personalized fields
// (progress, date viewed, plays) are valid for either collection kind.
func (r *CatalogResolver) EffectiveCollectionSort(
ctx context.Context,
access AccessFilter,
kind string,
collectionID string,
sortConfig []byte,
) (QuerySort, bool) {
if pref := r.collectionSortOverride(ctx, access, kind, collectionID); pref != nil {
if strings.TrimSpace(pref.SortField) == "" {
return QuerySort{}, false
}
if qs, ok := NormalizeCollectionSort(pref.SortField, pref.SortOrder, true); ok {
return qs, true
}
// A stored override that no longer validates (a sort field retired from
// the vocabulary) falls back to the creator's default rather than
// failing the browse.
}
return ParseCollectionDefaultSort(sortConfig, kind == userstore.CollectionKindUser)
}
// OrderCollectionItemsBySort reorders already-loaded collection members by a
// resolved sort, returning them in that order.
//
// Ordering runs through the catalog query executor constrained to the member
// IDs, so a rail orders items exactly as the collection's browse page does —
// same NULLS LAST handling, same sort_title collation — rather than through a
// second, drifting in-memory comparator. An empty sort field returns items
// untouched (source order). Items the executor does not return (no longer
// visible to this access scope) are dropped, matching the browse path.
func OrderCollectionItemsBySort(
ctx context.Context,
pool *pgxpool.Pool,
items []*models.MediaItem,
qs QuerySort,
access AccessFilter,
) ([]*models.MediaItem, error) {
if strings.TrimSpace(qs.Field) == "" || len(items) == 0 {
return items, nil
}
ids := make([]string, 0, len(items))
for _, item := range items {
if item != nil && strings.TrimSpace(item.ContentID) != "" {
ids = append(ids, item.ContentID)
}
}
if len(ids) == 0 {
return items, nil
}
ordered, _, err := QueryCollectionItemsBySort(ctx, pool, ids, qs, access, len(ids), "")
if err != nil {
return nil, fmt.Errorf("ordering collection items by %q: %w", qs.Field, err)
}
// Preview returns freshly loaded rows; re-project onto the caller's items so
// any enrichment already applied to them survives the reorder.
byID := make(map[string]*models.MediaItem, len(items))
for _, item := range items {
if item != nil {
byID[item.ContentID] = item
}
}
result := make([]*models.MediaItem, 0, len(ordered))
for _, item := range ordered {
if item == nil {
continue
}
if original, ok := byID[item.ContentID]; ok {
result = append(result, original)
}
}
return result, nil
}
// collectionSortOverride loads the viewing profile's saved sort for a
// collection, or nil when there is none. A lookup failure is logged and treated
// as "no override": a browse must not fail because a preference row could not
// be read.
func (r *CatalogResolver) collectionSortOverride(
ctx context.Context,
access AccessFilter,
kind string,
collectionID string,
) *userstore.CollectionSortPreference {
if r.storeProvider == nil || access.UserID <= 0 || strings.TrimSpace(access.ProfileID) == "" {
return nil
}
store, err := r.storeProvider.ForUser(ctx, access.UserID)
if err != nil || store == nil {
if err != nil {
slog.WarnContext(ctx, "loading user store for collection sort preference",
"collection_kind", kind, "collection_id", collectionID, "error", err)
}
return nil
}
pref, err := store.GetCollectionSortPreference(ctx, access.ProfileID, kind, collectionID)
if err != nil {
slog.WarnContext(ctx, "loading collection sort preference",
"collection_kind", kind, "collection_id", collectionID, "error", err)
return nil
}
return pref
}
-289
View File
@@ -1,289 +0,0 @@
package catalog
import (
"context"
"os"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userstore"
"github.com/Silo-Server/silo-server/internal/userstore/pgstore"
)
// These tests run the default-sort / saved-override flow against a real
// Postgres: real SQL ordering through the query executor, and a real
// PostgresUserStore writing the override row. They cover the reported
// behavior end to end — a collection's creator picks a default order, a viewer
// re-sorts, and the viewer's choice survives a fresh read.
//
// Set SILO_TEST_DATABASE_URL to a migrated database to run them.
func collectionSortTestPool(t *testing.T) *pgxpool.Pool {
t.Helper()
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
var tableName *string
if err := pool.QueryRow(ctx, `SELECT to_regclass('public.user_collection_sort_preferences')::text`).Scan(&tableName); err != nil {
t.Fatalf("check user_collection_sort_preferences table: %v", err)
}
if tableName == nil || *tableName == "" {
t.Skip("test database has not applied the collection sort preference migration")
}
return pool
}
// seedSortableItem inserts one sortable item. Callers choose title/year pairs
// that make a title sort and a year sort disagree, so an order assertion cannot
// pass by coincidence.
func seedSortableItem(t *testing.T, pool *pgxpool.Pool, contentID, title string, year int) {
t.Helper()
ctx := context.Background()
if _, err := pool.Exec(ctx, `
INSERT INTO media_items (content_id, type, title, sort_title, year, genres)
VALUES ($1, 'movie', $2, $2, $3, '{}'::text[])
`, contentID, title, year); err != nil {
t.Fatalf("seed media item %s: %v", contentID, err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM media_items WHERE content_id = $1`, contentID)
})
}
func seedSortTestUser(t *testing.T, pool *pgxpool.Pool) int {
t.Helper()
ctx := context.Background()
var userID int
// A bare fixture row: the override table has an FK to users(id) and the
// test needs something to point at. No credentials are set.
if err := pool.QueryRow(ctx, `
INSERT INTO users (username, email, password_hash, role)
VALUES ('collection-sort-fixture', 'collection-sort-fixture@invalid.test', '', 'user')
RETURNING id
`).Scan(&userID); err != nil {
t.Fatalf("seed fixture user: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM users WHERE id = $1`, userID)
})
return userID
}
func itemsForSort(contentIDs ...string) []*models.MediaItem {
items := make([]*models.MediaItem, 0, len(contentIDs))
for _, id := range contentIDs {
items = append(items, &models.MediaItem{ContentID: id})
}
return items
}
func orderedIDs(items []*models.MediaItem) []string {
ids := make([]string, 0, len(items))
for _, item := range items {
ids = append(ids, item.ContentID)
}
return ids
}
func assertOrder(t *testing.T, got []string, want ...string) {
t.Helper()
if len(got) != len(want) {
t.Fatalf("order = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("order = %v, want %v", got, want)
}
}
}
// TestCollectionDefaultSortAndOverrideEndToEnd walks the whole reported
// scenario against real Postgres, for both collection kinds.
func TestCollectionDefaultSortAndOverrideEndToEnd(t *testing.T) {
pool := collectionSortTestPool(t)
ctx := context.Background()
// Years are shuffled against titles on purpose: title-asc gives A,B,C but
// year-desc gives B,C,A. If the two agreed, an assertion on the override's
// order would also pass when the override was ignored entirely.
seedSortableItem(t, pool, "cs-a", "Alpha", 2001)
seedSortableItem(t, pool, "cs-b", "Bravo", 2003)
seedSortableItem(t, pool, "cs-c", "Charlie", 2002)
// Source order is deliberately neither sorted order.
sourceOrder := itemsForSort("cs-c", "cs-a", "cs-b")
limited, total, err := QueryCollectionItemsBySort(
ctx,
pool,
[]string{"cs-c", "cs-a", "cs-b"},
QuerySort{Field: "title", Order: "asc"},
AccessFilter{},
2,
"",
)
if err != nil {
t.Fatalf("querying limited sorted collection: %v", err)
}
assertOrder(t, orderedIDs(limited), "cs-a", "cs-b")
if total != 3 {
t.Fatalf("limited sorted collection total = %d, want 3", total)
}
userID := seedSortTestUser(t, pool)
provider := pgstore.NewPostgresProvider(pool)
store, err := provider.ForUser(ctx, userID)
if err != nil {
t.Fatalf("user store: %v", err)
}
if err := store.CreateProfile(ctx, userstore.Profile{ID: "profile-e2e", Name: "E2E sort profile"}); err != nil {
t.Fatalf("seed profile: %v", err)
}
t.Cleanup(func() { _ = store.DeleteProfile(context.Background(), "profile-e2e") })
resolver := &CatalogResolver{storeProvider: provider}
access := AccessFilter{UserID: userID, ProfileID: "profile-e2e"}
for _, kind := range []string{userstore.CollectionKindLibrary, userstore.CollectionKindUser} {
t.Run(kind, func(t *testing.T) {
collectionID := "cs-collection-" + kind
t.Cleanup(func() {
_ = store.ClearCollectionSortPreference(ctx, access.ProfileID, kind, collectionID)
})
// 1. No default, no override — the collection keeps source order.
if _, ok := resolver.EffectiveCollectionSort(ctx, access, kind, collectionID, []byte(`{}`)); ok {
t.Fatal("a collection with no default reported a sort")
}
// 2. The creator sets a default of Title (A→Z).
defaultSort := []byte(`{"field":"title","order":"asc"}`)
qs, ok := resolver.EffectiveCollectionSort(ctx, access, kind, collectionID, defaultSort)
if !ok {
t.Fatal("configured default sort was not applied")
}
ordered, err := OrderCollectionItemsBySort(ctx, pool, sourceOrder, qs, access)
if err != nil {
t.Fatalf("ordering by default sort: %v", err)
}
assertOrder(t, orderedIDs(ordered), "cs-a", "cs-b", "cs-c")
// 3. The viewer re-sorts to Year (newest first) and it is saved.
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: access.ProfileID,
CollectionKind: kind,
CollectionID: collectionID,
SortField: "year",
SortOrder: "desc",
}); err != nil {
t.Fatalf("saving override: %v", err)
}
// 4. A fresh resolution — the "leave and come back" case — reads the
// override back out of the database and beats the creator's default.
freshResolver := &CatalogResolver{storeProvider: pgstore.NewPostgresProvider(pool)}
qs, ok = freshResolver.EffectiveCollectionSort(ctx, access, kind, collectionID, defaultSort)
if !ok {
t.Fatal("saved override was not applied on a fresh read")
}
if qs.Field != "year" || qs.Order != "desc" {
t.Fatalf("effective sort = %q/%q, want year/desc", qs.Field, qs.Order)
}
ordered, err = OrderCollectionItemsBySort(ctx, pool, sourceOrder, qs, access)
if err != nil {
t.Fatalf("ordering by override: %v", err)
}
// Distinct from both the default's A,B,C and source order's C,A,B.
assertOrder(t, orderedIDs(ordered), "cs-b", "cs-c", "cs-a")
assertOrder(t, orderedIDs(sourceOrder), "cs-c", "cs-a", "cs-b")
// 5. The viewer pins back to the collection's own order. That is a
// real choice and must suppress the creator's default, not fall
// through to it.
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: access.ProfileID,
CollectionKind: kind,
CollectionID: collectionID,
SortField: "",
SortOrder: "",
}); err != nil {
t.Fatalf("pinning to source order: %v", err)
}
if _, ok := resolver.EffectiveCollectionSort(ctx, access, kind, collectionID, defaultSort); ok {
t.Fatal("source-order override fell through to the creator's default")
}
// 6. Clearing the override returns the viewer to the default.
if err := store.ClearCollectionSortPreference(ctx, access.ProfileID, kind, collectionID); err != nil {
t.Fatalf("clearing override: %v", err)
}
qs, ok = resolver.EffectiveCollectionSort(ctx, access, kind, collectionID, defaultSort)
if !ok || qs.Field != "title" {
t.Fatalf("after clearing, effective sort = %q (ok=%v), want title", qs.Field, ok)
}
})
}
}
// TestCollectionSortOverrideIsPerProfile guards the isolation that makes a
// server-wide collection safe to re-sort: one profile's choice must not change
// what another profile sees.
func TestCollectionSortOverrideIsPerProfile(t *testing.T) {
pool := collectionSortTestPool(t)
ctx := context.Background()
userID := seedSortTestUser(t, pool)
provider := pgstore.NewPostgresProvider(pool)
store, err := provider.ForUser(ctx, userID)
if err != nil {
t.Fatalf("user store: %v", err)
}
for _, profile := range []userstore.Profile{
{ID: "profile-one", Name: "Profile One"},
{ID: "profile-two", Name: "Profile Two"},
} {
if err := store.CreateProfile(ctx, profile); err != nil {
t.Fatalf("seed profile %s: %v", profile.ID, err)
}
}
t.Cleanup(func() {
_ = store.DeleteProfile(context.Background(), "profile-two")
_ = store.DeleteProfile(context.Background(), "profile-one")
})
resolver := &CatalogResolver{storeProvider: provider}
const collectionID = "cs-shared-collection"
defaultSort := []byte(`{"field":"title","order":"asc"}`)
kind := userstore.CollectionKindLibrary
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: "profile-one",
CollectionKind: kind,
CollectionID: collectionID,
SortField: "year",
SortOrder: "desc",
}); err != nil {
t.Fatalf("saving override: %v", err)
}
t.Cleanup(func() {
_ = store.ClearCollectionSortPreference(context.Background(), "profile-one", kind, collectionID)
})
one, ok := resolver.EffectiveCollectionSort(ctx, AccessFilter{UserID: userID, ProfileID: "profile-one"}, kind, collectionID, defaultSort)
if !ok || one.Field != "year" {
t.Fatalf("profile-one sort = %q (ok=%v), want year", one.Field, ok)
}
two, ok := resolver.EffectiveCollectionSort(ctx, AccessFilter{UserID: userID, ProfileID: "profile-two"}, kind, collectionID, defaultSort)
if !ok || two.Field != "title" {
t.Fatalf("profile-two sort = %q (ok=%v), want the collection default title", two.Field, ok)
}
}
@@ -1,189 +0,0 @@
package catalog
import (
"context"
"errors"
"testing"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// sortPrefStore is a UserStore that only answers collection-sort-preference
// lookups; every other method panics so a test that strays off this path fails
// loudly rather than silently exercising unrelated behavior.
type sortPrefStore struct {
userstore.UserStore
pref *userstore.CollectionSortPreference
err error
// calls records the lookups made, so tests can assert the store is not
// consulted when there is no user scope to consult it with.
calls int
}
func (s *sortPrefStore) GetCollectionSortPreference(_ context.Context, _, _, _ string) (*userstore.CollectionSortPreference, error) {
s.calls++
if s.err != nil {
return nil, s.err
}
return s.pref, nil
}
type sortPrefProvider struct {
store *sortPrefStore
err error
}
func (p *sortPrefProvider) ForUser(context.Context, int) (userstore.UserStore, error) {
if p.err != nil {
return nil, p.err
}
return p.store, nil
}
func (p *sortPrefProvider) Close() error { return nil }
func TestEffectiveCollectionSortPrecedence(t *testing.T) {
const (
libraryDefault = `{"field":"added_at","order":"desc"}`
noDefault = `{}`
)
scopedAccess := AccessFilter{UserID: 7, ProfileID: "profile-1"}
cases := []struct {
name string
access AccessFilter
sortConfig string
pref *userstore.CollectionSortPreference
storeErr error
wantOK bool
wantField string
wantOrder string
wantCalls int
}{
{
name: "no override falls back to the collection default",
access: scopedAccess,
sortConfig: libraryDefault,
wantOK: true,
wantField: "added_at",
wantOrder: "desc",
wantCalls: 1,
},
{
// The reported behavior: admin defaults to Date Added, the viewer
// switches to Title, and Title is what they get from then on.
name: "viewer override beats the collection default",
access: scopedAccess,
sortConfig: libraryDefault,
pref: &userstore.CollectionSortPreference{SortField: "title", SortOrder: "asc"},
wantOK: true,
wantField: "title",
wantOrder: "asc",
wantCalls: 1,
},
{
// An empty stored field is a deliberate "put it back to the list's
// own order" and must not fall through to the creator's default.
name: "override pinned to source order suppresses the default",
access: scopedAccess,
sortConfig: libraryDefault,
pref: &userstore.CollectionSortPreference{SortField: "", SortOrder: ""},
wantCalls: 1,
},
{
name: "no override and no default means source order",
access: scopedAccess,
sortConfig: noDefault,
wantCalls: 1,
},
{
name: "personalized viewer override is valid on a library collection",
access: scopedAccess,
sortConfig: libraryDefault,
pref: &userstore.CollectionSortPreference{SortField: "progress", SortOrder: "desc"},
wantOK: true,
wantField: "progress",
wantOrder: "desc",
wantCalls: 1,
},
{
name: "unusable override falls back to the default",
access: scopedAccess,
sortConfig: libraryDefault,
pref: &userstore.CollectionSortPreference{SortField: "retired_sort", SortOrder: "desc"},
wantOK: true,
wantField: "added_at",
wantOrder: "desc",
wantCalls: 1,
},
{
name: "store failure degrades to the default",
access: scopedAccess,
sortConfig: libraryDefault,
storeErr: errors.New("boom"),
wantOK: true,
wantField: "added_at",
wantOrder: "desc",
wantCalls: 1,
},
{
// Unauthenticated / profile-less reads (jellycompat, public paths)
// have no override to look up and must not query for one.
name: "no user scope skips the override lookup",
access: AccessFilter{},
sortConfig: libraryDefault,
pref: &userstore.CollectionSortPreference{SortField: "title", SortOrder: "asc"},
wantOK: true,
wantField: "added_at",
wantOrder: "desc",
wantCalls: 0,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
store := &sortPrefStore{pref: tc.pref, err: tc.storeErr}
resolver := &CatalogResolver{storeProvider: &sortPrefProvider{store: store}}
qs, ok := resolver.EffectiveCollectionSort(
context.Background(),
tc.access,
userstore.CollectionKindLibrary,
"collection-1",
[]byte(tc.sortConfig),
)
if ok != tc.wantOK {
t.Fatalf("ok = %v, want %v", ok, tc.wantOK)
}
if store.calls != tc.wantCalls {
t.Fatalf("override lookups = %d, want %d", store.calls, tc.wantCalls)
}
if !tc.wantOK {
return
}
if qs.Field != tc.wantField || qs.Order != tc.wantOrder {
t.Fatalf("got %q/%q, want %q/%q", qs.Field, qs.Order, tc.wantField, tc.wantOrder)
}
})
}
}
func TestEffectiveCollectionSortAllowsPersonalizedOnUserCollections(t *testing.T) {
store := &sortPrefStore{pref: &userstore.CollectionSortPreference{SortField: "progress", SortOrder: "desc"}}
resolver := &CatalogResolver{storeProvider: &sortPrefProvider{store: store}}
qs, ok := resolver.EffectiveCollectionSort(
context.Background(),
AccessFilter{UserID: 7, ProfileID: "profile-1"},
userstore.CollectionKindUser,
"collection-1",
[]byte(`{}`),
)
if !ok {
t.Fatal("personalized override rejected on a personal collection")
}
if qs.Field != "progress" || qs.Order != "desc" {
t.Fatalf("got %q/%q, want progress/desc", qs.Field, qs.Order)
}
}
-105
View File
@@ -1,105 +0,0 @@
package catalog
import "testing"
func TestNormalizeCollectionSort(t *testing.T) {
cases := []struct {
name string
field string
order string
allowPersonalized bool
wantOK bool
wantField string
wantOrder string
}{
{name: "applies the field's default order", field: "release_date", wantOK: true, wantField: "release_date", wantOrder: "desc"},
{name: "title defaults ascending", field: "title", wantOK: true, wantField: "title", wantOrder: "asc"},
{name: "explicit order wins", field: "title", order: "desc", wantOK: true, wantField: "title", wantOrder: "desc"},
{name: "trims and lowercases", field: " Title ", order: "ASC", wantOK: true, wantField: "title", wantOrder: "asc"},
{name: "garbage order is rejected", field: "year", order: "sideways"},
{name: "empty field means source order", field: ""},
{name: "unknown field is rejected", field: "not_a_sort"},
{name: "personalized rejected for library collections", field: "progress"},
{
name: "personalized allowed for personal collections",
field: "progress",
allowPersonalized: true,
wantOK: true,
wantField: "progress",
wantOrder: "desc",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
qs, ok := NormalizeCollectionSort(tc.field, tc.order, tc.allowPersonalized)
if ok != tc.wantOK {
t.Fatalf("ok = %v, want %v", ok, tc.wantOK)
}
if !tc.wantOK {
return
}
if qs.Field != tc.wantField || qs.Order != tc.wantOrder {
t.Fatalf("got %q/%q, want %q/%q", qs.Field, qs.Order, tc.wantField, tc.wantOrder)
}
})
}
}
func TestParseCollectionDefaultSort(t *testing.T) {
cases := []struct {
name string
raw string
wantOK bool
wantField string
wantOrder string
}{
{name: "empty column", raw: ""},
{name: "legacy empty object", raw: `{}`},
{name: "json null", raw: `null`},
{name: "unparseable json", raw: `{`},
// The unimplemented manual-pin mode carries no field, so it must not be
// mistaken for a configured sort.
{name: "manual pins mode", raw: `{"mode":"manual_pins"}`},
{name: "configured sort", raw: `{"field":"title","order":"asc"}`, wantOK: true, wantField: "title", wantOrder: "asc"},
{name: "order filled in", raw: `{"field":"added_at"}`, wantOK: true, wantField: "added_at", wantOrder: "desc"},
{name: "invalid order is rejected", raw: `{"field":"title","order":"sideways"}`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
qs, ok := ParseCollectionDefaultSort([]byte(tc.raw), false)
if ok != tc.wantOK {
t.Fatalf("ok = %v, want %v", ok, tc.wantOK)
}
if !tc.wantOK {
return
}
if qs.Field != tc.wantField || qs.Order != tc.wantOrder {
t.Fatalf("got %q/%q, want %q/%q", qs.Field, qs.Order, tc.wantField, tc.wantOrder)
}
})
}
}
func TestEncodeCollectionDefaultSort(t *testing.T) {
encoded, err := EncodeCollectionDefaultSort(QuerySort{Field: "title", Order: "asc"}, true)
if err != nil {
t.Fatalf("encoding: %v", err)
}
if encoded != `{"field":"title","order":"asc"}` {
t.Fatalf("encoded = %s", encoded)
}
// Source order round-trips through the empty object the column defaults to.
encoded, err = EncodeCollectionDefaultSort(QuerySort{}, false)
if err != nil {
t.Fatalf("encoding source order: %v", err)
}
if encoded != "{}" {
t.Fatalf("source order encoded = %s, want {}", encoded)
}
if _, ok := ParseCollectionDefaultSort([]byte(encoded), false); ok {
t.Fatal("re-parsing the source-order encoding reported a configured sort")
}
}
+1 -1
View File
@@ -200,7 +200,7 @@ func (r *LibraryCollectionRepository) Create(ctx context.Context, input CreateLi
input.SortConfig = json.RawMessage(`{}`)
}
if input.Visibility == "" {
input.Visibility = LibraryCollectionVisibilityVisible
input.Visibility = "visible"
}
if input.CollectionType == "" {
input.CollectionType = "manual"
-65
View File
@@ -1,7 +1,6 @@
package catalog
import (
"bytes"
"encoding/json"
"fmt"
"sort"
@@ -214,70 +213,6 @@ func NormalizePersonalListSort(field, order string) (QuerySort, bool) {
return QuerySort{Field: field, Order: order}, true
}
// NormalizeCollectionSort validates a collection's default sort (its stored
// sort_config) or a viewer's saved override, applying the field's default order
// when none is given. Returns false when the field or a supplied order is
// unsupported. An empty field means the collection's own source order should
// be kept.
//
// allowPersonalized gates the per-profile sort fields (progress, date viewed,
// plays). They are legitimate for personal collections, which always resolve
// with the owning profile in scope, but not for library collections: those
// resolve with user scope stripped (see stripCatalogUserScope), so the query
// builder's ensureUserScope would reject the sort at browse time.
func NormalizeCollectionSort(field, order string, allowPersonalized bool) (QuerySort, bool) {
field = strings.ToLower(strings.TrimSpace(field))
def, ok := querySortDefs[field]
if !ok {
return QuerySort{}, false
}
if def.personalized && !allowPersonalized {
return QuerySort{}, false
}
order = strings.ToLower(strings.TrimSpace(order))
if order == "" {
order = def.defaultOrder
} else if order != "asc" && order != "desc" {
return QuerySort{}, false
}
return QuerySort{Field: field, Order: order}, true
}
// collectionSortConfig is the shape a collection's sort_config carries when it
// pins a default order. Everything else the column may hold (legacy `{}`, the
// unimplemented manual-pin mode) leaves the collection on source order.
type collectionSortConfig struct {
Field string `json:"field"`
Order string `json:"order"`
}
// ParseCollectionDefaultSort reads the default sort a collection's creator
// configured. An absent, empty, or unrecognized sort_config yields false,
// meaning source order.
func ParseCollectionDefaultSort(raw []byte, allowPersonalized bool) (QuerySort, bool) {
if len(bytes.TrimSpace(raw)) == 0 {
return QuerySort{}, false
}
var cfg collectionSortConfig
if err := json.Unmarshal(raw, &cfg); err != nil {
return QuerySort{}, false
}
return NormalizeCollectionSort(cfg.Field, cfg.Order, allowPersonalized)
}
// EncodeCollectionDefaultSort renders a validated default sort back into the
// sort_config JSON shape. An empty field encodes as `{}` — source order.
func EncodeCollectionDefaultSort(qs QuerySort, ok bool) (string, error) {
if !ok || strings.TrimSpace(qs.Field) == "" {
return "{}", nil
}
encoded, err := json.Marshal(collectionSortConfig(qs))
if err != nil {
return "", fmt.Errorf("encoding collection sort_config: %w", err)
}
return string(encoded), nil
}
func (q QueryDefinition) Normalize() QueryDefinition {
normalized := q
normalized.MediaScope = strings.ToLower(strings.TrimSpace(normalized.MediaScope))
+1 -13
View File
@@ -259,10 +259,6 @@ var attrRegistry = map[string]map[string]attrValueType{
"bitrate_kbps": attrInteger,
"dropped_frames": attrInteger,
"audio_underruns": attrInteger,
"session_id": attrString,
"play_method": attrString,
"reason": attrString,
"position_ms": attrInteger,
},
"focus": {
"target": attrString,
@@ -273,17 +269,9 @@ var attrRegistry = map[string]map[string]attrValueType{
"path": attrString,
"status": attrInteger,
"duration_ms": attrInteger,
"outcome": attrString,
"error_code": attrString,
"attempt": attrInteger,
},
"lifecycle": {
"state": attrString,
"phase": attrString,
"duration_ms": attrInteger,
"outcome": attrString,
"reason": attrString,
"launch_type": attrString,
"state": attrString,
},
"crash": {
"fingerprint": attrString,
+8 -4
View File
@@ -74,12 +74,16 @@ func effectiveArtifactDir(artifactDir, transcodeDir string) string {
return config.EffectiveDownloadArtifactDir(artifactDir, transcodeDir)
}
// artifactOutputPath derives a deterministic output path from
// (media_file_id, format, params_hash) so a reclaimed job targets the same file.
func artifactOutputPath(dir string, mediaFileID int, format, hash string) string {
// artifactOutputPath derives an output path from
// (media_file_id, format, params_hash, artifact_id). It is deterministic for a
// given artifact row — a reclaimed job targets the same file — while the
// artifact id keeps a REPLACEMENT row's path distinct from its predecessor's.
// Without that, an evicted row's bytes could be unlinked after a successor with
// the same dedup key had already claimed the identical path.
func artifactOutputPath(dir string, mediaFileID int, format, hash, artifactID string) string {
short := hash
if len(short) > 16 {
short = short[:16]
}
return filepath.Join(dir, fmt.Sprintf("%d_%s_%s.mp4", mediaFileID, format, short))
return filepath.Join(dir, fmt.Sprintf("%d_%s_%s_%s.mp4", mediaFileID, format, short, artifactID))
}
+137 -21
View File
@@ -16,6 +16,13 @@ const artifactColumns = `id, media_file_id, format, params_hash, container, code
attempts, max_attempts, lease_owner, lease_expires_at, next_retry_at,
created_at, completed_at, last_used_at`
// activeArtifactLinkSQL is the EXISTS predicate that protects an artifact from
// eviction. Completed/ready/preparing/queued/downloading rows remain servable
// handles; canceled/failed/revoked do not.
const activeArtifactLinkSQL = `EXISTS(SELECT 1 FROM downloads
WHERE artifact_id = $1
AND status NOT IN ('cancelled','failed','revoked'))`
// ArtifactRepository provides CRUD + durable-queue operations for
// download_artifacts.
type ArtifactRepository struct {
@@ -401,23 +408,6 @@ func (r *ArtifactRepository) TotalReadyBytes(ctx context.Context) (int64, error)
return total, nil
}
// HasActiveLink reports whether any valid download row — managed or ephemeral
// (device-less web) — still references the artifact. Completed rows are
// retained because they remain re-downloadable handles; evicting an artifact a
// live row references would 404 a download the API advertises as servable.
func (r *ArtifactRepository) HasActiveLink(ctx context.Context, artifactID string) (bool, error) {
var exists bool
if err := r.pool.QueryRow(ctx,
`SELECT EXISTS(SELECT 1 FROM downloads
WHERE artifact_id = $1
AND status NOT IN ('cancelled','failed','revoked'))`,
artifactID,
).Scan(&exists); err != nil {
return false, fmt.Errorf("checking artifact links: %w", err)
}
return exists, nil
}
// ListFailedBefore returns terminally-failed artifacts cold since cutoff
// (last_used_at). Their linked downloads were already flipped to 'failed' by
// reconciliation, so the rows serve nothing and only block re-attempts.
@@ -447,11 +437,122 @@ func (r *ArtifactRepository) ListUnlinkedReadyBefore(ctx context.Context, cutoff
return scanArtifacts(rows)
}
// DeleteArtifact removes an artifact row.
func (r *ArtifactRepository) DeleteArtifact(ctx context.Context, id string) error {
_, err := r.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE id = $1`, id)
// DeleteReadyIfEvictable atomically deletes a ready artifact only when no
// active download row references it. Check-then-delete eviction races a new
// download linking the same id; the NOT-EXISTS predicate is the fast path and
// downloads_artifact_id_fkey (ON DELETE RESTRICT) is the enforcement backstop —
// a concurrent link that commits after this statement's snapshot makes the RI
// trigger raise 23503, which reads as "not evictable".
func (r *ArtifactRepository) DeleteReadyIfEvictable(ctx context.Context, id string) (bool, error) {
return r.deleteIfEvictable(ctx, id, ArtifactReady)
}
// DeleteFailedIfEvictable is the hygiene-sweep twin of DeleteReadyIfEvictable.
// The status pin matters even without a link: Ensure requeues a failed row and
// the drain can claim it as running before any download links it — a
// status-unrestricted delete would remove that live job out from under its
// worker.
func (r *ArtifactRepository) DeleteFailedIfEvictable(ctx context.Context, id string) (bool, error) {
return r.deleteIfEvictable(ctx, id, ArtifactFailed)
}
func (r *ArtifactRepository) deleteIfEvictable(ctx context.Context, id string, requireStatus string) (bool, error) {
query := `DELETE FROM download_artifacts WHERE id = $1
AND status = $2
AND NOT ` + activeArtifactLinkSQL
tag, err := r.pool.Exec(ctx, query, id, requireStatus)
if err != nil {
return fmt.Errorf("deleting artifact: %w", err)
// A download row linked the artifact concurrently: not evictable.
if isForeignKeyViolation(err, downloadsArtifactFKConstraint) {
return false, nil
}
return false, fmt.Errorf("evicting unlinked artifact: %w", err)
}
return tag.RowsAffected() == 1, nil
}
// LocalArtifactOrphan is a filesystem cleanup candidate recorded before its
// artifact row is deleted, so a crash or unlink failure between the row delete
// and the byte delete can never leak the file.
type LocalArtifactOrphan struct {
ID int64
DownloadArtifactID string
OutputPath string
Attempts int
}
// EnqueueLocalOrphan durably records a local artifact path for deletion. Paths
// are unique per artifact id, so a conflicting row already covers the same file.
func (r *ArtifactRepository) EnqueueLocalOrphan(ctx context.Context, artifactID, outputPath string) error {
if artifactID == "" || outputPath == "" {
return nil
}
_, err := r.pool.Exec(ctx,
`INSERT INTO download_artifact_local_orphans (download_artifact_id, output_path)
VALUES ($1, $2)
ON CONFLICT (output_path) DO NOTHING`,
artifactID, outputPath,
)
if err != nil {
return fmt.Errorf("enqueueing local artifact cleanup: %w", err)
}
return nil
}
// ListLocalOrphansDue returns due local cleanup candidates, oldest first.
func (r *ArtifactRepository) ListLocalOrphansDue(ctx context.Context, limit int) ([]LocalArtifactOrphan, error) {
if limit <= 0 {
limit = 100
}
rows, err := r.pool.Query(ctx,
`SELECT id, download_artifact_id, output_path, attempts
FROM download_artifact_local_orphans
WHERE next_retry_at IS NULL OR next_retry_at <= now()
ORDER BY attempts, created_at, id
LIMIT $1`, limit)
if err != nil {
return nil, fmt.Errorf("listing local artifact cleanup queue: %w", err)
}
defer rows.Close()
out := make([]LocalArtifactOrphan, 0, limit)
for rows.Next() {
var orphan LocalArtifactOrphan
if err := rows.Scan(&orphan.ID, &orphan.DownloadArtifactID, &orphan.OutputPath, &orphan.Attempts); err != nil {
return nil, fmt.Errorf("scanning local artifact cleanup row: %w", err)
}
out = append(out, orphan)
}
return out, rows.Err()
}
// DeleteLocalOrphan clears a local cleanup row by id.
func (r *ArtifactRepository) DeleteLocalOrphan(ctx context.Context, id int64) error {
if _, err := r.pool.Exec(ctx, `DELETE FROM download_artifact_local_orphans WHERE id = $1`, id); err != nil {
return fmt.Errorf("deleting local artifact cleanup row: %w", err)
}
return nil
}
// DeleteLocalOrphanByPath clears a local cleanup row after the inline unlink
// that followed its enqueue succeeded.
func (r *ArtifactRepository) DeleteLocalOrphanByPath(ctx context.Context, outputPath string) error {
if outputPath == "" {
return nil
}
if _, err := r.pool.Exec(ctx, `DELETE FROM download_artifact_local_orphans WHERE output_path = $1`, outputPath); err != nil {
return fmt.Errorf("deleting local artifact cleanup row by path: %w", err)
}
return nil
}
// BumpLocalOrphanRetry defers a failed unlink behind the same 2-minute backoff
// the remote queue uses.
func (r *ArtifactRepository) BumpLocalOrphanRetry(ctx context.Context, id int64) error {
if _, err := r.pool.Exec(ctx,
`UPDATE download_artifact_local_orphans
SET attempts = attempts + 1, next_retry_at = now() + interval '2 minutes'
WHERE id = $1`, id); err != nil {
return fmt.Errorf("deferring local artifact cleanup row: %w", err)
}
return nil
}
@@ -557,6 +658,21 @@ func (r *ArtifactRepository) PrepareRemoteOrphanCleanup(ctx context.Context, orp
return owned, true, nil
}
// DeleteRemoteOrphanByLocator clears a cleanup intent whose remote delete
// already succeeded inline.
func (r *ArtifactRepository) DeleteRemoteOrphanByLocator(ctx context.Context, nodeID int, artifactID string) error {
if nodeID <= 0 || artifactID == "" {
return nil
}
if _, err := r.pool.Exec(ctx,
`DELETE FROM download_artifact_orphans WHERE origin_node_id = $1 AND origin_artifact_id = $2`,
nodeID, artifactID,
); err != nil {
return fmt.Errorf("deleting remote artifact cleanup row by locator: %w", err)
}
return nil
}
func (r *ArtifactRepository) DeleteRemoteOrphan(ctx context.Context, id int64) error {
if _, err := r.pool.Exec(ctx, `DELETE FROM download_artifact_orphans WHERE id = $1`, id); err != nil {
return fmt.Errorf("deleting remote artifact cleanup row: %w", err)
+492 -9
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"sync"
"testing"
"time"
@@ -48,6 +49,9 @@ func newArtifactTestRepo(t *testing.T) (*ArtifactRepository, *pgxpool.Pool, int)
t.Fatalf("seed media file: %v", err)
}
t.Cleanup(func() {
// downloads_artifact_id_fkey is ON DELETE RESTRICT: unpin any surviving
// links before removing the artifact rows they reference.
_, _ = pool.Exec(ctx, `UPDATE downloads SET artifact_id = NULL WHERE media_file_id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
@@ -718,10 +722,10 @@ func TestListRemoteOrphansDueIsFairAcrossOrigins(t *testing.T) {
}
}
// TestHasActiveLinkCoversEphemeralRows pins the eviction guard: an ephemeral
// TestEvictionFenceCoversEphemeralRows pins the eviction guard: an ephemeral
// (device-less web) download row must protect its artifact from LRU cleanup
// exactly like a managed row does, and terminal rows must not.
func TestHasActiveLinkCoversEphemeralRows(t *testing.T) {
func TestEvictionFenceCoversEphemeralRows(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
@@ -729,6 +733,7 @@ func TestHasActiveLinkCoversEphemeralRows(t *testing.T) {
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("ensure artifact: %v", err)
}
markArtifactReady(t, pool, art.ID)
var userID int
if err := pool.QueryRow(ctx,
@@ -754,22 +759,500 @@ func TestHasActiveLinkCoversEphemeralRows(t *testing.T) {
t.Fatalf("create ephemeral download: %v", err)
}
active, err := repo.HasActiveLink(ctx, art.ID)
deleted, err := repo.DeleteReadyIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("HasActiveLink: %v", err)
t.Fatalf("DeleteReadyIfEvictable with ephemeral link: %v", err)
}
if !active {
if deleted {
t.Fatal("ephemeral ready row must protect its artifact from eviction")
}
if _, err := pool.Exec(ctx, `UPDATE downloads SET status = 'cancelled' WHERE id = $1`, dlID); err != nil {
if _, err := pool.Exec(ctx, `UPDATE downloads SET status = 'cancelled', artifact_id = NULL WHERE id = $1`, dlID); err != nil {
t.Fatalf("cancel download: %v", err)
}
active, err = repo.HasActiveLink(ctx, art.ID)
deleted, err = repo.DeleteReadyIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("HasActiveLink after cancel: %v", err)
t.Fatalf("DeleteReadyIfEvictable after cancel: %v", err)
}
if active {
if !deleted {
t.Fatal("terminal-only links must not protect an artifact")
}
}
// TestDeleteFailedIfEvictableFencesRequeuedArtifact covers the failed-artifact
// sweep: a linked row must survive the sweep, an unlinked row Ensure requeued
// (status back to 'queued'/'running') must survive it too, and the row becomes
// removable only once it is failed and unlinked.
func TestDeleteFailedIfEvictableFencesRequeuedArtifact(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
art := newArtifact(t, fileID, fmt.Sprintf("hash-requeued-%d", time.Now().UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("ensure artifact: %v", err)
}
userID := seedDownloadUser(t, pool)
dlRepo := NewRepository(pool)
now := time.Now()
dlID := fmt.Sprintf("dl-requeued-%d", now.UnixNano())
if err := dlRepo.Create(ctx, &Download{
ID: dlID, UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("requeued-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusPreparing, Format: FormatTranscode,
ArtifactID: art.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create preparing download: %v", err)
}
deleted, err := repo.DeleteFailedIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("DeleteFailedIfEvictable linked: %v", err)
}
if deleted {
t.Fatal("a linked non-failed artifact must not be swept")
}
if _, err := repo.GetByID(ctx, art.ID); err != nil {
t.Fatalf("linked artifact missing after sweep attempt: %v", err)
}
if _, err := pool.Exec(ctx, `UPDATE downloads SET status = 'failed', artifact_id = NULL WHERE id = $1`, dlID); err != nil {
t.Fatalf("fail download: %v", err)
}
// Unlinked but still queued: a requeued job the drain may already own must
// survive a status-unrestricted sweep would have removed it.
deleted, err = repo.DeleteFailedIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("DeleteFailedIfEvictable requeued: %v", err)
}
if deleted {
t.Fatal("an unlinked requeued artifact must not be swept")
}
if _, err := pool.Exec(ctx, `UPDATE download_artifacts SET status = 'failed' WHERE id = $1`, art.ID); err != nil {
t.Fatalf("fail artifact: %v", err)
}
deleted, err = repo.DeleteFailedIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("DeleteFailedIfEvictable after fail: %v", err)
}
if !deleted {
t.Fatal("an unlinked failed artifact must be swept")
}
}
// TestLocalOrphanQueueRoundTrip covers the durable local cleanup intent: an
// enqueue is idempotent per path, becomes due immediately, defers on a bumped
// retry, and clears by id and by path.
func TestLocalOrphanQueueRoundTrip(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
var present *string
if err := pool.QueryRow(ctx, `SELECT to_regclass('public.download_artifact_local_orphans')::text`).Scan(&present); err != nil {
t.Fatalf("check download_artifact_local_orphans: %v", err)
}
if present == nil {
t.Skip("download_artifact_local_orphans migration has not been applied")
}
art := newArtifact(t, fileID, fmt.Sprintf("hash-local-orphan-%d", time.Now().UnixNano()))
path := art.OutputPath
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM download_artifact_local_orphans WHERE output_path = $1`, path)
})
if err := repo.EnqueueLocalOrphan(ctx, art.ID, path); err != nil {
t.Fatalf("EnqueueLocalOrphan: %v", err)
}
if err := repo.EnqueueLocalOrphan(ctx, art.ID, path); err != nil {
t.Fatalf("EnqueueLocalOrphan (repeat): %v", err)
}
due, err := repo.ListLocalOrphansDue(ctx, 500)
if err != nil {
t.Fatalf("ListLocalOrphansDue: %v", err)
}
mine := localOrphansForPath(due, path)
if len(mine) != 1 {
t.Fatalf("due local orphans for path = %d, want exactly 1 (enqueue must dedupe)", len(mine))
}
if mine[0].DownloadArtifactID != art.ID {
t.Fatalf("orphan artifact id = %q, want %q", mine[0].DownloadArtifactID, art.ID)
}
if err := repo.BumpLocalOrphanRetry(ctx, mine[0].ID); err != nil {
t.Fatalf("BumpLocalOrphanRetry: %v", err)
}
due, err = repo.ListLocalOrphansDue(ctx, 500)
if err != nil {
t.Fatalf("ListLocalOrphansDue after bump: %v", err)
}
if len(localOrphansForPath(due, path)) != 0 {
t.Fatal("a bumped local orphan must not be due again immediately")
}
if err := repo.DeleteLocalOrphanByPath(ctx, path); err != nil {
t.Fatalf("DeleteLocalOrphanByPath: %v", err)
}
var remaining int
if err := pool.QueryRow(ctx,
`SELECT count(*) FROM download_artifact_local_orphans WHERE output_path = $1`, path,
).Scan(&remaining); err != nil {
t.Fatalf("count local orphans: %v", err)
}
if remaining != 0 {
t.Fatalf("local orphan rows after delete = %d, want 0", remaining)
}
}
func localOrphansForPath(orphans []LocalArtifactOrphan, path string) []LocalArtifactOrphan {
filtered := make([]LocalArtifactOrphan, 0, 1)
for _, orphan := range orphans {
if orphan.OutputPath == path {
filtered = append(filtered, orphan)
}
}
return filtered
}
func markArtifactReady(t *testing.T, pool *pgxpool.Pool, id string) {
t.Helper()
if _, err := pool.Exec(context.Background(),
`UPDATE download_artifacts SET status = 'ready', file_size = 1024, completed_at = now() WHERE id = $1`,
id,
); err != nil {
t.Fatalf("mark artifact ready: %v", err)
}
}
func seedDownloadUser(t *testing.T, pool *pgxpool.Pool) int {
t.Helper()
var userID int
if err := pool.QueryRow(context.Background(),
`INSERT INTO users (username, role, download_allowed) VALUES ($1, 'user', true) RETURNING id`,
fmt.Sprintf("evictuser-%d", time.Now().UnixNano()),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), `DELETE FROM downloads WHERE user_id = $1`, userID)
_, _ = pool.Exec(context.Background(), `DELETE FROM users WHERE id = $1`, userID)
})
return userID
}
// TestDeleteReadyIfEvictableFencesConcurrentLink pins the LRU eviction
// invariant: a ready download that lands after the unlinked check must keep
// the artifact row, and a download insert against a deleted id must fail
// rather than advertise a 404-forever handle.
func TestDeleteReadyIfEvictableFencesConcurrentLink(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
art := newArtifact(t, fileID, fmt.Sprintf("hash-evict-%d", time.Now().UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("ensure artifact: %v", err)
}
markArtifactReady(t, pool, art.ID)
userID := seedDownloadUser(t, pool)
dlRepo := NewRepository(pool)
now := time.Now()
contentID := fmt.Sprintf("evict-content-%d", now.UnixNano())
deleted, err := repo.DeleteReadyIfEvictable(ctx, art.ID)
if err != nil {
t.Fatalf("DeleteReadyIfEvictable unlinked: %v", err)
}
if !deleted {
t.Fatal("unlinked ready artifact must be evictable")
}
if err := dlRepo.Create(ctx, &Download{
ID: fmt.Sprintf("dl-evict-miss-%d", now.UnixNano()), UserID: userID, MediaFileID: fileID, ContentID: contentID,
Kind: KindQueued, Status: StatusReady, Format: FormatTranscode,
ArtifactID: art.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); !errors.Is(err, ErrArtifactEvicted) {
t.Fatalf("Create after eviction = %v, want ErrArtifactEvicted", err)
}
art2 := newArtifact(t, fileID, fmt.Sprintf("hash-evict-live-%d", time.Now().UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, art2); err != nil {
t.Fatalf("ensure second artifact: %v", err)
}
markArtifactReady(t, pool, art2.ID)
if err := dlRepo.Create(ctx, &Download{
ID: fmt.Sprintf("dl-evict-live-%d", now.UnixNano()), UserID: userID, MediaFileID: fileID, ContentID: contentID + "-live",
Kind: KindQueued, Status: StatusReady, Format: FormatTranscode,
ArtifactID: art2.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("Create live download: %v", err)
}
deleted, err = repo.DeleteReadyIfEvictable(ctx, art2.ID)
if err != nil {
t.Fatalf("DeleteReadyIfEvictable linked: %v", err)
}
if deleted {
t.Fatal("ready download must fence LRU eviction")
}
if _, err := repo.GetByID(ctx, art2.ID); err != nil {
t.Fatalf("linked artifact missing after eviction attempt: %v", err)
}
// Terminal transitions drop the link, as CancelByID and the fail paths do.
if _, err := pool.Exec(ctx, `UPDATE downloads SET status = 'cancelled', artifact_id = NULL WHERE artifact_id = $1`, art2.ID); err != nil {
t.Fatalf("cancel download: %v", err)
}
deleted, err = repo.DeleteReadyIfEvictable(ctx, art2.ID)
if err != nil {
t.Fatalf("DeleteReadyIfEvictable after cancel: %v", err)
}
if !deleted {
t.Fatal("terminal-only links must not fence eviction")
}
}
func TestCreateOriginalDownloadDoesNotRequireArtifact(t *testing.T) {
_, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
userID := seedDownloadUser(t, pool)
now := time.Now()
if err := NewRepository(pool).Create(ctx, &Download{
ID: fmt.Sprintf("dl-original-%d", now.UnixNano()), UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("original-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusQueued, Format: FormatOriginal,
FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("Create original download: %v", err)
}
}
// TestCreateAgainstMissingArtifactIsEvicted pins the enforcement layer itself:
// a download that names an artifact id with no row must be rejected as evicted,
// whichever mechanism catches it (the EXISTS fast path or the foreign key).
func TestCreateAgainstMissingArtifactIsEvicted(t *testing.T) {
_, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
userID := seedDownloadUser(t, pool)
now := time.Now()
missingID, err := idgen.NextID()
if err != nil {
t.Fatalf("id: %v", err)
}
d := &Download{
ID: fmt.Sprintf("dl-missing-%d", now.UnixNano()), UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("missing-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusReady, Format: FormatTranscode,
ArtifactID: missingID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}
if err := NewRepository(pool).Create(ctx, d); !errors.Is(err, ErrArtifactEvicted) {
t.Fatalf("Create against missing artifact = %v, want ErrArtifactEvicted", err)
}
if err := NewRepository(pool).CreateBatch(ctx, []*Download{d}); !errors.Is(err, ErrArtifactEvicted) {
t.Fatalf("CreateBatch against missing artifact = %v, want ErrArtifactEvicted", err)
}
}
// TestTerminalTransitionsUnpinArtifact pins the ON DELETE RESTRICT contract:
// canceling or failing a download must drop its artifact link, otherwise the
// artifact stays pinned against eviction forever.
func TestTerminalTransitionsUnpinArtifact(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
dlRepo := NewRepository(pool)
userID := seedDownloadUser(t, pool)
now := time.Now()
cancelArt := newArtifact(t, fileID, fmt.Sprintf("hash-cancel-%d", now.UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, cancelArt); err != nil {
t.Fatalf("ensure cancel artifact: %v", err)
}
markArtifactReady(t, pool, cancelArt.ID)
cancelID := fmt.Sprintf("dl-cancel-%d", now.UnixNano())
if err := dlRepo.Create(ctx, &Download{
ID: cancelID, UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("cancel-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusQueued, Format: FormatTranscode,
ArtifactID: cancelArt.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create cancelable download: %v", err)
}
if err := dlRepo.CancelByID(ctx, cancelID, userID); err != nil {
t.Fatalf("CancelByID: %v", err)
}
canceled, err := dlRepo.GetByID(ctx, cancelID)
if err != nil {
t.Fatalf("get canceled download: %v", err)
}
if canceled.ArtifactID != "" {
t.Fatalf("canceled download artifact_id = %q, want empty", canceled.ArtifactID)
}
deleted, err := repo.DeleteReadyIfEvictable(ctx, cancelArt.ID)
if err != nil {
t.Fatalf("DeleteReadyIfEvictable after cancel: %v", err)
}
if !deleted {
t.Fatal("a canceled download must not pin its artifact")
}
failArt := newArtifact(t, fileID, fmt.Sprintf("hash-fail-%d", now.UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, failArt); err != nil {
t.Fatalf("ensure fail artifact: %v", err)
}
failID := fmt.Sprintf("dl-fail-%d", now.UnixNano())
if err := dlRepo.Create(ctx, &Download{
ID: failID, UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("fail-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusPreparing, Format: FormatTranscode,
ArtifactID: failArt.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create preparing download: %v", err)
}
flipped, err := dlRepo.MarkLinkedDownloadsFailed(ctx, failArt.ID, "encode failed")
if err != nil {
t.Fatalf("MarkLinkedDownloadsFailed: %v", err)
}
if len(flipped) != 1 {
t.Fatalf("flipped rows = %d, want 1", len(flipped))
}
if flipped[0].ArtifactID != "" {
t.Fatalf("failed download artifact_id = %q, want empty", flipped[0].ArtifactID)
}
if _, err := pool.Exec(ctx, `UPDATE download_artifacts SET status = 'failed' WHERE id = $1`, failArt.ID); err != nil {
t.Fatalf("fail artifact: %v", err)
}
deleted, err = repo.DeleteFailedIfEvictable(ctx, failArt.ID)
if err != nil {
t.Fatalf("DeleteFailedIfEvictable after fail: %v", err)
}
if !deleted {
t.Fatal("a failed download must not pin its artifact")
}
}
// TestReconcileFailedDownloadsUnpinArtifact covers the reconcile-to-failed
// update: it must both preserve the artifact's error message (read from the OLD
// row) and clear the link in the same statement.
func TestReconcileFailedDownloadsUnpinArtifact(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
dlRepo := NewRepository(pool)
userID := seedDownloadUser(t, pool)
now := time.Now()
art := newArtifact(t, fileID, fmt.Sprintf("hash-reconcile-%d", now.UnixNano()))
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("ensure artifact: %v", err)
}
if _, err := pool.Exec(ctx,
`UPDATE download_artifacts SET status = 'failed', error_message = 'ffmpeg exploded' WHERE id = $1`, art.ID,
); err != nil {
t.Fatalf("mark artifact failed: %v", err)
}
dlID := fmt.Sprintf("dl-reconcile-%d", now.UnixNano())
if err := dlRepo.Create(ctx, &Download{
ID: dlID, UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("reconcile-content-%d", now.UnixNano()),
Kind: KindQueued, Status: StatusPreparing, Format: FormatTranscode,
ArtifactID: art.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("create preparing download: %v", err)
}
if _, _, err := dlRepo.ReconcileLinkedDownloads(ctx); err != nil {
t.Fatalf("ReconcileLinkedDownloads: %v", err)
}
row, err := dlRepo.GetByID(ctx, dlID)
if err != nil {
t.Fatalf("get reconciled download: %v", err)
}
if row.Status != StatusFailed {
t.Fatalf("reconciled status = %q, want failed", row.Status)
}
if row.ErrorMessage != "ffmpeg exploded" {
t.Fatalf("reconciled error_message = %q, want the artifact's message", row.ErrorMessage)
}
if row.ArtifactID != "" {
t.Fatalf("reconciled artifact_id = %q, want empty", row.ArtifactID)
}
}
// TestEvictionVersusLinkRace is the invariant test the fence alone could not
// satisfy: many rounds of a genuine two-goroutine race between linking an
// artifact and evicting it, asserting after every round that no download row
// points at a missing artifact and that exactly one side lost.
func TestEvictionVersusLinkRace(t *testing.T) {
repo, pool, fileID := newArtifactTestRepo(t)
ctx := context.Background()
dlRepo := NewRepository(pool)
userID := seedDownloadUser(t, pool)
const rounds = 100
for i := 0; i < rounds; i++ {
art := newArtifact(t, fileID, fmt.Sprintf("hash-race-%d-%d", time.Now().UnixNano(), i))
if _, _, err := repo.EnsureQueued(ctx, art); err != nil {
t.Fatalf("round %d: ensure artifact: %v", i, err)
}
markArtifactReady(t, pool, art.ID)
now := time.Now()
d := &Download{
ID: fmt.Sprintf("dl-race-%d-%d", now.UnixNano(), i), UserID: userID, MediaFileID: fileID,
ContentID: fmt.Sprintf("race-content-%d-%d", now.UnixNano(), i),
Kind: KindQueued, Status: StatusReady, Format: FormatTranscode,
ArtifactID: art.ID, FileSize: 1024, CreatedAt: now, UpdatedAt: now,
}
start := make(chan struct{})
var wg sync.WaitGroup
var createErr, deleteErr error
var evicted bool
wg.Add(2)
go func() {
defer wg.Done()
<-start
createErr = dlRepo.Create(ctx, d)
}()
go func() {
defer wg.Done()
<-start
evicted, deleteErr = repo.DeleteReadyIfEvictable(ctx, art.ID)
}()
close(start)
wg.Wait()
if deleteErr != nil {
t.Fatalf("round %d: DeleteReadyIfEvictable: %v", i, deleteErr)
}
if createErr != nil && !errors.Is(createErr, ErrArtifactEvicted) {
t.Fatalf("round %d: Create: %v", i, createErr)
}
// The invariant: never a download row whose artifact row is gone.
var dangling int
if err := pool.QueryRow(ctx,
`SELECT count(*) FROM downloads d
WHERE d.artifact_id IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM download_artifacts a WHERE a.id = d.artifact_id)`,
).Scan(&dangling); err != nil {
t.Fatalf("round %d: dangling check: %v", i, err)
}
if dangling != 0 {
t.Fatalf("round %d: %d download rows point at a deleted artifact", i, dangling)
}
// And the eviction direction: exactly one side may win.
linked := createErr == nil
if linked && evicted {
t.Fatalf("round %d: both the link and the eviction succeeded", i)
}
if linked {
if _, err := repo.GetByID(ctx, art.ID); err != nil {
t.Fatalf("round %d: linked artifact missing: %v", i, err)
}
}
if _, err := pool.Exec(ctx, `DELETE FROM downloads WHERE id = $1`, d.ID); err != nil {
t.Fatalf("round %d: cleanup download: %v", i, err)
}
if _, err := pool.Exec(ctx, `DELETE FROM download_artifacts WHERE id = $1`, art.ID); err != nil {
t.Fatalf("round %d: cleanup artifact: %v", i, err)
}
}
}
+11 -2
View File
@@ -39,8 +39,8 @@ func TestParamsHashStableAndDistinct(t *testing.T) {
}
func TestArtifactOutputPathDeterministic(t *testing.T) {
p1 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef")
p2 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef")
p1 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef", "art-1")
p2 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef", "art-1")
if p1 != p2 {
t.Fatalf("output path not deterministic: %q != %q", p1, p2)
}
@@ -50,6 +50,15 @@ func TestArtifactOutputPathDeterministic(t *testing.T) {
if !strings.Contains(p1, "42_transcode_") {
t.Fatalf("output path missing identity components: %q", p1)
}
// A replacement row for the same dedup key must NOT reuse the evicted row's
// path, or its bytes could be unlinked by the predecessor's cleanup.
p3 := artifactOutputPath("/var/artifacts", 42, "transcode", "abcdef0123456789deadbeef", "art-2")
if p3 == p1 {
t.Fatalf("distinct artifact ids share an output path: %q", p3)
}
if !strings.Contains(p3, "art-2") {
t.Fatalf("output path missing artifact id: %q", p3)
}
}
type lifecycleTestPreparer struct {
+157 -24
View File
@@ -243,7 +243,7 @@ func (m *ArtifactManager) Ensure(ctx context.Context, file *models.MediaFile, fo
Resolution: target.Resolution,
AudioTrackIndex: target.AudioTrackIndex,
TargetBitrateKbps: target.TargetBitrateKbps,
OutputPath: artifactOutputPath(m.artifactDir(), file.ID, format, hash),
OutputPath: artifactOutputPath(m.artifactDir(), file.ID, format, hash, id),
MaxAttempts: artifactMaxAttempts,
}
row, created, err := m.repo.EnsureQueued(ctx, a)
@@ -793,6 +793,7 @@ const (
// all terminal are evictable.
func (m *ArtifactManager) Cleanup(ctx context.Context) error {
m.cleanupRemoteOrphans(ctx)
m.cleanupLocalOrphans(ctx)
m.sweepStale(ctx)
budget := m.downloadConfig().ArtifactMaxBytes
if budget <= 0 {
@@ -813,27 +814,120 @@ func (m *ArtifactManager) Cleanup(ctx context.Context) error {
if total <= budget {
break
}
active, err := m.repo.HasActiveLink(ctx, a.ID)
// Record the byte-cleanup intent BEFORE deleting the row: once the row
// is gone nothing else remembers the file, so a crash or unlink failure
// after the delete would leak it. A failed enqueue skips the candidate
// entirely — the row survives and the next pass retries.
if !m.enqueueEvictionCleanup(ctx, a) {
continue
}
// Delete the row first, gated on the active-link predicate. Checking,
// then deleting bytes, then deleting the row lets a concurrent
// Ensure+Create land a ready download on an id that this pass then
// removes — the download stays advertised and never heals. The fenced
// delete plus downloads_artifact_id_fkey closes that window.
deleted, err := m.repo.DeleteReadyIfEvictable(ctx, a.ID)
if err != nil {
slog.WarnContext(ctx, "artifact link check failed", "component", "downloads", "artifact_id", a.ID, "error", err)
slog.WarnContext(ctx, "artifact eviction fence failed", "component", "downloads", "artifact_id", a.ID, "error", err)
continue
}
if active {
if !deleted {
continue
}
if !m.deleteArtifactBytes(ctx, a) {
continue
}
if err := m.repo.DeleteArtifact(ctx, a.ID); err != nil {
slog.WarnContext(ctx, "deleting evicted artifact row failed", "component", "downloads", "artifact_id", a.ID, "error", err)
continue
}
slog.InfoContext(ctx, "evicted download artifact (LRU)", "component", "downloads", "artifact_id", a.ID, "bytes", a.FileSize)
// The bytes are now guaranteed to be reclaimed — inline here, or by the
// orphan processor from the intent row — so the budget can be credited
// as soon as the row is gone.
freedInline := m.deleteEvictedArtifactBytes(ctx, a)
total -= a.FileSize
slog.InfoContext(ctx, "evicted download artifact (LRU)", "component", "downloads",
"artifact_id", a.ID, "bytes", a.FileSize, "bytes_freed_inline", freedInline)
}
return nil
}
// enqueueEvictionCleanup durably records how an artifact's bytes are to be
// removed once its row is deleted. Returns false when the intent could not be
// persisted, in which case the caller must leave the row in place.
func (m *ArtifactManager) enqueueEvictionCleanup(ctx context.Context, a *Artifact) bool {
if a.OriginArtifactID != "" {
if err := m.repo.EnqueueRemoteOrphan(ctx, a.ID, a.OriginNodeID, a.OriginNodeURL, a.OriginArtifactID); err != nil {
slog.WarnContext(ctx, "recording remote artifact cleanup intent failed", "component", "downloads", "artifact_id", a.ID, "node", a.OriginNodeURL, "error", err)
return false
}
return true
}
if a.OutputPath == "" {
return true
}
if err := m.repo.EnqueueLocalOrphan(ctx, a.ID, a.OutputPath); err != nil {
slog.WarnContext(ctx, "recording local artifact cleanup intent failed", "component", "downloads", "artifact_id", a.ID, "error", err)
return false
}
return true
}
// cleanupLocalOrphans retries filesystem deletions recorded before their
// artifact row was removed. Each candidate is verified against the artifact
// table first: paths carry the artifact id, so no successor row can ever claim
// the same path — a row that still exists means the eviction never happened and
// the file is live.
func (m *ArtifactManager) cleanupLocalOrphans(ctx context.Context) {
budget := m.remoteCleanupBudget
if budget <= 0 {
budget = defaultRemoteCleanupBudget
}
cleanupCtx, cancel := context.WithTimeout(ctx, budget)
defer cancel()
orphans, err := m.repo.ListLocalOrphansDue(cleanupCtx, 100)
if err != nil {
slog.WarnContext(ctx, "listing local artifact cleanup queue failed", "component", "downloads", "error", err)
return
}
for _, orphan := range orphans {
if cleanupCtx.Err() != nil {
return
}
_, err := m.repo.GetByID(cleanupCtx, orphan.DownloadArtifactID)
switch {
case err == nil:
// The row survived (eviction refused): the file is still owned.
if derr := m.repo.DeleteLocalOrphan(cleanupCtx, orphan.ID); derr != nil {
slog.WarnContext(ctx, "clearing local artifact cleanup row failed", "component", "downloads", "artifact_id", orphan.DownloadArtifactID, "error", derr)
}
continue
case !errors.Is(err, ErrNotFound):
slog.WarnContext(ctx, "verifying local artifact cleanup candidate failed", "component", "downloads", "artifact_id", orphan.DownloadArtifactID, "error", err)
continue
}
if !removeLocalArtifactFiles(ctx, orphan.DownloadArtifactID, orphan.OutputPath) {
if berr := m.repo.BumpLocalOrphanRetry(cleanupCtx, orphan.ID); berr != nil {
slog.WarnContext(ctx, "deferring local artifact cleanup row failed", "component", "downloads", "artifact_id", orphan.DownloadArtifactID, "error", berr)
}
continue
}
if derr := m.repo.DeleteLocalOrphan(cleanupCtx, orphan.ID); derr != nil {
slog.WarnContext(ctx, "clearing local artifact cleanup row failed", "component", "downloads", "artifact_id", orphan.DownloadArtifactID, "error", derr)
}
}
}
// removeLocalArtifactFiles unlinks an artifact's output file and its .part
// leftover, tolerating an already-missing file.
func removeLocalArtifactFiles(ctx context.Context, artifactID, outputPath string) bool {
if outputPath == "" {
return true
}
if err := os.Remove(outputPath); err != nil && !os.IsNotExist(err) {
slog.WarnContext(ctx, "removing artifact file failed", "component", "downloads", "artifact_id", artifactID, "error", err)
return false
}
if err := os.Remove(outputPath + ".part"); err != nil && !os.IsNotExist(err) {
slog.WarnContext(ctx, "removing artifact partial failed", "component", "downloads", "artifact_id", artifactID, "error", err)
return false
}
return true
}
func (m *ArtifactManager) cleanupRemoteOrphans(ctx context.Context) {
lifecycle, ok := m.preparer.(remoteArtifactLifecycle)
if !ok {
@@ -931,14 +1025,40 @@ func (m *ArtifactManager) sweepStale(ctx context.Context) {
// removeArtifact deletes an artifact's output file, its .part leftover, and
// its row. Used by the hygiene sweep for rows nothing can serve again.
func (m *ArtifactManager) removeArtifact(ctx context.Context, a *Artifact, reason string) {
if !m.deleteArtifactBytes(ctx, a) {
// Both branches record the byte-cleanup intent before removing the row, so
// a failure after the delete defers to the orphan processor rather than
// leaking the file.
if !m.enqueueEvictionCleanup(ctx, a) {
return
}
if err := m.repo.DeleteArtifact(ctx, a.ID); err != nil {
slog.WarnContext(ctx, "deleting swept artifact row failed", "component", "downloads", "artifact_id", a.ID, "error", err)
// A non-ready candidate came from ListFailedBefore, but it is not
// necessarily still dead: Ensure requeues a failed row and the drain can
// claim it as running before any download links it. Both branches therefore
// pin the row's status in the delete predicate and go through the
// active-link fence (plus the FK).
ready := a.Status == ArtifactReady
var deleted bool
var err error
if ready {
deleted, err = m.repo.DeleteReadyIfEvictable(ctx, a.ID)
} else {
deleted, err = m.repo.DeleteFailedIfEvictable(ctx, a.ID)
}
if err != nil {
if ready {
slog.WarnContext(ctx, "evicting swept ready artifact row failed", "component", "downloads", "artifact_id", a.ID, "error", err)
} else {
slog.WarnContext(ctx, "deleting swept artifact row failed", "component", "downloads", "artifact_id", a.ID, "error", err)
}
return
}
slog.InfoContext(ctx, "swept stale download artifact", "component", "downloads", "artifact_id", a.ID, "reason", reason, "bytes", a.FileSize)
if !deleted {
slog.InfoContext(ctx, "swept artifact retained; a download links it", "component", "downloads", "artifact_id", a.ID, "reason", reason)
return
}
freedInline := m.deleteEvictedArtifactBytes(ctx, a)
slog.InfoContext(ctx, "swept stale download artifact", "component", "downloads",
"artifact_id", a.ID, "reason", reason, "bytes", a.FileSize, "bytes_freed_inline", freedInline)
}
func (m *ArtifactManager) deleteArtifactBytes(ctx context.Context, a *Artifact) bool {
@@ -954,16 +1074,29 @@ func (m *ArtifactManager) deleteArtifactBytes(ctx context.Context, a *Artifact)
}
return true
}
if a.OutputPath == "" {
return removeLocalArtifactFiles(ctx, a.ID, a.OutputPath)
}
// deleteEvictedArtifactBytes removes bytes inline after the artifact row is
// already gone. A cleanup intent was persisted before the row delete, so this
// is only the fast path: on success the now-redundant intent row is cleared, on
// failure it is left for the orphan processor to retry. Paths carry the
// artifact id, so no replacement row can ever own this file.
func (m *ArtifactManager) deleteEvictedArtifactBytes(ctx context.Context, a *Artifact) bool {
if !m.deleteArtifactBytes(ctx, a) {
return false
}
if a.OriginArtifactID != "" {
// The remote queue dedupes on (origin_node_id, origin_artifact_id) and
// its processor re-checks ownership, so a stale row is harmless; clear
// it anyway to keep the queue short.
if err := m.repo.DeleteRemoteOrphanByLocator(ctx, a.OriginNodeID, a.OriginArtifactID); err != nil {
slog.WarnContext(ctx, "clearing remote artifact cleanup intent failed", "component", "downloads", "artifact_id", a.ID, "error", err)
}
return true
}
if err := os.Remove(a.OutputPath); err != nil && !os.IsNotExist(err) {
slog.WarnContext(ctx, "removing artifact file failed", "component", "downloads", "artifact_id", a.ID, "error", err)
return false
}
if err := os.Remove(a.OutputPath + ".part"); err != nil && !os.IsNotExist(err) {
slog.WarnContext(ctx, "removing artifact partial failed", "component", "downloads", "artifact_id", a.ID, "error", err)
return false
if err := m.repo.DeleteLocalOrphanByPath(ctx, a.OutputPath); err != nil {
slog.WarnContext(ctx, "clearing local artifact cleanup intent failed", "component", "downloads", "artifact_id", a.ID, "error", err)
}
return true
}
+3
View File
@@ -86,6 +86,9 @@ var (
// ErrArtifactOriginRemoved means a durable remote locator points at a node
// no longer present in the enabled transcode pool.
ErrArtifactOriginRemoved = errors.New("download artifact origin node was removed")
// ErrArtifactEvicted means a download tried to link to an artifact that LRU
// or hygiene eviction removed between Ensure and the row write.
ErrArtifactEvicted = errors.New("download artifact was evicted")
// ErrQualityUnavailable means the requested quality is valid and permitted in
// principle but cannot be fulfilled by the current server wiring.
ErrQualityUnavailable = errors.New("requested download quality is not available")
+109 -14
View File
@@ -9,6 +9,7 @@ import (
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
)
@@ -16,11 +17,67 @@ const downloadColumns = `id, user_id, profile_id, device_id, media_file_id, cont
kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at`
const insertDownloadSQL = `INSERT INTO downloads (id, user_id, profile_id, device_id, media_file_id, content_id,
episode_id, batch_id, kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at)
// downloadInsertColumns is the number of columns in downloadColumns; the insert
// placeholder lists and the batch builder both derive their arity from it.
const downloadInsertColumns = 22
const insertDownloadSQL = `INSERT INTO downloads (` + downloadColumns + `)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22)`
// insertDownloadRequiringArtifactSQL is a fast-path filter: it skips the insert
// when the artifact row is already gone, so the common eviction case surfaces as
// a clean zero-row result instead of a constraint error. It is NOT the
// enforcement mechanism — under READ COMMITTED its subquery reads an independent
// snapshot and takes no conflicting lock, so a concurrent eviction can still
// commit alongside it. downloads_artifact_id_fkey is what makes a dangling
// artifact_id impossible; a 23503 from it maps to ErrArtifactEvicted below.
// $16 is artifact_id.
const insertDownloadRequiringArtifactSQL = `INSERT INTO downloads (` + downloadColumns + `)
SELECT $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22
WHERE EXISTS (SELECT 1 FROM download_artifacts WHERE id = $16)`
// downloadsArtifactFKConstraint is the foreign key from downloads.artifact_id to
// download_artifacts.id. Violating it in either direction means the artifact was
// evicted concurrently.
const downloadsArtifactFKConstraint = "downloads_artifact_id_fkey"
// isForeignKeyViolation reports whether err is a Postgres foreign-key violation
// (SQLSTATE 23503) raised by the named constraint.
func isForeignKeyViolation(err error, constraint string) bool {
var pgErr *pgconn.PgError
if !errors.As(err, &pgErr) {
return false
}
return pgErr.Code == "23503" && pgErr.ConstraintName == constraint
}
// execer is satisfied by both *pgxpool.Pool and pgx.Tx, so the insert helper
// works inside and outside a transaction.
type execer interface {
Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error)
}
// execInsertDownload runs the right insert for d (fenced when it links an
// artifact) and maps both the zero-row fast path and the FK violation to
// ErrArtifactEvicted.
func (r *Repository) execInsertDownload(ctx context.Context, q execer, d *Download) error {
query := insertDownloadSQL
if d.ArtifactID != "" {
query = insertDownloadRequiringArtifactSQL
}
tag, err := q.Exec(ctx, query, r.insertArgs(d)...)
if err != nil {
if isForeignKeyViolation(err, downloadsArtifactFKConstraint) {
return ErrArtifactEvicted
}
return err
}
if d.ArtifactID != "" && tag.RowsAffected() == 0 {
return ErrArtifactEvicted
}
return nil
}
// Repository provides CRUD operations for the downloads table.
type Repository struct {
pool *pgxpool.Pool
@@ -122,9 +179,14 @@ func (r *Repository) insertArgs(d *Download) []any {
}
}
// Create inserts a new download record.
// Create inserts a new download record. Artifact-backed rows are inserted only
// while the artifact row still exists, so LRU eviction cannot leave a ready
// download pointing at a deleted id.
func (r *Repository) Create(ctx context.Context, d *Download) error {
if _, err := r.pool.Exec(ctx, insertDownloadSQL, r.insertArgs(d)...); err != nil {
if err := r.execInsertDownload(ctx, r.pool, d); err != nil {
if errors.Is(err, ErrArtifactEvicted) {
return err
}
return fmt.Errorf("inserting download: %w", err)
}
return nil
@@ -139,7 +201,10 @@ func (r *Repository) CreateBatch(ctx context.Context, downloads []*Download) err
defer func() { _ = tx.Rollback(ctx) }()
for _, d := range downloads {
if _, err := tx.Exec(ctx, insertDownloadSQL, r.insertArgs(d)...); err != nil {
if err := r.execInsertDownload(ctx, tx, d); err != nil {
if errors.Is(err, ErrArtifactEvicted) {
return err
}
return fmt.Errorf("inserting batch download: %w", err)
}
}
@@ -248,9 +313,11 @@ func (r *Repository) Delete(ctx context.Context, id string, userID int) error {
}
// CancelByID sets a download to canceled if it is still queued or downloading.
// The artifact link is dropped with the transition: a terminal row must not pin
// an artifact against eviction (downloads_artifact_id_fkey is ON DELETE RESTRICT).
func (r *Repository) CancelByID(ctx context.Context, id string, userID int) error {
tag, err := r.pool.Exec(ctx,
`UPDATE downloads SET status = 'cancelled', updated_at = now()
`UPDATE downloads SET status = 'cancelled', artifact_id = NULL, updated_at = now()
WHERE id = $1 AND user_id = $2 AND status IN ('queued', 'downloading')`,
id, userID,
)
@@ -320,6 +387,11 @@ func (r *Repository) GetManagedEntry(ctx context.Context, userID int, profileID,
// Callers have already classified the item as new.
func (r *Repository) CreateManagedEntry(ctx context.Context, d *Download) (*Download, error) {
if err := r.Create(ctx, d); err != nil {
// An evicted artifact is retryable by the caller (re-Ensure, then
// re-insert); returning some unrelated existing row would hide it.
if errors.Is(err, ErrArtifactEvicted) {
return nil, err
}
if existing, gerr := r.GetManagedEntry(ctx, d.UserID, d.ProfileID, d.DeviceID, d.ContentID, d.EpisodeID); gerr == nil {
return existing, nil
}
@@ -385,11 +457,9 @@ func (r *Repository) CreateManagedEntriesBatch(ctx context.Context, ds []*Downlo
if len(ds) == 0 {
return nil, nil
}
const insertCols = 22
const insertCols = downloadInsertColumns
var sb strings.Builder
sb.WriteString(`INSERT INTO downloads (id, user_id, profile_id, device_id, media_file_id, content_id,
episode_id, batch_id, kind, status, format, quality, effective_quality, target_bitrate_kbps, revision, artifact_id, file_size, bytes_sent, error_message,
created_at, updated_at, completed_at) VALUES `)
sb.WriteString(`INSERT INTO downloads (` + downloadColumns + `) VALUES `)
args := make([]any, 0, len(ds)*insertCols)
for i, d := range ds {
if i > 0 {
@@ -409,14 +479,31 @@ func (r *Repository) CreateManagedEntriesBatch(ctx context.Context, ds []*Downlo
sb.WriteString(` ON CONFLICT (user_id, profile_id, device_id, content_id, (COALESCE(episode_id, ''))) WHERE device_id IS NOT NULL DO NOTHING RETURNING ` + downloadColumns)
rows, err := r.pool.Query(ctx, sb.String(), args...)
if err != nil {
// Managed originals carry no artifact today, so this is future-proofing:
// any artifact-backed row in the batch gets the same retryable error the
// single-row path returns.
if isForeignKeyViolation(err, downloadsArtifactFKConstraint) {
return nil, ErrArtifactEvicted
}
return nil, fmt.Errorf("batch inserting managed entries: %w", err)
}
defer rows.Close()
return scanDownloads(rows)
inserted, err := scanDownloads(rows)
if err != nil {
if isForeignKeyViolation(err, downloadsArtifactFKConstraint) {
return nil, ErrArtifactEvicted
}
return nil, err
}
return inserted, nil
}
// ReplaceManagedEntry updates an existing managed row to a new file/quality
// target, incrementing its revision so clients can replace stale local bytes.
// An artifact evicted concurrently surfaces as downloads_artifact_id_fkey
// (23503) and maps to ErrArtifactEvicted, so a zero-row result means only that
// the identity/revision fence rejected the write — another writer won the row,
// and the caller gets that winner back.
func (r *Repository) ReplaceManagedEntry(ctx context.Context, existing *Download, replacement *Download) (*Download, error) {
query := `UPDATE downloads SET
media_file_id = $6,
@@ -447,6 +534,9 @@ func (r *Repository) ReplaceManagedEntry(ctx context.Context, existing *Download
if errors.Is(err, ErrNotFound) {
return r.GetManagedEntry(ctx, existing.UserID, existing.ProfileID, existing.DeviceID, existing.ContentID, existing.EpisodeID)
}
if isForeignKeyViolation(err, downloadsArtifactFKConstraint) {
return nil, ErrArtifactEvicted
}
return nil, fmt.Errorf("replacing managed download: %w", err)
}
return d, nil
@@ -610,10 +700,12 @@ func (r *Repository) MarkLinkedDownloadsReady(ctx context.Context, artifactID st
}
// MarkLinkedDownloadsFailed flips every preparing download linked to a failed
// artifact to failed, returning the affected rows for client notification.
// artifact to failed, returning the affected rows for client notification. The
// artifact link is dropped with the transition so the now-terminal row does not
// pin the artifact against eviction (the FK is ON DELETE RESTRICT).
func (r *Repository) MarkLinkedDownloadsFailed(ctx context.Context, artifactID, errMsg string) ([]*Download, error) {
rows, err := r.pool.Query(ctx,
`UPDATE downloads SET status = 'failed', error_message = $2, updated_at = now()
`UPDATE downloads SET status = 'failed', artifact_id = NULL, error_message = $2, updated_at = now()
WHERE artifact_id = $1 AND status = 'preparing'
RETURNING `+downloadColumns,
artifactID, errMsg,
@@ -653,8 +745,11 @@ func (r *Repository) ReconcileLinkedDownloads(ctx context.Context) (ready []*Dow
}
failedRows, err := r.pool.Query(ctx,
// Every SET expression reads the OLD row, so error_message still resolves
// against the pre-update artifact_id while the same statement clears it.
`UPDATE downloads SET status = 'failed',
error_message = COALESCE((SELECT NULLIF(a.error_message, '') FROM download_artifacts a WHERE a.id = downloads.artifact_id), 'artifact encode failed'),
artifact_id = NULL,
updated_at = now()
WHERE status = 'preparing' AND artifact_id IS NOT NULL
AND artifact_id IN (SELECT id FROM download_artifacts WHERE status = 'failed')
+81
View File
@@ -276,6 +276,10 @@ func TestReconcileLinkedDownloads(t *testing.T) {
}
arepo := NewArtifactRepository(f.pool)
t.Cleanup(func() {
// Cleanups run LIFO, so this fires before the fixture's downloads
// delete; unpin the links first or downloads_artifact_id_fkey
// (ON DELETE RESTRICT) refuses the artifact delete.
_, _ = f.pool.Exec(ctx, `UPDATE downloads SET artifact_id = NULL WHERE media_file_id = $1`, f.fileID)
_, _ = f.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id = $1`, f.fileID)
})
@@ -501,3 +505,80 @@ func TestRegisterManagedItemsBatchAndCount(t *testing.T) {
t.Fatalf("grown register = %d rows, want 2", len(grown))
}
}
// TestReplaceManagedEntryRevisionConflictReturnsWinner pins the simplified
// outcome contract: with the artifact foreign key doing the enforcement, a
// zero-row UPDATE can only mean the identity/revision fence rejected the write,
// so the caller gets the winning row rather than an eviction error.
func TestReplaceManagedEntryRevisionConflictReturnsWinner(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
id := f.createManagedEntry(t)
existing, err := f.repo.GetByID(ctx, id)
if err != nil {
t.Fatalf("get managed entry: %v", err)
}
// A concurrent writer bumps the revision out from under the snapshot.
if _, err := f.pool.Exec(ctx, `UPDATE downloads SET revision = revision + 1 WHERE id = $1`, id); err != nil {
t.Fatalf("bump revision: %v", err)
}
replacement := &Download{
UserID: f.userID, ProfileID: f.profileA, DeviceID: f.deviceA,
MediaFileID: f.fileID, ContentID: f.contentID, Kind: KindQueued,
Status: StatusReady, Format: FormatOriginal, Quality: QualityOriginal,
EffectiveQuality: QualityOriginal, FileSize: 2048,
}
row, err := f.repo.ReplaceManagedEntry(ctx, existing, replacement)
if err != nil {
t.Fatalf("ReplaceManagedEntry on revision conflict: %v", err)
}
if row == nil || row.ID != id {
t.Fatalf("ReplaceManagedEntry returned %+v, want the winning row %q", row, id)
}
if row.Revision != existing.Revision+1 {
t.Fatalf("winning revision = %d, want %d (the concurrent writer's)", row.Revision, existing.Revision+1)
}
}
// TestReplaceManagedEntryEvictedArtifact pins the other half: an artifact
// evicted between Ensure and the replace surfaces as ErrArtifactEvicted (via
// downloads_artifact_id_fkey), never as a silent no-op.
func TestReplaceManagedEntryEvictedArtifact(t *testing.T) {
f := seedManagedFixture(t)
ctx := context.Background()
var present *string
if err := f.pool.QueryRow(ctx, `SELECT to_regclass('public.download_artifacts')::text`).Scan(&present); err != nil {
t.Fatalf("check download_artifacts: %v", err)
}
if present == nil {
t.Skip("download_artifacts migration has not been applied")
}
var fkPresent bool
if err := f.pool.QueryRow(ctx,
`SELECT EXISTS(SELECT 1 FROM pg_constraint WHERE conname = 'downloads_artifact_id_fkey')`,
).Scan(&fkPresent); err != nil {
t.Fatalf("check artifact fk: %v", err)
}
if !fkPresent {
t.Skip("downloads artifact foreign key migration has not been applied")
}
id := f.createManagedEntry(t)
existing, err := f.repo.GetByID(ctx, id)
if err != nil {
t.Fatalf("get managed entry: %v", err)
}
missingArtifactID := fmt.Sprintf("gone-artifact-%d", time.Now().UnixNano())
replacement := &Download{
UserID: f.userID, ProfileID: f.profileA, DeviceID: f.deviceA,
MediaFileID: f.fileID, ContentID: f.contentID, Kind: KindQueued,
Status: StatusReady, Format: FormatTranscode, Quality: QualityOriginal,
EffectiveQuality: QualityOriginal, ArtifactID: missingArtifactID, FileSize: 2048,
}
if _, err := f.repo.ReplaceManagedEntry(ctx, existing, replacement); !errors.Is(err, ErrArtifactEvicted) {
t.Fatalf("ReplaceManagedEntry with missing artifact = %v, want ErrArtifactEvicted", err)
}
}
+84 -33
View File
@@ -427,13 +427,7 @@ func (s *Service) createArtifactDownload(ctx context.Context, userID int, req Cr
}
}
if existing != nil {
artifact, err := s.artifacts.Ensure(ctx, file, decision.DeliveryFormat, decision.PrepareTarget)
if err != nil {
return nil, err
}
status, size := artifactRowStatus(artifact, file)
replacement := buildManagedDownload(userID, req.ProfileID, req.DeviceID, managedItem{file: file, contentID: file.ContentID, episodeID: file.EpisodeID}, decision, "", status, size, artifact.ID)
return s.reuseOrReplaceManaged(ctx, existing, replacement)
return s.replaceManagedArtifactDownload(ctx, existing, userID, req, file, decision)
}
// New row: the quota lock serializes check + insert across concurrent
@@ -446,24 +440,85 @@ func (s *Service) createArtifactDownload(ctx context.Context, userID int, req Cr
if err := s.limiter.Check(ctx, userID, 1); err != nil {
return err
}
artifact, err := s.artifacts.Ensure(ctx, file, decision.DeliveryFormat, decision.PrepareTarget)
row, err := s.insertArtifactDownload(ctx, userID, req, file, decision, managed)
if err != nil {
return err
}
status, size := artifactRowStatus(artifact, file)
d = row
return nil
})
if err != nil {
return nil, err
}
return d, nil
}
if managed {
if err := s.repo.EnsureDevice(ctx, userID, req.ProfileID, req.DeviceID, req.DeviceName, req.DevicePlatform); err != nil {
return err
}
const artifactLinkAttempts = 2
// withEnsuredArtifact runs attempt against a freshly ensured artifact, retrying
// once when the artifact is evicted between Ensure and the linking write. Ensure
// must stay inside the loop: an evicted artifact has to be recreated before the
// next attempt can link anything.
func (s *Service) withEnsuredArtifact(
ctx context.Context,
file *models.MediaFile,
decision QualityDecision,
attempt func(artifact *Artifact, status string, size int64) (*Download, error),
) (*Download, error) {
var last error
for i := 0; i < artifactLinkAttempts; i++ {
artifact, err := s.artifacts.Ensure(ctx, file, decision.DeliveryFormat, decision.PrepareTarget)
if err != nil {
return nil, err
}
status, size := artifactRowStatus(artifact, file)
row, err := attempt(artifact, status, size)
if errors.Is(err, ErrArtifactEvicted) {
last = err
slog.DebugContext(ctx, "retrying download after artifact eviction", "component", "downloads", "artifact_id", artifact.ID, "attempt", i+1)
continue
}
return row, err
}
return nil, last
}
func (s *Service) replaceManagedArtifactDownload(ctx context.Context, existing *Download, userID int, req CreateRequest, file *models.MediaFile, decision QualityDecision) (*Download, error) {
return s.withEnsuredArtifact(ctx, file, decision, func(artifact *Artifact, status string, size int64) (*Download, error) {
row, err := s.replaceManagedWithArtifact(ctx, &existing, userID, req, file, decision, artifact, status, size)
return row, err
})
}
// replaceManagedWithArtifact points an existing managed entry at artifact. On
// eviction it refreshes the caller's *existing snapshot so the next attempt's
// revision fence can succeed against whatever the row looks like now.
func (s *Service) replaceManagedWithArtifact(ctx context.Context, existing **Download, userID int, req CreateRequest, file *models.MediaFile, decision QualityDecision, artifact *Artifact, status string, size int64) (*Download, error) {
replacement := buildManagedDownload(userID, req.ProfileID, req.DeviceID, managedItem{file: file, contentID: file.ContentID, episodeID: file.EpisodeID}, decision, "", status, size, artifact.ID)
row, err := s.reuseOrReplaceManaged(ctx, *existing, replacement)
if errors.Is(err, ErrArtifactEvicted) {
if fresh, gerr := s.repo.GetManagedEntry(ctx, userID, req.ProfileID, req.DeviceID, file.ContentID, file.EpisodeID); gerr == nil {
*existing = fresh
}
}
return row, err
}
func (s *Service) insertArtifactDownload(ctx context.Context, userID int, req CreateRequest, file *models.MediaFile, decision QualityDecision, managed bool) (*Download, error) {
// The device upsert is idempotent and independent of the artifact, so it
// runs once rather than on every retry.
if managed {
if err := s.repo.EnsureDevice(ctx, userID, req.ProfileID, req.DeviceID, req.DeviceName, req.DevicePlatform); err != nil {
return nil, err
}
}
return s.withEnsuredArtifact(ctx, file, decision, func(artifact *Artifact, status string, size int64) (*Download, error) {
id, err := idgen.NextID()
if err != nil {
return fmt.Errorf("generating download ID: %w", err)
return nil, fmt.Errorf("generating download ID: %w", err)
}
now := time.Now()
d = &Download{
d := &Download{
ID: id,
UserID: userID,
MediaFileID: file.ID,
@@ -485,26 +540,22 @@ func (s *Service) createArtifactDownload(ctx context.Context, userID int, req Cr
d.ProfileID = req.ProfileID
d.DeviceID = req.DeviceID
}
if err := s.repo.Create(ctx, d); err != nil {
if managed {
if existing, gerr := s.repo.GetManagedEntry(ctx, userID, req.ProfileID, req.DeviceID, file.ContentID, file.EpisodeID); gerr == nil {
replacement := buildManagedDownload(userID, req.ProfileID, req.DeviceID, managedItem{file: file, contentID: file.ContentID, episodeID: file.EpisodeID}, decision, "", status, size, artifact.ID)
row, rerr := s.reuseOrReplaceManaged(ctx, existing, replacement)
if rerr != nil {
return rerr
}
d = row
return nil
}
}
return err
err = s.repo.Create(ctx, d)
if err == nil {
return d, nil
}
if errors.Is(err, ErrArtifactEvicted) {
return nil, err
}
// A concurrent create won this managed identity: fall back to replacing
// the winning row, reusing the artifact already ensured this attempt.
if managed {
if existing, gerr := s.repo.GetManagedEntry(ctx, userID, req.ProfileID, req.DeviceID, file.ContentID, file.EpisodeID); gerr == nil {
return s.replaceManagedWithArtifact(ctx, &existing, userID, req, file, decision, artifact, status, size)
}
}
return nil
})
if err != nil {
return nil, err
}
return d, nil
})
}
// artifactRowStatus maps an ensured artifact to the download row status and
+5
View File
@@ -52,6 +52,10 @@ func TestArtifactQuotaCheckedBeforeEnqueue(t *testing.T) {
t.Fatalf("seed second media file: %v", err)
}
t.Cleanup(func() {
// Cleanups run LIFO, so this fires before the fixture's downloads
// delete; unpin the links first or downloads_artifact_id_fkey
// (ON DELETE RESTRICT) refuses the artifact delete.
_, _ = f.pool.Exec(ctx, `UPDATE downloads SET artifact_id = NULL WHERE media_file_id IN ($1, $2)`, f.fileID, fileID2)
_, _ = f.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id IN ($1, $2)`, f.fileID, fileID2)
_, _ = f.pool.Exec(ctx, `DELETE FROM media_files WHERE id = $1`, fileID2)
})
@@ -136,6 +140,7 @@ func TestConcurrentArtifactCreatesCannotBypassQuotaDB(t *testing.T) {
}
}
t.Cleanup(func() {
_, _ = f.pool.Exec(ctx, `UPDATE downloads SET artifact_id = NULL WHERE media_file_id = ANY($1)`, fileIDs)
_, _ = f.pool.Exec(ctx, `DELETE FROM download_artifacts WHERE media_file_id = ANY($1)`, fileIDs)
_, _ = f.pool.Exec(ctx, `DELETE FROM media_files WHERE id = ANY($1)`, fileIDs)
})
-50
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"strings"
"time"
)
type matcherRepository interface {
@@ -12,7 +11,6 @@ type matcherRepository interface {
MatchMediaByTitleYear(ctx context.Context, kind, title string, year int) ([]mediaLookupRow, error)
MatchEpisodeByExternalID(ctx context.Context, column, value string) ([]mediaLookupRow, error)
MatchEpisodeBySeries(ctx context.Context, seriesID string, seasonNumber, episodeNumber int) (*Match, error)
MatchEpisodesBySeries(ctx context.Context, seriesID string, seasonNumber *int, watchedAt *time.Time) ([]Match, error)
}
type Matcher struct {
@@ -36,54 +34,6 @@ func (m *Matcher) Match(ctx context.Context, record Record) (*Match, string, err
}
}
// MatchLeaves resolves a provider watched marker to concrete playable items.
// Movies and episodes have one leaf. Whole-show and whole-season markers must
// expand to the local episodes that existed when the provider marker was
// recorded; Silo derives series/season completion from those episode leaves.
func (m *Matcher) MatchLeaves(ctx context.Context, record Record) ([]Match, string, error) {
switch record.Kind {
case KindMovie, KindEpisode:
match, reason, err := m.Match(ctx, record)
if err != nil || match == nil {
return nil, reason, err
}
return []Match{*match}, "", nil
case KindSeries, KindSeason:
seriesRecord := record
var seasonNumber *int
if record.Kind == KindSeason {
seriesRecord = Record{
Kind: KindSeries,
Title: record.SeriesTitle,
Year: record.SeriesYear,
IMDbID: record.SeriesIMDbID,
TMDBID: record.SeriesTMDBID,
TVDBID: record.SeriesTVDBID,
}
season := record.SeasonNumber
seasonNumber = &season
}
seriesRecord.Kind = KindSeries
series, reason, err := m.matchSeries(ctx, seriesRecord)
if err != nil || series == nil {
return nil, reason, err
}
matches, err := m.repo.MatchEpisodesBySeries(ctx, series.MediaItemID, seasonNumber, record.LastPlayedAt)
if err != nil {
return nil, "", err
}
if len(matches) == 0 {
if seasonNumber != nil {
return nil, fmt.Sprintf("no available episodes matched for season %d", *seasonNumber), nil
}
return nil, "no available episodes matched for series", nil
}
return matches, "", nil
default:
return nil, "unsupported item kind", nil
}
}
func (m *Matcher) matchMovie(ctx context.Context, record Record) (*Match, string, error) {
match, reason, err := m.matchMedia(ctx, KindMovie, record.TMDBID, record.IMDbID, "", record.Title, record.Year, false)
if err != nil {
-69
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"testing"
"time"
)
type matcherRepoStub struct {
@@ -12,15 +11,11 @@ type matcherRepoStub struct {
mediaByExternal map[string][]mediaLookupRow
mediaByTitleYear map[string][]mediaLookupRow
episodeBySeries *Match
episodesBySeries []Match
episodeBySeriesCalls int
episodeBySeriesID string
episodeBySeriesSeason int
episodeBySeriesEpisode int
episodesBySeriesID string
episodesBySeriesSeason *int
episodesBySeriesAt *time.Time
mediaByExternalCalls int
}
@@ -45,70 +40,6 @@ func (s *matcherRepoStub) MatchEpisodeBySeries(_ context.Context, seriesID strin
return s.episodeBySeries, nil
}
func (s *matcherRepoStub) MatchEpisodesBySeries(_ context.Context, seriesID string, seasonNumber *int, watchedAt *time.Time) ([]Match, error) {
s.episodesBySeriesID = seriesID
if seasonNumber != nil {
season := *seasonNumber
s.episodesBySeriesSeason = &season
}
if watchedAt != nil {
at := *watchedAt
s.episodesBySeriesAt = &at
}
return s.episodesBySeries, nil
}
func TestMatcherMatchLeavesExpandsWholeShowToEpisodes(t *testing.T) {
t.Parallel()
watchedAt := time.Date(2025, time.October, 15, 20, 0, 0, 0, time.UTC)
repo := &matcherRepoStub{
mediaByExternal: map[string][]mediaLookupRow{
"series:tmdb_id:1396": {{ContentID: "series-1", Title: "Breaking Bad", Year: 2008}},
},
episodesBySeries: []Match{
{MediaItemID: "episode-1", Kind: KindEpisode},
{MediaItemID: "episode-2", Kind: KindEpisode},
},
}
matches, reason, err := NewMatcher(repo).MatchLeaves(context.Background(), Record{
Kind: KindSeries, Title: "Breaking Bad", Year: 2008, TMDBID: "1396", LastPlayedAt: &watchedAt,
})
if err != nil || reason != "" {
t.Fatalf("MatchLeaves error = %v, reason = %q", err, reason)
}
if len(matches) != 2 || matches[0].MediaItemID != "episode-1" || matches[1].MediaItemID != "episode-2" {
t.Fatalf("matches = %#v", matches)
}
if repo.episodesBySeriesID != "series-1" || repo.episodesBySeriesSeason != nil ||
repo.episodesBySeriesAt == nil || !repo.episodesBySeriesAt.Equal(watchedAt) {
t.Fatalf("unexpected expansion scope: id=%q season=%v watched_at=%v",
repo.episodesBySeriesID, repo.episodesBySeriesSeason, repo.episodesBySeriesAt)
}
}
func TestMatcherMatchLeavesPreservesSpecialsSeasonZero(t *testing.T) {
t.Parallel()
repo := &matcherRepoStub{
mediaByExternal: map[string][]mediaLookupRow{
"series:tvdb_id:75978": {{ContentID: "series-1", Title: "Family Guy", Year: 1999}},
},
episodesBySeries: []Match{{MediaItemID: "special-1", Kind: KindEpisode}},
}
matches, reason, err := NewMatcher(repo).MatchLeaves(context.Background(), Record{
Kind: KindSeason, SeriesTitle: "Family Guy", SeriesYear: 1999,
SeriesTVDBID: "75978", SeasonNumber: 0,
})
if err != nil || reason != "" || len(matches) != 1 {
t.Fatalf("matches = %#v, reason = %q, err = %v", matches, reason, err)
}
if repo.episodesBySeriesSeason == nil || *repo.episodesBySeriesSeason != 0 {
t.Fatalf("season scope = %v, want explicit season 0", repo.episodesBySeriesSeason)
}
}
func TestMatcherMatchEpisode_UsesExternalIDsWithoutSeasonEpisodeNumbers(t *testing.T) {
t.Parallel()
-39
View File
@@ -1050,45 +1050,6 @@ func (r *Repository) MatchEpisodeBySeries(ctx context.Context, seriesID string,
return &match, nil
}
func (r *Repository) MatchEpisodesBySeries(
ctx context.Context,
seriesID string,
seasonNumber *int,
watchedAt *time.Time,
) ([]Match, error) {
rows, err := r.pool.Query(ctx, `
SELECT e.content_id, COALESCE(e.title, ''), COALESCE(series.year, 0)
FROM episodes e
LEFT JOIN media_items series ON series.content_id = e.series_id
WHERE e.series_id = $1
AND ($2::integer IS NULL OR e.season_number = $2)
AND ($3::timestamptz IS NULL OR e.air_date IS NULL OR e.air_date <= ($3::timestamptz AT TIME ZONE 'UTC')::date)
AND EXISTS (
SELECT 1 FROM episode_libraries el WHERE el.episode_id = e.content_id
)
ORDER BY e.season_number, e.episode_number, e.content_id`,
seriesID, seasonNumber, watchedAt,
)
if err != nil {
return nil, fmt.Errorf("matching episodes by series: %w", err)
}
defer rows.Close()
matches := make([]Match, 0)
for rows.Next() {
var match Match
if err := rows.Scan(&match.MediaItemID, &match.Title, &match.Year); err != nil {
return nil, fmt.Errorf("scanning series episode match: %w", err)
}
match.Kind = KindEpisode
matches = append(matches, match)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterating series episode matches: %w", err)
}
return matches, nil
}
func scanSource(scanner interface{ Scan(dest ...any) error }) (*Source, error) {
var source Source
if err := scanner.Scan(
-1
View File
@@ -21,7 +21,6 @@ const (
KindMovie = "movie"
KindSeries = "series"
KindSeason = "season"
KindEpisode = "episode"
)
@@ -526,15 +526,6 @@ func (s *progressCountingStore) GetSeriesPlaybackPreference(context.Context, str
func (s *progressCountingStore) DeleteSeriesPlaybackPreference(context.Context, string, string) error {
panic("unused")
}
func (s *progressCountingStore) SetCollectionSortPreference(context.Context, userstore.CollectionSortPreference) error {
panic("unused")
}
func (s *progressCountingStore) GetCollectionSortPreference(context.Context, string, string, string) (*userstore.CollectionSortPreference, error) {
panic("unused")
}
func (s *progressCountingStore) ClearCollectionSortPreference(context.Context, string, string, string) error {
panic("unused")
}
func (s *progressCountingStore) GetLibraryPlaybackPreference(context.Context, string, int) (*userstore.LibraryPlaybackPreference, error) {
panic("unused")
}
+9 -42
View File
@@ -193,27 +193,24 @@ func (p *Planner) PlanSessionWith(sessionID, currentTranscodeURL string, needsTr
}
proxies := p.proxies.Nodes()
transcodes := p.transcodes.Nodes()
// Group health is computed over the full pool before any narrowing:
// eligibility restricts what may be selected, not co-location semantics.
groupHealthy := groupHealth(proxies, transcodes)
if eligible != nil {
filtered := make([]*Node, 0, len(transcodes))
for _, node := range transcodes {
if eligible(node) {
filtered = append(filtered, node)
}
}
transcodes = filtered
}
var plan Plan
if needsTranscode {
if eligible != nil {
transcodes = filterNodes(transcodes, eligible)
}
plan.TranscodeNode = p.pickTranscode(transcodes, proxies, groupHealthy, currentTranscodeURL, estBitrateKbps, now)
if plan.TranscodeNode != nil {
plan.ProxyNode = p.pickProxy(proxies, groupHealthy, plan.TranscodeNode.Group, estBitrateKbps, now)
}
} else {
// A proxy-only plan has no transcode node, so the predicate applies to
// the proxy: it is the node that will execute the recipe. Filtering
// before selection means a capability mismatch skips to a capable
// sibling instead of abandoning the pool after one round-robin pick.
if eligible != nil {
proxies = filterNodes(proxies, eligible)
}
plan.ProxyNode = p.pickProxy(proxies, groupHealthy, nil, estBitrateKbps, now)
}
@@ -231,36 +228,6 @@ func (p *Planner) PlanSessionWith(sessionID, currentTranscodeURL string, needsTr
return plan
}
// filterNodes returns the nodes accepted by keep, preserving pool order so
// round-robin cursors stay meaningful across selections.
func filterNodes(nodes []*Node, keep func(*Node) bool) []*Node {
filtered := make([]*Node, 0, len(nodes))
for _, node := range nodes {
if keep(node) {
filtered = append(filtered, node)
}
}
return filtered
}
// ProxyNodeURLs lists the URLs of every enabled pooled proxy node, healthy or
// not, mirroring TranscodeNodeURLs. Capability planning wants the deployment's
// toolchain; an unreachable node excludes itself when its capability fetch
// fails.
func (p *Planner) ProxyNodeURLs() []string {
if p == nil || p.proxies == nil {
return nil
}
nodes := p.proxies.Nodes()
urls := make([]string, 0, len(nodes))
for _, node := range nodes {
if node != nil && node.URL != "" {
urls = append(urls, node.URL)
}
}
return urls
}
// TranscodeNodeURLs lists the URLs of every enabled pooled transcode node,
// healthy or not: capability planning wants the deployment's toolchain, and
// an unreachable node excludes itself when its capability fetch fails. An
-56
View File
@@ -1,7 +1,6 @@
package nodepool
import (
"strconv"
"testing"
"time"
)
@@ -616,58 +615,3 @@ func TestPlanDownloadFallsBackWhenOriginGroupHasNoProxy(t *testing.T) {
t.Fatalf("plan = %+v, want proxy-a fallback", plan)
}
}
// A proxy-only plan applies the eligibility predicate to the proxy: it is the
// node that executes the recipe. Without this, one incapable round-robin pick
// would abandon a pool that still holds a capable sibling.
func TestPlanSessionWithFiltersProxiesForProxyOnlyPlans(t *testing.T) {
proxies := NewProxyPool()
proxies.SetNodes([]*Node{
{ID: 1, URL: "http://proxy-old", Enabled: true, Healthy: true},
{ID: 2, URL: "http://proxy-new", Enabled: true, Healthy: true},
})
planner := NewPlanner(proxies, NewTranscodePool())
// Only the upgraded proxy can run the recipe; every selection must land
// there regardless of where the round-robin cursor happens to be.
for i := 0; i < 4; i++ {
plan := planner.PlanSessionWith("session-"+strconv.Itoa(i), "", false, 0, func(n *Node) bool {
return n.URL == "http://proxy-new"
})
if plan.ProxyNode == nil || plan.ProxyNode.URL != "http://proxy-new" {
t.Fatalf("selection %d = %#v, want the capable proxy", i, plan.ProxyNode)
}
}
}
func TestPlanSessionWithReturnsNoProxyWhenNoneAreCapable(t *testing.T) {
proxies := NewProxyPool()
proxies.SetNodes([]*Node{{ID: 1, URL: "http://proxy-old", Enabled: true, Healthy: true}})
planner := NewPlanner(proxies, NewTranscodePool())
plan := planner.PlanSessionWith("session-none", "", false, 0, func(*Node) bool { return false })
if plan.ProxyNode != nil {
t.Fatalf("proxy = %#v, want none when the pool cannot execute the recipe", plan.ProxyNode)
}
// An empty plan must not leave a reservation pinning capacity.
if _, reserved := planner.reserved["session-none"]; reserved {
t.Fatal("an unsatisfiable plan left a reservation behind")
}
}
func TestProxyNodeURLsListsEnabledProxies(t *testing.T) {
proxies := NewProxyPool()
proxies.SetNodes([]*Node{
{ID: 1, URL: "http://proxy-1", Enabled: true, Healthy: true},
{ID: 2, URL: "http://proxy-2", Enabled: true, Healthy: false},
})
planner := NewPlanner(proxies, NewTranscodePool())
// Unhealthy nodes are still listed: capability planning wants the
// deployment's toolchain, and an unreachable node excludes itself when its
// capability fetch fails.
urls := planner.ProxyNodeURLs()
if len(urls) != 2 {
t.Fatalf("proxy urls = %v, want both pooled proxies", urls)
}
}
+2 -133
View File
@@ -39,28 +39,12 @@ func (p *interestTrackingProvider) ForUser(ctx context.Context, userID int) (use
return store, err
}
tracked := &interestTrackingStore{UserStore: store, userID: userID, system: p.system, updater: p.system.Interest}
// Preserve the interface upgrades callers probe for. Both are conditional
// on the backing store: advertising a capability it does not have would
// send callers down a fast path that can only fail.
registry, hasDevices := store.(userstore.DeviceRegistry)
rollup, hasRollup := store.(userstore.SeriesEpisodeRollupStore)
switch {
case hasDevices && hasRollup:
return &interestTrackingStoreWithDevicesAndRollup{
interestTrackingStore: tracked,
DeviceRegistry: registry,
SeriesEpisodeRollupStore: rollup,
}, nil
case hasDevices:
// Preserve the DeviceRegistry interface upgrade some callers probe for.
if registry, ok := store.(userstore.DeviceRegistry); ok {
return &interestTrackingStoreWithDevices{
interestTrackingStore: tracked,
DeviceRegistry: registry,
}, nil
case hasRollup:
return &interestTrackingStoreWithRollup{
interestTrackingStore: tracked,
SeriesEpisodeRollupStore: rollup,
}, nil
}
return tracked, nil
}
@@ -81,48 +65,11 @@ type interestTrackingStoreWithDevices struct {
userstore.DeviceRegistry
}
// interestTrackingStoreWithRollup adds the series-rollup capability only when
// the backing store actually has it. Unlike the other capabilities, this one
// cannot be forwarded unconditionally: the per-user SQLite backend has no
// catalog tables and genuinely cannot answer the query, and callers treat
// "implements the interface" as "can do this". Advertising it anyway would
// send every jellycompat series request down the fast path to fail, logging a
// warning each time before falling back — turning an expected capability
// absence into recurring noise on successful requests.
type interestTrackingStoreWithRollup struct {
*interestTrackingStore
userstore.SeriesEpisodeRollupStore
}
type interestTrackingStoreWithDevicesAndRollup struct {
*interestTrackingStore
userstore.DeviceRegistry
userstore.SeriesEpisodeRollupStore
}
var _ userstore.SettingValueCompareAndSetter = (*interestTrackingStore)(nil)
var _ userstore.SettingMutationTransactioner = (*interestTrackingStore)(nil)
var _ userstore.SettingValueCompareAndSetter = (*interestTrackingStoreWithDevices)(nil)
var _ userstore.SettingMutationTransactioner = (*interestTrackingStoreWithDevices)(nil)
// Optional store capabilities must survive the decorator. Embedding the
// UserStore interface promotes only that interface's methods, so each of these
// needs an explicit forward below; the assertions make a missing one a compile
// error instead of a silent production slowdown.
//
// SeriesEpisodeRollupStore is deliberately absent here: it is conditional on
// the backing store, so it lives on the wrapper types above rather than being
// forwarded unconditionally.
var _ userstore.WatchedBatchWriter = (*interestTrackingStore)(nil)
var _ userstore.VisibleHistoryAdder = (*interestTrackingStore)(nil)
var _ userstore.HistoryVisibilityStore = (*interestTrackingStore)(nil)
var _ userstore.WatchedBatchWriter = (*interestTrackingStoreWithDevices)(nil)
var _ userstore.VisibleHistoryAdder = (*interestTrackingStoreWithDevices)(nil)
var _ userstore.HistoryVisibilityStore = (*interestTrackingStoreWithDevices)(nil)
var _ userstore.SeriesEpisodeRollupStore = (*interestTrackingStoreWithRollup)(nil)
var _ userstore.SeriesEpisodeRollupStore = (*interestTrackingStoreWithDevicesAndRollup)(nil)
var _ userstore.DeviceRegistry = (*interestTrackingStoreWithDevicesAndRollup)(nil)
// WithPreferenceSettingsTransaction preserves the optional atomic-settings
// capability of the wrapped store. Preference writes do not affect interest
// signals, so the transaction can pass through unchanged; keeping the method
@@ -351,84 +298,6 @@ func (s *interestTrackingStore) RemoveHistoryItems(ctx context.Context, profileI
return err
}
// --- Optional store capabilities.
//
// This decorator embeds the userstore.UserStore *interface*, which promotes
// only the methods declared on that interface. Any optional capability the
// backing store implements is invisible through the wrapper unless it is
// forwarded explicitly here — and because callers reach these via type
// assertion with a working fallback, a missing forward is silent: no error, no
// test failure, just a much slower path in production. Marking a series
// watched regressed exactly this way. When adding a new optional capability to
// userstore, forward it here and extend the compile-time assertions below.
// MarkWatchedBatch forwards the transactional batch mark-watched write, whose
// whole purpose is that a large series commits as one unit. Falling through to
// the per-target loop would restore the partial-write window it removes.
func (s *interestTrackingStore) MarkWatchedBatch(
ctx context.Context,
profileID string,
targets []userstore.MarkWatchedTarget,
entries []userstore.WatchHistoryEntry,
) ([]userstore.WatchHistoryEntry, error) {
writer, ok := s.UserStore.(userstore.WatchedBatchWriter)
if !ok {
// The helper runs against the inner store, so this decorator's own
// MarkWatched hook never fires; queue here or the recompute is lost.
// A mid-loop error still leaves earlier targets written, so queue
// regardless of err — a redundant queue only costs one recompute,
// while a missing one leaves profile_series_interest stale until some
// unrelated mutation or the rebuild task touches the series.
written, err := userstore.MarkWatchedBatch(ctx, s.UserStore, profileID, targets, entries)
s.queueTargetMutations(profileID, targets)
return written, err
}
written, err := writer.MarkWatchedBatch(ctx, profileID, targets, entries)
// The batch write is one transaction: on error nothing landed, so there is
// nothing to recompute.
if err == nil {
s.queueItemMutations(profileID, written)
}
return written, err
}
// queueItemMutations records one interest mutation per written entry, matching
// what the per-target path queues through MarkWatched.
func (s *interestTrackingStore) queueItemMutations(profileID string, entries []userstore.WatchHistoryEntry) {
for _, entry := range entries {
s.updater.QueueItemMutation(s.userID, profileID, entry.MediaItemID)
}
}
// queueTargetMutations queues by requested target rather than written entry,
// for the non-transactional path where a partial write may have occurred.
func (s *interestTrackingStore) queueTargetMutations(profileID string, targets []userstore.MarkWatchedTarget) {
for _, target := range targets {
s.updater.QueueItemMutation(s.userID, profileID, target.MediaItemID)
}
}
// AddVisibleHistory forwards the watermark-aware history insert. The generic
// fallback needs two round-trips (timestamp lookup, then insert) to do the
// same job.
func (s *interestTrackingStore) AddVisibleHistory(ctx context.Context, entry userstore.WatchHistoryEntry) (userstore.WatchHistoryEntry, error) {
adder, ok := s.UserStore.(userstore.VisibleHistoryAdder)
if !ok {
return userstore.AddVisibleHistory(ctx, s.UserStore, entry)
}
return adder.AddVisibleHistory(ctx, entry)
}
// VisibleHistoryTimestamps forwards the batched watermark lookup; without it
// callers assume a single wall-clock timestamp for every item.
func (s *interestTrackingStore) VisibleHistoryTimestamps(ctx context.Context, profileID string, mediaItemIDs []string, at time.Time) (map[string]string, error) {
visibility, ok := s.UserStore.(userstore.HistoryVisibilityStore)
if !ok {
return userstore.VisibleHistoryTimestamps(ctx, s.UserStore, profileID, mediaItemIDs, at)
}
return visibility.VisibleHistoryTimestamps(ctx, profileID, mediaItemIDs, at)
}
func (s *interestTrackingStore) DeleteHistoryBySource(ctx context.Context, profileID string, mediaItemIDs []string, source userstore.WatchHistorySource) error {
err := s.UserStore.DeleteHistoryBySource(ctx, profileID, mediaItemIDs, source)
if err == nil {
@@ -100,231 +100,3 @@ func TestInterestTrackingStorePreservesSettingCapabilities(t *testing.T) {
t.Fatalf("wrapped receipt = %+v (%v)", receipt, err)
}
}
// TestInterestTrackingStorePreservesWatchStateCapabilities pins the optional
// watch-state capabilities through the wrapper. These are type-asserted at
// their call sites (userstore.MarkWatchedBatch, AddVisibleHistory,
// VisibleHistoryTimestamps, jellycompat's rollup), and the decorator embeds
// the UserStore *interface* — which promotes only the methods in that
// interface. A capability the decorator does not forward explicitly is
// therefore invisible in production, and the caller silently degrades to its
// slow per-item fallback with no error and no test failure.
func TestInterestTrackingStorePreservesWatchStateCapabilities(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
if err := userdb.InitSchema(db); err != nil {
t.Fatalf("init schema: %v", err)
}
inner := userdb.NewSQLiteUserStore(db)
provider := WrapUserStoreProvider(
preferenceTransactionTestProvider{store: inner},
&System{},
)
wrapped, err := provider.ForUser(context.Background(), 1)
if err != nil {
t.Fatalf("ForUser: %v", err)
}
// Every capability the bare store advertises must survive wrapping,
// otherwise the wrapped store is a silent performance downgrade.
for _, capability := range []struct {
name string
has func(userstore.UserStore) bool
}{
{"WatchedBatchWriter", func(s userstore.UserStore) bool {
_, ok := s.(userstore.WatchedBatchWriter)
return ok
}},
{"VisibleHistoryAdder", func(s userstore.UserStore) bool {
_, ok := s.(userstore.VisibleHistoryAdder)
return ok
}},
{"HistoryVisibilityStore", func(s userstore.UserStore) bool {
_, ok := s.(userstore.HistoryVisibilityStore)
return ok
}},
} {
if !capability.has(inner) {
t.Fatalf("test setup: bare store does not implement %s", capability.name)
}
if !capability.has(wrapped) {
t.Errorf("interest-tracking wrapper dropped %s; callers silently fall back to the per-item path",
capability.name)
}
}
// SeriesEpisodeRollupStore is the exception: it must be advertised only
// when the backing store can actually answer it. The per-user SQLite store
// has no catalog tables, so claiming the capability would send every
// jellycompat series request down a fast path that can only fail and log.
if _, ok := userstore.UserStore(inner).(userstore.SeriesEpisodeRollupStore); ok {
t.Fatal("test setup: SQLite store unexpectedly implements SeriesEpisodeRollupStore")
}
if _, ok := wrapped.(userstore.SeriesEpisodeRollupStore); ok {
t.Error("wrapper advertises SeriesEpisodeRollupStore for a store that cannot perform the rollup")
}
// The forwarded batch write must actually reach the backing store.
writer, ok := wrapped.(userstore.WatchedBatchWriter)
if !ok {
t.Fatal("wrapper dropped WatchedBatchWriter; cannot verify pass-through")
}
if err := wrapped.CreateProfile(context.Background(), userstore.Profile{ID: "p1", Name: "Test"}); err != nil {
t.Fatalf("CreateProfile: %v", err)
}
watchedAt := time.Date(2026, 8, 15, 12, 0, 0, 0, time.UTC).Format(time.RFC3339)
written, err := writer.MarkWatchedBatch(context.Background(), "p1",
[]userstore.MarkWatchedTarget{
{MediaItemID: "ep-1", DurationSeconds: 1200},
{MediaItemID: "ep-2", DurationSeconds: 1500},
},
[]userstore.WatchHistoryEntry{
{ProfileID: "p1", MediaItemID: "ep-1", WatchedAt: watchedAt, DurationSeconds: 1200, Completed: true, Source: userstore.WatchHistorySourceManual},
{ProfileID: "p1", MediaItemID: "ep-2", WatchedAt: watchedAt, DurationSeconds: 1500, Completed: true, Source: userstore.WatchHistorySourceManual},
},
)
if err != nil {
t.Fatalf("MarkWatchedBatch through wrapper: %v", err)
}
if len(written) != 2 {
t.Fatalf("MarkWatchedBatch returned %d entries, want 2", len(written))
}
for _, id := range []string{"ep-1", "ep-2"} {
progress, err := wrapped.GetProgress(context.Background(), "p1", id)
if err != nil || progress == nil || !progress.Completed {
t.Fatalf("GetProgress(%s) = %+v (%v), want completed", id, progress, err)
}
}
}
// storeWithoutBatchWriter wraps a UserStore and hides the WatchedBatchWriter
// capability, standing in for a backend that has not implemented it.
type storeWithoutBatchWriter struct {
userstore.UserStore
}
// TestInterestTrackingStoreQueuesMutationsOnBatchFallback covers the branch
// taken when the backing store lacks WatchedBatchWriter. The decorator hands
// the work to the generic helper against s.UserStore — the *inner* store — so
// the decorator's own MarkWatched hook never runs and nothing queues an
// interest recompute. Without this, marking a series watched on such a backend
// updates progress and history but leaves profile_series_interest stale until
// some unrelated mutation or the rebuild task happens to touch the series.
func TestInterestTrackingStoreQueuesMutationsOnBatchFallback(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
if err := userdb.InitSchema(db); err != nil {
t.Fatalf("init schema: %v", err)
}
inner := storeWithoutBatchWriter{UserStore: userdb.NewSQLiteUserStore(db)}
if _, ok := userstore.UserStore(inner).(userstore.WatchedBatchWriter); ok {
t.Fatal("test setup: inner store must not implement WatchedBatchWriter")
}
updater := &InterestUpdater{pending: map[interestMutation]int{}}
store := &interestTrackingStore{
UserStore: inner,
userID: 1,
system: &System{},
updater: updater,
}
if err := store.CreateProfile(context.Background(), userstore.Profile{ID: "p1", Name: "Test"}); err != nil {
t.Fatalf("CreateProfile: %v", err)
}
watchedAt := time.Date(2026, 8, 15, 12, 0, 0, 0, time.UTC).Format(time.RFC3339)
if _, err := store.MarkWatchedBatch(context.Background(), "p1",
[]userstore.MarkWatchedTarget{
{MediaItemID: "ep-1", DurationSeconds: 1200},
{MediaItemID: "ep-2", DurationSeconds: 1500},
},
[]userstore.WatchHistoryEntry{
{ProfileID: "p1", MediaItemID: "ep-1", WatchedAt: watchedAt, DurationSeconds: 1200, Completed: true, Source: userstore.WatchHistorySourceManual},
{ProfileID: "p1", MediaItemID: "ep-2", WatchedAt: watchedAt, DurationSeconds: 1500, Completed: true, Source: userstore.WatchHistorySourceManual},
},
); err != nil {
t.Fatalf("MarkWatchedBatch (fallback path): %v", err)
}
updater.mu.Lock()
queued := make(map[string]struct{}, len(updater.pending))
for mutation := range updater.pending {
queued[mutation.itemID] = struct{}{}
}
updater.mu.Unlock()
for _, itemID := range []string{"ep-1", "ep-2"} {
if _, ok := queued[itemID]; !ok {
t.Errorf("no interest mutation queued for %s on the batch fallback path; interest goes stale", itemID)
}
}
}
// rollupCapableStore is a UserStore that also implements the series rollup,
// standing in for the Postgres backend.
type rollupCapableStore struct {
userstore.UserStore
called bool
}
func (s *rollupCapableStore) SeriesEpisodeWatchCounts(_ context.Context, _ string, seriesIDs []string) (map[string]userstore.SeriesWatchCounts, error) {
s.called = true
counts := make(map[string]userstore.SeriesWatchCounts, len(seriesIDs))
for _, seriesID := range seriesIDs {
counts[seriesID] = userstore.SeriesWatchCounts{TotalEpisodes: 3, WatchedCount: 2}
}
return counts, nil
}
// TestInterestTrackingStoreForwardsRollupWhenSupported is the other half of the
// conditional: a backend that can do the rollup must keep advertising it
// through the wrapper, and calls must reach it. Losing this would silently
// push jellycompat back onto the per-episode rollup for every series.
func TestInterestTrackingStoreForwardsRollupWhenSupported(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
if err := userdb.InitSchema(db); err != nil {
t.Fatalf("init schema: %v", err)
}
inner := &rollupCapableStore{UserStore: userdb.NewSQLiteUserStore(db)}
provider := WrapUserStoreProvider(preferenceTransactionTestProvider{store: inner}, &System{})
wrapped, err := provider.ForUser(context.Background(), 1)
if err != nil {
t.Fatalf("ForUser: %v", err)
}
rollup, ok := wrapped.(userstore.SeriesEpisodeRollupStore)
if !ok {
t.Fatal("wrapper dropped SeriesEpisodeRollupStore for a store that supports it")
}
counts, err := rollup.SeriesEpisodeWatchCounts(context.Background(), "p1", []string{"series-1"})
if err != nil {
t.Fatalf("SeriesEpisodeWatchCounts: %v", err)
}
if !inner.called {
t.Error("rollup call did not reach the backing store")
}
if counts["series-1"].WatchedCount != 2 {
t.Errorf("counts = %+v, want WatchedCount 2", counts["series-1"])
}
// The other capabilities must still survive alongside the rollup.
if _, ok := wrapped.(userstore.WatchedBatchWriter); !ok {
t.Error("rollup-capable wrapper dropped WatchedBatchWriter")
}
if _, ok := wrapped.(userstore.SettingValueCompareAndSetter); !ok {
t.Error("rollup-capable wrapper dropped SettingValueCompareAndSetter")
}
}
+23 -71
View File
@@ -2,6 +2,7 @@ package notifications
import (
"context"
"errors"
"fmt"
"log/slog"
"sync"
@@ -9,6 +10,7 @@ import (
"github.com/Silo-Server/silo-server/internal/access"
"github.com/Silo-Server/silo-server/internal/userstore"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
@@ -125,34 +127,18 @@ func (u *InterestUpdater) flush(ctx context.Context) {
profileID string
seriesID string
}
// Resolve every queued item to its series in one query. Marking or
// unmarking a whole series queues one mutation per episode, and those
// thousands of items collapse to a single series — resolving them
// one-at-a-time cost thousands of round-trips before the dedupe below
// could discard them.
itemIDs := make([]string, 0, len(batch))
for mutation := range batch {
itemIDs = append(itemIDs, mutation.itemID)
}
seriesByItem, resolveErr := u.resolveSeriesIDs(ctx, itemIDs)
if resolveErr != nil {
u.logger.WarnContext(ctx, "interest series resolution failed",
"item_count", len(itemIDs), "error", resolveErr)
}
seen := make(map[recomputeKey]struct{}, len(batch))
for mutation, failures := range batch {
if ctx.Err() != nil {
return // shutting down; the rebuild task repairs anything dropped
}
if resolveErr != nil {
// The lookup failed for the whole batch, so no item can be
// classified as "not a series". Requeue rather than silently
// dropping interest updates.
seriesID, ok, err := u.resolveSeriesID(ctx, mutation.itemID)
if err != nil {
u.logger.WarnContext(ctx, "interest series resolution failed",
"item_id", mutation.itemID, "error", err)
requeue[mutation] = failures + 1
continue
}
seriesID, ok := seriesByItem[mutation.itemID]
if !ok {
continue // movies and unknown items have no series interest
}
@@ -163,7 +149,7 @@ func (u *InterestUpdater) flush(ctx context.Context) {
seen[key] = struct{}{}
recomputeCtx, cancel := context.WithTimeout(ctx, interestRecomputeTime)
err := u.RecomputeSeries(recomputeCtx, mutation.userID, mutation.profileID, seriesID)
err = u.RecomputeSeries(recomputeCtx, mutation.userID, mutation.profileID, seriesID)
cancel()
if err != nil {
u.logger.WarnContext(ctx, "interest recompute failed",
@@ -192,60 +178,26 @@ func (u *InterestUpdater) flush(ctx context.Context) {
u.mu.Unlock()
}
// resolveSeriesIDs maps many media item IDs (series, season, or episode) to
// their series content IDs in one query. Items with no series — movies and
// unknown IDs — are absent from the result, which is what lets the caller keep
// treating "missing" as "no series interest". Semantics match resolveSeriesID
// per item; only the number of round-trips differs.
func (u *InterestUpdater) resolveSeriesIDs(ctx context.Context, itemIDs []string) (map[string]string, error) {
result := make(map[string]string, len(itemIDs))
ids := make([]string, 0, len(itemIDs))
seen := make(map[string]struct{}, len(itemIDs))
for _, itemID := range itemIDs {
if itemID == "" {
continue
}
if _, dup := seen[itemID]; dup {
continue
}
seen[itemID] = struct{}{}
ids = append(ids, itemID)
}
if len(ids) == 0 {
return result, nil
}
// One branch per item kind, unioned, mirroring the single-item lookup.
rows, err := u.pool.Query(ctx, `
SELECT content_id, series_id FROM episodes WHERE content_id = ANY($1::text[])
// resolveSeriesID maps a media item ID (series, season, or episode) to its
// series content ID. Returns ok=false for movies and unknown items.
func (u *InterestUpdater) resolveSeriesID(ctx context.Context, itemID string) (string, bool, error) {
var seriesID string
err := u.pool.QueryRow(ctx, `
SELECT series_id FROM episodes WHERE content_id = $1
UNION ALL
SELECT content_id, series_id FROM seasons WHERE content_id = ANY($1::text[])
SELECT series_id FROM seasons WHERE content_id = $1
UNION ALL
SELECT content_id, content_id FROM media_items WHERE content_id = ANY($1::text[]) AND type = 'series'`,
ids,
)
SELECT content_id FROM media_items WHERE content_id = $1 AND type = 'series'
LIMIT 1`,
itemID,
).Scan(&seriesID)
if errors.Is(err, pgx.ErrNoRows) {
return "", false, nil
}
if err != nil {
return nil, err
return "", false, err
}
defer rows.Close()
for rows.Next() {
var itemID, seriesID string
if err := rows.Scan(&itemID, &seriesID); err != nil {
return nil, err
}
if seriesID == "" {
continue
}
// An ID matching more than one branch keeps the first row, matching
// the single-item query's UNION ALL ... LIMIT 1 ordering.
if _, exists := result[itemID]; !exists {
result[itemID] = seriesID
}
}
if err := rows.Err(); err != nil {
return nil, err
}
return result, nil
return seriesID, seriesID != "", nil
}
// RecomputeSeries rebuilds the (profile, series) interest rows from
@@ -1,85 +0,0 @@
package notifications
import (
"context"
"os"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
)
// TestResolveSeriesIDsBatchesLookups pins the batched item→series resolution
// used by the interest flush loop. Marking or unmarking a whole series queues
// one mutation per episode, and those all collapse to a single series; the
// previous per-item lookup issued one query each, which is what made a
// large-series unmark spend minutes in `episodes` lookups.
//
// Uses temp tables shadowing episodes/seasons/media_items, with the pool
// pinned to one connection so every query sees them.
func TestResolveSeriesIDsBatchesLookups(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("set SILO_TEST_DATABASE_URL to run the DB-backed series resolution test")
}
ctx := context.Background()
config, err := pgxpool.ParseConfig(dsn)
if err != nil {
t.Fatalf("parse db config: %v", err)
}
config.MaxConns = 1
pool, err := pgxpool.NewWithConfig(ctx, config)
if err != nil {
t.Fatalf("connect db: %v", err)
}
t.Cleanup(pool.Close)
for _, ddl := range []string{
`CREATE TEMP TABLE episodes (content_id text PRIMARY KEY, series_id text NOT NULL)`,
`CREATE TEMP TABLE seasons (content_id text PRIMARY KEY, series_id text NOT NULL)`,
`CREATE TEMP TABLE media_items (content_id text PRIMARY KEY, type text NOT NULL)`,
} {
if _, err := pool.Exec(ctx, ddl); err != nil {
t.Fatalf("create temp table: %v", err)
}
}
if _, err := pool.Exec(ctx, `
INSERT INTO episodes (content_id, series_id) VALUES ('ep-1','series-1'),('ep-2','series-1'),('ep-3','series-2');
INSERT INTO seasons (content_id, series_id) VALUES ('season-1','series-1');
INSERT INTO media_items (content_id, type) VALUES ('series-1','series'),('movie-1','movie');
`); err != nil {
t.Fatalf("seed: %v", err)
}
updater := &InterestUpdater{pool: pool}
got, err := updater.resolveSeriesIDs(ctx, []string{
"ep-1", "ep-2", "ep-3", "season-1", "series-1", "movie-1", "missing", "", "ep-1",
})
if err != nil {
t.Fatalf("resolveSeriesIDs: %v", err)
}
for itemID, wantSeries := range map[string]string{
"ep-1": "series-1",
"ep-2": "series-1",
"ep-3": "series-2",
"season-1": "series-1",
"series-1": "series-1",
} {
if got[itemID] != wantSeries {
t.Errorf("resolveSeriesIDs[%s] = %q, want %q", itemID, got[itemID], wantSeries)
}
}
// Movies, unknown IDs, and blanks must be absent rather than mapped: the
// flush loop reads "missing" as "no series interest" and skips them.
for _, itemID := range []string{"movie-1", "missing", ""} {
if seriesID, ok := got[itemID]; ok {
t.Errorf("resolveSeriesIDs[%q] = %q, want absent", itemID, seriesID)
}
}
if empty, err := updater.resolveSeriesIDs(ctx, nil); err != nil || len(empty) != 0 {
t.Fatalf("resolveSeriesIDs(nil) = %v (%v), want empty", empty, err)
}
}
@@ -512,34 +512,6 @@ func TestRequestSchemaRequiredFieldsMatchValidators(t *testing.T) {
assertStringsEqual(t, "route_event.required", schemaStrings(t, routeEvent, "required"), []string{"protocol_version", "playback_attempt_id", "event"})
}
// TestRequestSchemasShareIdenticalDefs pins the one thing two copies of a
// definition can never be trusted to keep on their own. Start and replan
// deserialize the *same* Go types — ClientCodecCapabilitiesV3 and
// ClientPlaybackContextV3 — through the same validator, so any $def they both
// declare must be byte-identical. Adding a field to one schema and not the
// other compiles, validates, and ships a contract that lies about one of the
// two endpoints; this is what catches it.
func TestRequestSchemasShareIdenticalDefs(t *testing.T) {
start := schemaValue(t, mustReadObject(t, filepath.Join(schemaRootV3, "v3", "start-request.schema.json")), "$defs").(map[string]any)
replan := schemaValue(t, mustReadObject(t, filepath.Join(schemaRootV3, "v3", "replan-request.schema.json")), "$defs").(map[string]any)
shared := make([]string, 0, len(start))
for name := range start {
if _, ok := replan[name]; ok {
shared = append(shared, name)
}
}
sort.Strings(shared)
if !slices.Contains(shared, "client_playback_context") || !slices.Contains(shared, "client_capabilities") {
t.Fatalf("shared $defs = %v, want the client contract definitions in both request schemas", shared)
}
for _, name := range shared {
if !reflect.DeepEqual(start[name], replan[name]) {
t.Errorf("$defs.%s differs between start-request and replan-request; both deserialize the same Go type", name)
}
}
}
func compileSchemasV3(t *testing.T) map[string]*jsonschema.Schema {
t.Helper()
+3 -6
View File
@@ -360,16 +360,13 @@ func (s *Postgres) RecordRouteEvent(ctx context.Context, record playback.RouteEv
INSERT INTO playback_route_events (
playback_attempt_id, session_id, plan_id, plan_attempt_id, plan_attempt_key,
event, failure_classification, fallback_reason, output_context_id,
diagnostics, user_id, profile_id, client_name, client_version, client_build,
client_channel, client_model
diagnostics, user_id, profile_id, client_name, client_version, client_model
) VALUES ($1, NULLIF($2, '')::uuid, NULLIF($3, ''), NULLIF($4, ''), NULLIF($5, ''),
$6, NULLIF($7, ''), NULLIF($8, ''), NULLIF($9, ''), $10, $11, $12,
NULLIF($13, ''), NULLIF($14, ''), NULLIF($15, ''), NULLIF($16, ''),
NULLIF($17, ''))`,
NULLIF($13, ''), NULLIF($14, ''), NULLIF($15, ''))`,
record.PlaybackAttemptID, record.SessionID, record.PlanID, record.PlanAttemptID, record.PlanAttemptKey,
record.Event, record.FailureClassification, record.FallbackReason, record.OutputContextID,
diagnostics, record.UserID, record.ProfileID, record.ClientName, record.ClientVersion,
record.ClientBuild, record.ClientChannel, record.ClientModel)
diagnostics, record.UserID, record.ProfileID, record.ClientName, record.ClientVersion, record.ClientModel)
return err
}
-4
View File
@@ -57,10 +57,6 @@ type RouteEventRecordV3 struct {
ProfileID string
ClientName string
ClientVersion string
// ClientBuild and ClientChannel are opaque client-reported identifiers,
// recorded so a route decision can be attributed to an exact app build.
ClientBuild string
ClientChannel string
ClientModel string
}
+6 -18
View File
@@ -331,17 +331,12 @@ type DeliveryCapabilityV3 struct {
// advertisement lives exclusively in the request's top-level client_features
// list; there is deliberately no second features location here.
type ClientPlaybackContextV3 struct {
ProtocolVersion int `json:"protocol_version"`
FormFactor string `json:"form_factor"`
AppVersion string `json:"app_version"`
// AppBuild and AppChannel are the request-body fallback for the
// X-Silo-Client-Build / X-Silo-Client-Channel headers. Both are opaque
// strings the server stores verbatim.
AppBuild string `json:"app_build,omitempty"`
AppChannel string `json:"app_channel,omitempty"`
Device DeviceContextV3 `json:"device"`
Output OutputContextV3 `json:"output"`
Deliveries map[string]DeliveryCapabilityV3 `json:"deliveries"`
ProtocolVersion int `json:"protocol_version"`
FormFactor string `json:"form_factor"`
AppVersion string `json:"app_version"`
Device DeviceContextV3 `json:"device"`
Output OutputContextV3 `json:"output"`
Deliveries map[string]DeliveryCapabilityV3 `json:"deliveries"`
}
type StartRequestV3 struct {
@@ -895,13 +890,6 @@ func validateCapabilitiesV3(c *ClientCodecCapabilitiesV3, ctx *ClientPlaybackCon
if len(c.CodecsVideo) > 64 || len(c.CodecsVideoHardware) > 64 || len(c.CodecsAudio) > 64 || len(c.Containers) > 64 || len(c.VideoDecode) > 64 || len(ctx.Deliveries) > 16 || len(ctx.Device.Platform) > 32 || len(ctx.FormFactor) > 32 || len(ctx.AppVersion) > 64 {
return errors.New("capability list exceeds supported size")
}
// Build and channel are opaque diagnostic labels, so an over-long value is
// worth clamping and never worth refusing playback over. The header route
// (X-Silo-Client-Build / -Channel) clamps with the same helper; rejecting
// here would mean the same string plays from a header and 400s from the
// body.
ctx.AppBuild = normalizeClientMetadataValue(ctx.AppBuild, 64)
ctx.AppChannel = normalizeClientMetadataValue(ctx.AppChannel, 32)
deviceValues := []string{
ctx.Device.OSVersion, ctx.Device.Manufacturer, ctx.Device.Model,
ctx.Output.CurrentSink, ctx.Output.SinkType, ctx.Output.OutputContextID,
+9 -178
View File
@@ -8,8 +8,6 @@ import (
"strings"
"sync"
"time"
"unicode"
"unicode/utf8"
"github.com/google/uuid"
)
@@ -28,8 +26,6 @@ type Session struct {
ClientIP string // resolved client IP for the playback session
ClientName string // reported playback client name, when available
ClientVersion string // reported playback client version, when available
ClientBuild string // opaque reported client build identifier, when available
ClientChannel string // opaque reported client distribution channel, when available
ClientUserAgent string // trimmed request user agent for the playback session
IsJellyfinCompat bool // immutable origin identity for Jellyfin compatibility sessions
@@ -66,10 +62,6 @@ type Session struct {
activeTransportCount int
replacementPlayMethod PlayMethod
streamRevision uint64
// remoteTransport marks a session whose media bytes are served by another
// node, so this server never sees the transport request that would
// otherwise keep it alive. See SetRemoteTransport.
remoteTransport bool
}
// SessionStreamState stores the mutable stream-specific details that can
@@ -151,76 +143,12 @@ type clientInfoContextKey struct{}
// ClientInfo carries best-effort client metadata from request handling into
// the playback session manager.
type ClientInfo struct {
Name string
Version string
// Build is the client's opaque per-platform build identifier (Android
// versionCode, Apple CFBundleVersion, …). The server never parses or
// compares it; it exists so an admin can name the exact build.
Build string
// Channel is the client's opaque distribution channel ("release", "beta",
// "sideload", "dev", …). Stored verbatim — deliberately not validated
// against an enum so a new channel needs no server change.
Channel string
Name string
Version string
UserAgent string
IsCompat bool
}
// Normalized returns the identity with every field trimmed and clamped to the
// bound published for it (docs/settings-api.md, and maxLength in the v3 request
// schemas). This is the single definition of those bounds, and callers apply it
// at the request boundary: a session stamps normalized values, but the decision
// logs and playback_route_events are written straight from the resolved
// identity, so clamping only at session creation would let a client's oversized
// header through to both.
func (c ClientInfo) Normalized() ClientInfo {
c.Name = normalizeClientMetadataValue(c.Name, 128)
c.Version = normalizeClientMetadataValue(c.Version, 64)
c.Build = normalizeClientMetadataValue(c.Build, 64)
c.Channel = normalizeClientMetadataValue(c.Channel, 32)
c.UserAgent = normalizeClientMetadataValue(c.UserAgent, 512)
return c
}
// LogAttrs renders the app identity as slog key/value pairs, skipping the
// fields the client did not report. This is the single definition of those log
// keys — every playback decision and the session-expiry line share it, so a
// rename cannot leave one surface keyed differently from another. Skipping
// empty values matters as much: browsers and Jellyfin-ecosystem clients report
// none of them, and opslog persists the attrs it is handed, so emitting four
// empty keys per decision would grow /admin/logs for no diagnostic value.
func (c ClientInfo) LogAttrs() []any {
attrs := make([]any, 0, 8)
for _, pair := range [...]struct{ key, value string }{
{"client_name", c.Name},
{"client_version", c.Version},
{"client_build", c.Build},
{"client_channel", c.Channel},
} {
if pair.value != "" {
attrs = append(attrs, pair.key, pair.value)
}
}
return attrs
}
// ClientInfo returns the app identity stamped on the session when it was
// created. Surfaces that only hold a session — expiry logging, route events
// posted out of band — recover the reporting client through this instead of
// re-reading request headers that may no longer be present.
func (s *Session) ClientInfo() ClientInfo {
if s == nil {
return ClientInfo{}
}
return ClientInfo{
Name: s.ClientName,
Version: s.ClientVersion,
Build: s.ClientBuild,
Channel: s.ClientChannel,
UserAgent: s.ClientUserAgent,
IsCompat: s.IsJellyfinCompat,
}
}
// WithClientInfo stores playback client metadata on a context.
func WithClientInfo(ctx context.Context, info ClientInfo) context.Context {
if ctx == nil {
@@ -303,12 +231,6 @@ const (
// observed playback activity before it stops counting toward limits.
DefaultActiveSessionGrace = 45 * time.Second
// remoteTransportIdleGrace is the floor on the idle windows for a session
// whose media is served by another node. See remoteTransportGrace: it must
// outlast a heartbeat gap on an otherwise healthy proxy stream, while still
// reaping a session whose client vanished without stopping.
remoteTransportIdleGrace = 5 * time.Minute
// DefaultPausedSessionGrace is the longer grace period for paused
// sessions. It must comfortably cover an intentional pause (dinner
// break, phone call): reaping a paused session kills its transcode
@@ -370,41 +292,9 @@ func (m *SessionManager) SetExpirationHook(fn func(*Session)) {
}
func normalizeClientMetadataValue(value string, maxLen int) string {
// A text column takes neither invalid UTF-8 nor a NUL, and Postgres refuses
// the whole statement for either. The per-node session upserts share one
// transaction, so a single malformed client string would stop that entire
// node from reconciling — not just its own row — until the session goes
// away. Both scrubs are needed: NUL is perfectly valid UTF-8, so
// ToValidUTF8 leaves it, and a v3 start body can carry one as the JSON
// escape (headers cannot — net/http rejects bytes below 0x20).
// Control characters are stripped wholesale rather than just NUL: none of
// them belong in an identity label rendered in the admin UI and written to
// structured logs.
if !utf8.ValidString(value) {
value = strings.ToValidUTF8(value, "")
}
if strings.ContainsFunc(value, unicode.IsControl) {
value = strings.Map(func(r rune) rune {
if unicode.IsControl(r) {
return -1
}
return r
}, value)
}
value = strings.TrimSpace(value)
if maxLen <= 0 || len(value) <= maxLen {
return value
}
// Clamp by runes, not bytes. These bounds are published to clients as JSON
// Schema maxLength, which counts characters — a byte clamp would silently
// cut a value the contract calls valid, and cutting mid-rune would produce
// exactly the invalid UTF-8 scrubbed above.
runes := 0
for offset := range value {
if runes == maxLen {
return value[:offset]
}
runes++
if maxLen > 0 && len(value) > maxLen {
value = value[:maxLen]
}
return value
}
@@ -539,10 +429,7 @@ func newSession(
transcodeAudio bool,
) *Session {
now := time.Now()
// Normalize here as well as at the request boundary: identities also reach
// the manager from the Jellyfin and Audiobookshelf compat surfaces, which
// build a ClientInfo from their own header vocabularies.
clientInfo := ClientInfoFromContext(ctx).Normalized()
clientInfo := ClientInfoFromContext(ctx)
return &Session{
ID: uuid.New().String(),
UserID: userID,
@@ -554,11 +441,9 @@ func newSession(
TranscodeAudio: transcodeAudio,
Position: 0,
IsPaused: false,
ClientName: clientInfo.Name,
ClientVersion: clientInfo.Version,
ClientBuild: clientInfo.Build,
ClientChannel: clientInfo.Channel,
ClientUserAgent: clientInfo.UserAgent,
ClientName: normalizeClientMetadataValue(clientInfo.Name, 128),
ClientVersion: normalizeClientMetadataValue(clientInfo.Version, 64),
ClientUserAgent: normalizeClientMetadataValue(clientInfo.UserAgent, 512),
IsJellyfinCompat: clientInfo.IsCompat,
StartedAt: now,
UpdatedAt: now,
@@ -1210,34 +1095,6 @@ func (m *SessionManager) BeginTransport(sessionID string) error {
return nil
}
// SetRemoteTransport records whether this session's media bytes are served by
// another node (a proxy), rather than by a transport request this server
// handles itself.
//
// A locally-served stream is protected from the idle reaper by
// BeginTransport/EndTransport around the serve call. A proxy-served stream
// never enters that path: the client talks to the proxy directly, so from this
// server's point of view a healthy multi-hour stream looks identical to an
// abandoned session, and a heartbeat gap longer than the active grace would
// reap it mid-playback — after which progress, stop, and replan all fail with
// session-not-found while bytes are still flowing.
//
// This marks the session as remotely transported for its whole lifetime rather
// than per-request, because there is no request to bracket. Progress
// heartbeats still drive UI liveness; this only prevents reaping.
func (m *SessionManager) SetRemoteTransport(sessionID string, remote bool) error {
m.mu.Lock()
defer m.mu.Unlock()
s, ok := m.sessions[sessionID]
if !ok {
return ErrSessionNotFound
}
s.remoteTransport = remote
m.touchSessionLocked(s)
return nil
}
// EndTransport decrements the count of in-flight media transport requests for
// the session and refreshes its activity timestamp.
func (m *SessionManager) EndTransport(sessionID string) error {
@@ -1418,7 +1275,6 @@ func (m *SessionManager) CleanInactive(activeIdle, pausedIdle time.Duration) []*
if s.activeTransportCount > 0 {
continue
}
activeIdle, pausedIdle := remoteTransportGrace(s, activeIdle, pausedIdle)
if m.sessionIsInactiveLocked(s, now, activeIdle, pausedIdle) {
cp := *s
expired = append(expired, &cp)
@@ -1449,32 +1305,7 @@ func (m *SessionManager) countsTowardLimitsLocked(s *Session, now time.Time) boo
if s.activeTransportCount > 0 {
return true
}
activeGrace, pausedGrace := remoteTransportGrace(s, m.activeGrace, m.pausedGrace)
return !m.sessionIsInactiveLocked(s, now, activeGrace, pausedGrace)
}
// remoteTransportGrace widens the idle windows for a session whose bytes are
// served by another node.
//
// A locally-served stream is held open by an in-flight transport request. A
// proxy-served one has no such request here, so it is protected only by the
// client's progress heartbeats — and a gap longer than the active grace would
// reap it while media is still flowing. The windows are widened rather than
// made infinite: there is no absolute session lifetime cap in this manager, so
// unconditional immunity would leak a session forever whenever a client
// disappears without stopping. A client that has gone quiet for this long has
// genuinely stopped watching.
func remoteTransportGrace(s *Session, activeIdle, pausedIdle time.Duration) (time.Duration, time.Duration) {
if s == nil || !s.remoteTransport {
return activeIdle, pausedIdle
}
if activeIdle > 0 && activeIdle < remoteTransportIdleGrace {
activeIdle = remoteTransportIdleGrace
}
if pausedIdle > 0 && pausedIdle < remoteTransportIdleGrace {
pausedIdle = remoteTransportIdleGrace
}
return activeIdle, pausedIdle
return !m.sessionIsInactiveLocked(s, now, m.activeGrace, m.pausedGrace)
}
func (m *SessionManager) sessionIsInactiveLocked(s *Session, now time.Time, activeIdle, pausedIdle time.Duration) bool {
@@ -1,97 +0,0 @@
package playback
import (
"testing"
"time"
)
// A proxy-served session has no in-flight transport request on this server, so
// the idle sweep is all that stands between a healthy multi-hour stream and
// being reaped mid-playback.
func TestRemoteTransportGraceWidensIdleWindows(t *testing.T) {
local := &Session{}
remote := &Session{remoteTransport: true}
if active, paused := remoteTransportGrace(local, 45*time.Second, 2*time.Minute); active != 45*time.Second || paused != 2*time.Minute {
t.Fatalf("local grace = (%v, %v), want the configured windows unchanged", active, paused)
}
active, paused := remoteTransportGrace(remote, 45*time.Second, 2*time.Minute)
if active != remoteTransportIdleGrace || paused != remoteTransportIdleGrace {
t.Fatalf("remote grace = (%v, %v), want both widened to %v", active, paused, remoteTransportIdleGrace)
}
}
// The widened window must not become immunity: this manager has no absolute
// session lifetime, so a client that disappears without stopping would leak the
// session forever.
func TestRemoteTransportGraceStillExpires(t *testing.T) {
manager := NewSessionManager(0, 0)
session, err := manager.StartSession(7, "profile-1", 42, PlayDirect, false)
if err != nil {
t.Fatalf("start session: %v", err)
}
if err := manager.SetRemoteTransport(session.ID, true); err != nil {
t.Fatalf("mark remote transport: %v", err)
}
// Backdate the session past the widened window, as an abandoned client would.
manager.mu.Lock()
stale := time.Now().Add(-2 * remoteTransportIdleGrace)
manager.sessions[session.ID].LastActivityAt = stale
manager.sessions[session.ID].UpdatedAt = stale
manager.mu.Unlock()
if expired := manager.CleanInactive(45*time.Second, 2*time.Minute); len(expired) != 1 {
t.Fatalf("expired %d abandoned proxy sessions, want 1", len(expired))
}
}
// A session inside the widened window survives a heartbeat gap that would have
// reaped it before, which is the reported failure this guards against.
func TestRemoteTransportSurvivesHeartbeatGap(t *testing.T) {
manager := NewSessionManager(0, 0)
session, err := manager.StartSession(7, "profile-1", 42, PlayDirect, false)
if err != nil {
t.Fatalf("start session: %v", err)
}
if err := manager.SetRemoteTransport(session.ID, true); err != nil {
t.Fatalf("mark remote transport: %v", err)
}
manager.mu.Lock()
gap := time.Now().Add(-90 * time.Second) // twice the default active grace
manager.sessions[session.ID].LastActivityAt = gap
manager.sessions[session.ID].UpdatedAt = gap
manager.mu.Unlock()
if expired := manager.CleanInactive(DefaultActiveSessionGrace, DefaultPausedSessionGrace); len(expired) != 0 {
t.Fatalf("reaped %d healthy proxy-served sessions during a heartbeat gap", len(expired))
}
}
// Clearing the mark restores normal reaping, so a re-plan that moves a session
// back onto this server does not leave it with a stale widened grace.
func TestRemoteTransportCanBeCleared(t *testing.T) {
manager := NewSessionManager(0, 0)
session, err := manager.StartSession(7, "profile-1", 42, PlayDirect, false)
if err != nil {
t.Fatalf("start session: %v", err)
}
if err := manager.SetRemoteTransport(session.ID, true); err != nil {
t.Fatalf("mark remote transport: %v", err)
}
if err := manager.SetRemoteTransport(session.ID, false); err != nil {
t.Fatalf("clear remote transport: %v", err)
}
manager.mu.Lock()
gap := time.Now().Add(-90 * time.Second)
manager.sessions[session.ID].LastActivityAt = gap
manager.sessions[session.ID].UpdatedAt = gap
manager.mu.Unlock()
if expired := manager.CleanInactive(DefaultActiveSessionGrace, DefaultPausedSessionGrace); len(expired) != 1 {
t.Fatalf("expired %d sessions after clearing the mark, want 1", len(expired))
}
}
@@ -10,7 +10,6 @@
"playback_route_diagnostics",
"device_quirks_v1",
"seek_reanchor_v1",
"output_change_v1",
"direct_stream_resume_v1",
"plan_source_duration_v1"
],
@@ -5406,7 +5406,6 @@
"playback_route_diagnostics",
"device_quirks_v1",
"seek_reanchor_v1",
"output_change_v1",
"direct_stream_resume_v1",
"plan_source_duration_v1"
],
@@ -7,7 +7,6 @@
"playback_route_diagnostics",
"device_quirks_v1",
"seek_reanchor_v1",
"output_change_v1",
"direct_stream_resume_v1",
"plan_source_duration_v1"
],
-50
View File
@@ -1,50 +0,0 @@
package proxy
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/Silo-Server/silo-server/internal/playback"
)
// The API validates a frozen remux recipe against the proxy that will execute
// it, so a proxy must answer the same capability probe a transcode node does.
// Without it the API cannot tell a capable proxy from one on an older ffmpeg,
// and the mismatch only surfaces as a failed stream.
func TestProxyAdvertisesTransformationCapabilities(t *testing.T) {
const secret = "capability-secret"
server := newDownloadProxyServer(t, secret)
request := httptest.NewRequest(http.MethodGet, "/hw-capabilities", nil)
request.Header.Set("Authorization", "Bearer "+secret)
recorder := httptest.NewRecorder()
server.Handler().ServeHTTP(recorder, request)
if recorder.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", recorder.Code)
}
var info playback.HWAccelInfo
if err := json.Unmarshal(recorder.Body.Bytes(), &info); err != nil {
t.Fatalf("decode capabilities: %v", err)
}
// The probe reflects whatever ffmpeg this machine has; the contract under
// test is that the endpoint exists and answers in the shared shape.
for _, transformation := range info.Transformations {
if transformation.Name == "" || transformation.RecipeVersion == "" {
t.Fatalf("advertised transformation is missing identity: %#v", transformation)
}
}
}
func TestProxyCapabilitiesRequireBearer(t *testing.T) {
server := newDownloadProxyServer(t, "capability-secret")
recorder := httptest.NewRecorder()
server.Handler().ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/hw-capabilities", nil))
if recorder.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401 for an unauthenticated capability probe", recorder.Code)
}
}
-10
View File
@@ -77,16 +77,6 @@ func (w *meteredResponseWriter) Flush() {
}
}
// Unwrap lets http.ResponseController traverse to the underlying writer.
//
// Without it the metering wrapper is a dead end: RollingDeadlineWriter's
// SetWriteDeadline call fails, it degrades to a plain pass-through, and since
// the standalone proxy runs with WriteTimeout 0 there is no server-level guard
// behind it. A client that stops reading without closing its connection would
// then block a stream write forever, holding the tracked session, the open
// file, the goroutine and the connection.
func (w *meteredResponseWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }
// meterEgress wraps stream handlers so their responses count toward the
// node's measured egress bandwidth.
func (s *Server) meterEgress(next http.Handler) http.Handler {
+1 -39
View File
@@ -103,17 +103,6 @@ func (s *Server) Handler() http.Handler {
"Accept", "Authorization", "Content-Type", "Range",
"If-Match", "If-Modified-Since", "If-None-Match", "If-Range", "If-Unmodified-Since",
},
// direct_stream_resume_v1 has the client re-request a byte range with
// If-Range against the entity tag it stored. Cross-origin JavaScript
// can only read a response header that is explicitly exposed, so
// without these the client can send the conditional request headers
// above but never learn the values to put in them — the resume
// contract silently degrades to a full restart on a proxy that is on a
// different origin than the web app, which is the normal deployment.
ExposedHeaders: []string{
"Accept-Ranges", "Content-Encoding", "Content-Length", "Content-Range",
"ETag", "Last-Modified",
},
MaxAge: 86400,
}))
r.Get("/api/v1/health", s.handleHealth)
@@ -136,34 +125,12 @@ func (s *Server) Handler() http.Handler {
// Admin routes — bearer-auth protected.
r.Group(func(r chi.Router) {
r.Use(s.requireBearer)
r.Get("/hw-capabilities", s.handleHWCapabilities)
r.Post("/admin/force-reload", s.handleForceReload)
r.Get("/status", s.handleStatus)
})
return r
}
// handleHWCapabilities advertises what this proxy's ffmpeg can actually do, in
// the same shape and at the same path as a transcode node.
//
// A proxy executes recipes too: /stream/remux runs ffmpeg to convert audio or
// strip a Dolby Vision RPU. Without this endpoint the API has no way to tell
// whether the proxy it just picked can run the transformations a plan froze, so
// a pool whose proxies carry a different ffmpeg build (a rolling upgrade, a
// custom image) would fail at stream time rather than at selection time.
func (s *Server) handleHWCapabilities(w http.ResponseWriter, r *http.Request) {
ffmpegPath := ""
if cfg := s.watcher.Config(); cfg != nil {
ffmpegPath = cfg.Playback.FFmpegPath
}
info := playback.DetectHWAccelWithFFmpeg(ffmpegPath)
info.Transformations = playback.ProbeTransformationRegistryV3(r.Context(), ffmpegPath).Advertised()
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(info); err != nil {
slog.WarnContext(r.Context(), "encode proxy capabilities", "component", "proxy", "error", err)
}
}
type healthResponse struct {
Status string `json:"status"`
ActiveJobs int `json:"active_jobs"`
@@ -218,12 +185,7 @@ func (s *Server) handleDirectPlay(w http.ResponseWriter, r *http.Request) {
s.tracker.Track(r.Context(), info)
defer s.tracker.Remove(r.Context(), claims.SessionID)
// Serve through the same path the integrated server uses rather than a bare
// http.ServeFile: direct_stream_resume_v1 requires the strong ETag that
// ServeDirectPlay sets before ServeContent (ServeFile sets none, so
// If-Range never validates and a resumed range silently restarts at 200),
// and it carries the rolling write deadline and stream metrics with it.
_ = playback.ServeDirectPlay(w, r, claims.MediaPath)
http.ServeFile(w, r, claims.MediaPath)
}
func (s *Server) handleDownloadFile(w http.ResponseWriter, r *http.Request) {
-20
View File
@@ -1943,26 +1943,6 @@ func (r *FileRepository) GetByPath(ctx context.Context, path string) (*models.Me
return scanMediaFile(r.pool.QueryRow(ctx, query, path))
}
// IsActivePath reports whether path is the exact logical path of a media file
// that is still active in the catalog. Scanner paths are authoritative here:
// they deliberately preserve readable symlinks instead of replacing them with
// their physical targets.
func (r *FileRepository) IsActivePath(ctx context.Context, path string) (bool, error) {
var active bool
err := r.pool.QueryRow(ctx, `
SELECT EXISTS (
SELECT 1
FROM media_files
WHERE file_path = $1
AND missing_since IS NULL
)
`, path).Scan(&active)
if err != nil {
return false, fmt.Errorf("checking active media file path: %w", err)
}
return active, nil
}
// GetByHash retrieves a media file by its file hash.
func (r *FileRepository) GetByHash(ctx context.Context, hash string) (*models.MediaFile, error) {
query := `SELECT ` + fileColumns + ` FROM media_files WHERE file_hash = $1 LIMIT 1`
@@ -1,69 +0,0 @@
package scanner
import (
"context"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestFileRepositoryIsActivePath(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
suffix := time.Now().UnixNano()
var folderID int
if err := pool.QueryRow(ctx, `
INSERT INTO media_folders (type, name, enabled)
VALUES ('movies', 'Active Path Test', true)
RETURNING id
`).Scan(&folderID); err != nil {
t.Fatalf("seed folder: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM media_files WHERE media_folder_id = $1`, folderID)
_, _ = pool.Exec(ctx, `DELETE FROM media_folders WHERE id = $1`, folderID)
})
activePath := fmt.Sprintf("/tmp/silo-active-path-%d.mkv", suffix)
missingPath := fmt.Sprintf("/tmp/silo-missing-path-%d.mkv", suffix)
if _, err := pool.Exec(ctx, `
INSERT INTO media_files (media_folder_id, file_path, missing_since)
VALUES ($1, $2, NULL), ($1, $3, NOW())
`, folderID, activePath, missingPath); err != nil {
t.Fatalf("seed media files: %v", err)
}
repo := NewFileRepository(pool)
for _, test := range []struct {
name string
path string
want bool
}{
{name: "active", path: activePath, want: true},
{name: "missing", path: missingPath, want: false},
{name: "unknown", path: fmt.Sprintf("/tmp/silo-unknown-path-%d.mkv", suffix), want: false},
} {
t.Run(test.name, func(t *testing.T) {
got, err := repo.IsActivePath(ctx, test.path)
if err != nil {
t.Fatalf("IsActivePath: %v", err)
}
if got != test.want {
t.Fatalf("IsActivePath(%q) = %v, want %v", test.path, got, test.want)
}
})
}
}
@@ -1,43 +0,0 @@
package sections
import (
"testing"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestCollectionRailItemsToFetchPreservesLegacyBoundWithoutDefaultSort(t *testing.T) {
items := make([]*models.LibraryCollectionItem, 100_000)
bounded := collectionRailItemsToFetch(items, 24)
if len(bounded) != 24 {
t.Fatalf("unsorted rail selected %d items, want 24", len(bounded))
}
visible, total := unsortedCollectionRailResult(make([]*models.MediaItem, len(bounded)))
if len(visible) != 24 || total != 24 {
t.Fatalf("unsorted rail result = %d items, total %d; want 24/24", len(visible), total)
}
}
func TestCollectionRailQueryAccessIntersectsExplicitLibrary(t *testing.T) {
requestedLibraryID := 7
matching := collectionRailQueryAccess(
catalog.AccessFilter{AllowedLibraryIDs: []int{4, 7}},
&requestedLibraryID,
nil,
)
if len(matching.AllowedLibraryIDs) != 1 || matching.AllowedLibraryIDs[0] != requestedLibraryID {
t.Fatalf("matching access = %v, want [%d]", matching.AllowedLibraryIDs, requestedLibraryID)
}
blocked := collectionRailQueryAccess(
catalog.AccessFilter{AllowedLibraryIDs: []int{4}},
&requestedLibraryID,
nil,
)
if blocked.AllowedLibraryIDs == nil || len(blocked.AllowedLibraryIDs) != 0 {
t.Fatalf("blocked access = %v, want non-nil empty scope", blocked.AllowedLibraryIDs)
}
}
+10 -90
View File
@@ -1309,8 +1309,7 @@ func (f *Fetcher) fetchCollection(ctx context.Context, s ResolvedSection, librar
return []*models.MediaItem{}, 0, nil
}
collection, err := f.CollectionRepo.GetByID(ctx, cfg.LibraryCollectionID)
if err != nil {
if _, err := f.CollectionRepo.GetByID(ctx, cfg.LibraryCollectionID); err != nil {
return nil, 0, fmt.Errorf("loading library collection: %w", err)
}
@@ -1322,28 +1321,13 @@ func (f *Fetcher) fetchCollection(ctx context.Context, s ResolvedSection, librar
return []*models.MediaItem{}, 0, nil
}
// Library collection rails are shared across profiles (see
// isCacheableSectionType), so only the creator's user-agnostic default
// applies here — a viewer's personal override is honored on the collection's
// own browse page.
defaultSort, hasDefaultSort := catalog.ParseCollectionDefaultSort(collection.SortConfig, false)
if hasDefaultSort {
contentIDs := make([]string, 0, len(collectionItems))
for _, item := range collectionItems {
contentIDs = append(contentIDs, item.MediaItemID)
}
queryAccess := collectionRailQueryAccess(filter, libraryID, libraryIDs)
items, total, err := catalog.QueryCollectionItemsBySort(ctx, f.pool, contentIDs, defaultSort, queryAccess, s.ItemLimit, "")
if err != nil {
return nil, 0, err
}
return items, total, nil
limit := s.ItemLimit
if limit <= 0 || limit > len(collectionItems) {
limit = len(collectionItems)
}
selectedCollectionItems := collectionRailItemsToFetch(collectionItems, s.ItemLimit)
contentIDs := make([]string, 0, len(selectedCollectionItems))
for _, item := range selectedCollectionItems {
contentIDs := make([]string, 0, limit)
for _, item := range collectionItems[:limit] {
contentIDs = append(contentIDs, item.MediaItemID)
}
@@ -1366,23 +1350,7 @@ func (f *Fetcher) fetchCollection(ctx context.Context, s ResolvedSection, librar
orderedItems = append(orderedItems, item)
}
// Historically total represented the number of visible rail items, not the
// collection membership. Keep that contract for unsorted rails.
orderedItems, total := unsortedCollectionRailResult(orderedItems)
return orderedItems, total, nil
}
func collectionRailItemsToFetch(items []*models.LibraryCollectionItem, itemLimit int) []*models.LibraryCollectionItem {
if itemLimit > 0 && itemLimit < len(items) {
// Preserve the legacy rail path when no default sort is configured:
// bound the lookup before expanding content IDs into SQL parameters.
return items[:itemLimit]
}
return items
}
func unsortedCollectionRailResult(items []*models.MediaItem) ([]*models.MediaItem, int) {
return items, len(items)
return orderedItems, len(orderedItems), nil
}
// fetchUserCollection resolves a personal (profile-scoped) user collection.
@@ -1479,29 +1447,6 @@ func (f *Fetcher) fetchUserCollection(ctx context.Context, s ResolvedSection, li
contentIDs = append(contentIDs, item.MediaItemID)
}
displayAccess := collectionRailQueryAccess(filter, libraryID, libraryIDs)
displayAccess.UserID = userID
displayAccess.ProfileID = profileID
// A configured default is executed together with the display filter and
// rail limit. Querying before hydration avoids loading an entire large
// collection merely to render a bounded home rail.
if qs, ok := catalog.ParseCollectionDefaultSort([]byte(collection.SortConfig), true); ok {
items, total, err := catalog.QueryCollectionItemsBySort(
ctx,
f.pool,
contentIDs,
qs,
displayAccess,
s.ItemLimit,
collection.DisplayQueryDefinition,
)
if err != nil {
return nil, 0, err
}
return items, total, nil
}
items, err := f.fetchItemsByContentIDs(ctx, contentIDs, libraryID, libraryIDs, filter)
if err != nil {
return nil, 0, err
@@ -1521,6 +1466,9 @@ func (f *Fetcher) fetchUserCollection(ctx context.Context, s ResolvedSection, li
orderedItems = append(orderedItems, item)
}
displayAccess := filter
displayAccess.UserID = userID
displayAccess.ProfileID = profileID
orderedItems, err = catalog.FilterCollectionItemsByDisplayQuery(ctx, f.pool, orderedItems, collection.DisplayQueryDefinition, displayAccess)
if err != nil {
return nil, 0, err
@@ -2796,34 +2744,6 @@ func effectiveFetchLibraryIDs(libraryIDs []int, filter catalog.AccessFilter) []i
return nil
}
// collectionRailQueryAccess expresses the section's explicit library scope as
// an AccessFilter for collection queries routed through QueryExecutor.
func collectionRailQueryAccess(filter catalog.AccessFilter, libraryID *int, libraryIDs []int) catalog.AccessFilter {
result := filter
effectiveLibraryIDs := effectiveFetchLibraryIDs(libraryIDs, filter)
if libraryID == nil {
if effectiveLibraryIDs == nil {
result.AllowedLibraryIDs = nil
} else {
result.AllowedLibraryIDs = append([]int(nil), effectiveLibraryIDs...)
}
return result
}
if effectiveLibraryIDs == nil {
result.AllowedLibraryIDs = []int{*libraryID}
return result
}
for _, id := range effectiveLibraryIDs {
if id == *libraryID {
result.AllowedLibraryIDs = []int{*libraryID}
return result
}
}
result.AllowedLibraryIDs = []int{}
return result
}
func recentSeasonPremiereBadges(seasonNumber, episodeNumber int, airDate *time.Time) []string {
if seasonNumber <= 1 || episodeNumber != 1 || airDate == nil {
return nil
+48 -25
View File
@@ -5,50 +5,54 @@ import (
"os"
"path/filepath"
"strings"
"github.com/Silo-Server/silo-server/internal/models"
)
// InputPathAuthorizer approves a local media input before a node passes it to
// FFmpeg. Implementations must reject protocol URLs and paths outside the
// authoritative media catalog.
// FFmpeg. Implementations must reject protocol URLs and paths outside managed
// library roots.
type InputPathAuthorizer interface {
Allowed(ctx context.Context, path string) (bool, error)
}
type catalogPathSource interface {
IsActivePath(ctx context.Context, path string) (bool, error)
type mediaFolderSource interface {
List(ctx context.Context) ([]*models.MediaFolder, error)
}
// CatalogPathAuthorizer permits only existing regular files whose exact
// logical path is active in the media catalog. The scanner deliberately keeps
// logical paths for readable symlinks, so catalog membership is the correct
// authority: resolving the target and requiring it to remain under the logical
// library root would reject media layouts the scanner explicitly supports.
type CatalogPathAuthorizer struct {
paths catalogPathSource
// MediaRootAuthorizer resolves the deployment's current library roots and
// permits only existing, absolute filesystem paths contained by those roots.
type MediaRootAuthorizer struct {
folders mediaFolderSource
}
// NewCatalogPathAuthorizer creates an FFmpeg input authorizer backed by the
// authoritative media-file catalog.
func NewCatalogPathAuthorizer(paths catalogPathSource) *CatalogPathAuthorizer {
return &CatalogPathAuthorizer{paths: paths}
// NewMediaRootAuthorizer creates an FFmpeg input authorizer backed by the
// authoritative media-folder repository.
func NewMediaRootAuthorizer(folders mediaFolderSource) *MediaRootAuthorizer {
return &MediaRootAuthorizer{folders: folders}
}
// Allowed reports whether path is an active catalog entry that resolves to a
// regular file on this node. os.Stat follows scanner-approved symlinks while
// rejecting dangling links, directories, and other non-regular inputs.
func (a *CatalogPathAuthorizer) Allowed(ctx context.Context, path string) (bool, error) {
if a == nil || a.paths == nil || !plainAbsolutePath(path) {
// Allowed reports whether path resolves inside one of the configured media
// roots. Symlinks are resolved on both sides so a link cannot escape a root.
func (a *MediaRootAuthorizer) Allowed(ctx context.Context, path string) (bool, error) {
if a == nil || a.folders == nil || !plainAbsolutePath(path) {
return false, nil
}
active, err := a.paths.IsActivePath(ctx, path)
folders, err := a.folders.List(ctx)
if err != nil {
return false, err
}
if !active {
return false, nil
for _, folder := range folders {
if folder == nil {
continue
}
for _, root := range folder.Paths {
if existingPathWithinRoot(root, path) {
return true, nil
}
}
}
info, err := os.Stat(path)
return err == nil && info.Mode().IsRegular(), nil
return false, nil
}
func plainAbsolutePath(path string) bool {
@@ -56,6 +60,25 @@ func plainAbsolutePath(path string) bool {
return path != "" && !strings.ContainsRune(path, '\x00') && filepath.IsAbs(path)
}
func existingPathWithinRoot(root, target string) bool {
if !plainAbsolutePath(root) || !plainAbsolutePath(target) {
return false
}
resolvedRoot, err := filepath.EvalSymlinks(filepath.Clean(root))
if err != nil {
return false
}
resolvedTarget, err := filepath.EvalSymlinks(filepath.Clean(target))
if err != nil {
return false
}
info, err := os.Stat(resolvedTarget)
if err != nil || !info.Mode().IsRegular() {
return false
}
return resolvedPathContained(resolvedRoot, resolvedTarget)
}
// pathWithinRoot validates a not-yet-created output by resolving the root and
// target parent. The caller creates the basename only after this check.
func pathWithinRoot(root, target string) bool {
+48 -44
View File
@@ -7,63 +7,41 @@ import (
"path/filepath"
"runtime"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
)
type staticCatalogPaths struct {
active map[string]bool
err error
type staticMediaFolders struct {
folders []*models.MediaFolder
err error
}
func (s staticCatalogPaths) IsActivePath(_ context.Context, path string) (bool, error) {
return s.active[path], s.err
func (s staticMediaFolders) List(context.Context) ([]*models.MediaFolder, error) {
return s.folders, s.err
}
func TestCatalogPathAuthorizerAllowsCataloguedRegularFilesAndSymlinks(t *testing.T) {
mediaPath := filepath.Join(t.TempDir(), "movie.mkv")
func TestMediaRootAuthorizerAllowsOnlyExistingFilesWithinLibraryRoots(t *testing.T) {
root := t.TempDir()
mediaPath := filepath.Join(root, "movie.mkv")
if err := os.WriteFile(mediaPath, []byte("media"), 0o600); err != nil {
t.Fatal(err)
}
target := filepath.Join(t.TempDir(), "outside.mkv")
if err := os.WriteFile(target, []byte("media"), 0o600); err != nil {
t.Fatal(err)
}
link := filepath.Join(t.TempDir(), "linked.mkv")
if err := os.Symlink(target, link); err != nil {
t.Skipf("symlinks not supported on this platform: %v", err)
}
authorizer := NewCatalogPathAuthorizer(staticCatalogPaths{active: map[string]bool{
mediaPath: true,
link: true,
}})
authorizer := NewMediaRootAuthorizer(staticMediaFolders{
folders: []*models.MediaFolder{{Paths: []string{root}}},
})
for _, path := range []string{mediaPath, link} {
allowed, err := authorizer.Allowed(context.Background(), path)
if err != nil || !allowed {
t.Fatalf("approved media path %q: allowed = %v, err = %v", path, allowed, err)
}
allowed, err := authorizer.Allowed(context.Background(), mediaPath)
if err != nil || !allowed {
t.Fatalf("approved media path: allowed = %v, err = %v", allowed, err)
}
}
func TestCatalogPathAuthorizerRejectsUnsafeOrUncataloguedInputs(t *testing.T) {
existingUncatalogued := filepath.Join(t.TempDir(), "outside.mkv")
if err := os.WriteFile(existingUncatalogued, []byte("media"), 0o600); err != nil {
t.Fatal(err)
}
missingCatalogued := filepath.Join(t.TempDir(), "missing.mkv")
directoryCatalogued := t.TempDir()
authorizer := NewCatalogPathAuthorizer(staticCatalogPaths{active: map[string]bool{
missingCatalogued: true,
directoryCatalogued: true,
}})
for _, candidate := range []string{
"relative/movie.mkv",
"http://example.test/movie.mkv",
"file:" + existingUncatalogued,
"concat:" + existingUncatalogued + "|" + existingUncatalogued,
existingUncatalogued,
missingCatalogued,
directoryCatalogued,
"file:" + mediaPath,
"concat:" + mediaPath + "|" + mediaPath,
filepath.Join(t.TempDir(), "outside.mkv"),
filepath.Join(root, "missing.mkv"),
} {
allowed, err := authorizer.Allowed(context.Background(), candidate)
if err != nil {
@@ -75,9 +53,35 @@ func TestCatalogPathAuthorizerRejectsUnsafeOrUncataloguedInputs(t *testing.T) {
}
}
func TestCatalogPathAuthorizerPropagatesRepositoryErrors(t *testing.T) {
func TestMediaRootAuthorizerRejectsSymlinkEscape(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlink semantics differ on Windows")
}
root := t.TempDir()
outside := filepath.Join(t.TempDir(), "outside.mkv")
if err := os.WriteFile(outside, []byte("media"), 0o600); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "linked.mkv")
if err := os.Symlink(outside, link); err != nil {
t.Fatal(err)
}
authorizer := NewMediaRootAuthorizer(staticMediaFolders{
folders: []*models.MediaFolder{{Paths: []string{root}}},
})
allowed, err := authorizer.Allowed(context.Background(), link)
if err != nil {
t.Fatal(err)
}
if allowed {
t.Fatal("symlink escape was allowed")
}
}
func TestMediaRootAuthorizerPropagatesRepositoryErrors(t *testing.T) {
wantErr := errors.New("database unavailable")
authorizer := NewCatalogPathAuthorizer(staticCatalogPaths{err: wantErr})
authorizer := NewMediaRootAuthorizer(staticMediaFolders{err: wantErr})
allowed, err := authorizer.Allowed(context.Background(), "/media/movie.mkv")
if allowed || !errors.Is(err, wantErr) {
t.Fatalf("allowed = %v, err = %v", allowed, err)
+1 -1
View File
@@ -375,7 +375,7 @@ func TestHandleDownloadPrepareRejectsUnavailableConfig(t *testing.T) {
func TestHandleStartRejectsUnapprovedInputPath(t *testing.T) {
server := newTestServer(t)
server.inputPaths = NewCatalogPathAuthorizer(staticCatalogPaths{})
server.inputPaths = NewMediaRootAuthorizer(staticMediaFolders{})
body := []byte(`{"session_id":"unsafe-input","input_path":"http://example.test/movie.mkv"}`)
req := httptest.NewRequest(http.MethodPost, "/transcode/start", bytes.NewReader(body))
-82
View File
@@ -1,82 +0,0 @@
package userdb
import (
"database/sql"
"fmt"
"github.com/Silo-Server/silo-server/internal/userstore"
)
// CollectionSortPreference is an alias for the canonical type in userstore.
type CollectionSortPreference = userstore.CollectionSortPreference
// SetCollectionSortPreference creates or replaces a profile's sort override for
// a collection. Timestamps should be ISO 8601 UTC strings.
func SetCollectionSortPreference(db *sql.DB, pref CollectionSortPreference) error {
if pref.UpdatedAt == "" {
pref.UpdatedAt = nowUTC()
}
_, err := db.Exec(`
INSERT INTO collection_sort_preferences (
profile_id, collection_kind, collection_id, sort_field, sort_order, updated_at
) VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(profile_id, collection_kind, collection_id) DO UPDATE SET
sort_field = excluded.sort_field,
sort_order = excluded.sort_order,
updated_at = excluded.updated_at`,
pref.ProfileID,
pref.CollectionKind,
pref.CollectionID,
pref.SortField,
pref.SortOrder,
pref.UpdatedAt,
)
if err != nil {
return fmt.Errorf("setting collection sort preference for profile %q collection %q/%q: %w",
pref.ProfileID, pref.CollectionKind, pref.CollectionID, err)
}
return nil
}
// GetCollectionSortPreference returns nil (not an error) when the profile has
// never changed this collection's sort. A preference with an empty SortField
// means the viewer explicitly chose the collection's own source order.
func GetCollectionSortPreference(db *sql.DB, profileID, collectionKind, collectionID string) (*CollectionSortPreference, error) {
var pref CollectionSortPreference
err := db.QueryRow(`
SELECT profile_id, collection_kind, collection_id, sort_field, sort_order, updated_at
FROM collection_sort_preferences
WHERE profile_id = ? AND collection_kind = ? AND collection_id = ?`,
profileID, collectionKind, collectionID,
).Scan(
&pref.ProfileID,
&pref.CollectionKind,
&pref.CollectionID,
&pref.SortField,
&pref.SortOrder,
&pref.UpdatedAt,
)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("getting collection sort preference for profile %q collection %q/%q: %w",
profileID, collectionKind, collectionID, err)
}
return &pref, nil
}
// ClearCollectionSortPreference drops the override so the collection falls back
// to its creator-configured default.
func ClearCollectionSortPreference(db *sql.DB, profileID, collectionKind, collectionID string) error {
_, err := db.Exec(`
DELETE FROM collection_sort_preferences
WHERE profile_id = ? AND collection_kind = ? AND collection_id = ?`,
profileID, collectionKind, collectionID,
)
if err != nil {
return fmt.Errorf("clearing collection sort preference for profile %q collection %q/%q: %w",
profileID, collectionKind, collectionID, err)
}
return nil
}
-5
View File
@@ -243,11 +243,6 @@ func DeleteCollection(db *sql.DB, id string) error {
if _, err := tx.Exec(`DELETE FROM personal_collection_items WHERE collection_id = ?`, id); err != nil {
return err
}
if _, err := tx.Exec(`
DELETE FROM collection_sort_preferences
WHERE collection_kind = 'user' AND collection_id = ?`, id); err != nil {
return err
}
if _, err := tx.Exec(`DELETE FROM personal_collections WHERE id = ?`, id); err != nil {
return err
}
-12
View File
@@ -27,14 +27,6 @@ func TestSQLiteProgressSince(t *testing.T) {
storetest.RunProgressSince(t, newConformanceStore)
}
// TestSQLiteMarkWatchedBatch runs the batch mark-watched conformance test
// (series/season mark-watched) against the real SQLite backend. The Postgres
// backend runs the same suite in internal/userstore/pgstore, which is what
// keeps the two transactional implementations from drifting.
func TestSQLiteMarkWatchedBatch(t *testing.T) {
storetest.RunMarkWatchedBatch(t, newConformanceStore)
}
// TestSQLiteSettingValues runs the canonical settings-contract storage
// conformance tests against the per-user SQLite backend. The Postgres backend
// runs the same suite in internal/userstore/pgstore, which is what keeps the two
@@ -50,10 +42,6 @@ func TestSQLiteJellycompatDisplayPrefs(t *testing.T) {
storetest.RunJellycompatDisplayPrefs(t, newConformanceStore)
}
func TestSQLiteCollectionSortPreferences(t *testing.T) {
storetest.RunCollectionSortPreferences(t, newConformanceStore)
}
func TestSQLiteAddFavoriteAtReportsInsertion(t *testing.T) {
ctx := context.Background()
store := newConformanceStore(t)
+1 -20
View File
@@ -5,7 +5,7 @@ import (
"fmt"
)
const schemaVersion = 19
const schemaVersion = 18
func runMigrations(db *sql.DB) error {
version, err := userVersion(db)
@@ -178,28 +178,9 @@ func runMigrations(db *sql.DB) error {
}
}
if version < 19 {
if err := migrateToV19(tx); err != nil {
return err
}
if _, err := tx.Exec("PRAGMA user_version = 19"); err != nil {
return fmt.Errorf("setting sqlite user_version 19: %w", err)
}
}
return tx.Commit()
}
// migrateToV19 adds the per-profile collection sort override table. An empty
// sort_field is a real choice ("keep this collection's own source order") and
// is distinct from having no row.
func migrateToV19(tx *sql.Tx) error {
if _, err := tx.Exec(collectionSortPreferencesSchema); err != nil {
return fmt.Errorf("creating collection_sort_preferences: %w", err)
}
return nil
}
// migrateToV18 seeds the family-neutral navigation shortcut catalog from the
// existing web sidebar pins. InitSchema has already rebuilt
// user_setting_values with the profile_client identity before this versioned
-42
View File
@@ -1,42 +0,0 @@
package userdb
import (
"database/sql"
"testing"
)
// An existing v18 store from main must gain the collection sort preferences
// table without replaying or replacing any of main's earlier migrations.
func TestMigrateToV19AddsCollectionSortPreferences(t *testing.T) {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
if err := InitSchema(db); err != nil {
t.Fatalf("InitSchema: %v", err)
}
if _, err := db.Exec(`DROP TABLE IF EXISTS collection_sort_preferences`); err != nil {
t.Fatalf("drop table: %v", err)
}
if _, err := db.Exec("PRAGMA user_version = 18"); err != nil {
t.Fatalf("set user_version: %v", err)
}
if err := runMigrations(db); err != nil {
t.Fatalf("runMigrations: %v", err)
}
version, err := userVersion(db)
if err != nil {
t.Fatalf("userVersion: %v", err)
}
if version != schemaVersion {
t.Fatalf("user_version = %d, want %d", version, schemaVersion)
}
var tableName string
if err := db.QueryRow(`SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'collection_sort_preferences'`).Scan(&tableName); err != nil {
t.Fatalf("collection_sort_preferences table: %v", err)
}
}
-12
View File
@@ -292,17 +292,6 @@ func DeleteProfile(db *sql.DB, id string) error {
}
defer tx.Rollback() //nolint:errcheck
// Preferences belong to viewers, not creators, so deleting only this
// profile's rows below would leave other profiles' overrides pointing at
// personal collections that are about to be removed.
if _, err := tx.Exec(`
DELETE FROM collection_sort_preferences
WHERE collection_kind = 'user' AND collection_id IN (
SELECT id FROM personal_collections WHERE creator_profile_id = ?
)`, id); err != nil {
return fmt.Errorf("deleting collection sort preferences for profile %s: %w", id, err)
}
// Delete personal_collection_items for collections owned by this profile.
_, err = tx.Exec(`
DELETE FROM personal_collection_items
@@ -323,7 +312,6 @@ func DeleteProfile(db *sql.DB, id string) error {
"favorites",
"watchlist",
"watch_progress",
"collection_sort_preferences",
"personal_collections",
"profile_allowed_libraries",
"series_playback_preferences",
+19 -136
View File
@@ -1,7 +1,6 @@
package userdb
import (
"context"
"database/sql"
"encoding/json"
"fmt"
@@ -172,9 +171,14 @@ func SetProgressIfNewer(db *sql.DB, profileID, mediaItemID string, position, dur
return rows > 0, nil
}
// markWatchedProgressSQL upserts one completed progress row, honoring the
// hidden-history watermark.
const markWatchedProgressSQL = `
// MarkWatched creates or replaces progress with a completed entry.
func MarkWatched(db *sql.DB, profileID, mediaItemID string, duration float64) error {
if duration < 0 {
duration = 0
}
now := nowUTC()
query := `
INSERT INTO watch_progress (profile_id, media_item_id, position_seconds, duration_seconds, completed, updated_at)
SELECT ?, ?, 0, ?, 1, ` + visibleTimestampSQL + `
FROM (SELECT 1) seed
@@ -189,133 +193,7 @@ const markWatchedProgressSQL = `
updated_at = excluded.updated_at,
event_at = excluded.updated_at
`
// markWatchedBatchProgressSQL is markWatchedProgressSQL with one difference: a
// zero duration leaves any known duration in place. The batch path now also
// serves jellycompat's series mark-played, which supplies no durations and
// previously ran through MarkProgressBatch without touching the column.
const markWatchedBatchProgressSQL = `
INSERT INTO watch_progress (profile_id, media_item_id, position_seconds, duration_seconds, completed, updated_at)
SELECT ?, ?, 0, ?, 1, ` + visibleTimestampSQL + `
FROM (SELECT 1) seed
LEFT JOIN hidden_history_items hhi
ON hhi.profile_id = ?
AND hhi.media_item_id = ?
WHERE true
ON CONFLICT(profile_id, media_item_id) DO UPDATE SET
position_seconds = 0,
duration_seconds = CASE
WHEN excluded.duration_seconds > 0 THEN excluded.duration_seconds
ELSE watch_progress.duration_seconds
END,
completed = 1,
updated_at = excluded.updated_at,
event_at = excluded.updated_at
`
// addVisibleHistorySQL inserts one history row at a watermark-adjusted
// timestamp and returns the timestamp actually stored.
const addVisibleHistorySQL = `
INSERT INTO watch_history (id, profile_id, media_item_id, watched_at, duration_seconds, completed, source, watch_identity)
SELECT ?, ?, ?, ` + visibleTimestampSQL + `, ?, ?, ?, ?
FROM (SELECT 1) seed
LEFT JOIN hidden_history_items hhi
ON hhi.profile_id = ?
AND hhi.media_item_id = ?
WHERE true
RETURNING watched_at
`
// MarkWatchedBatch marks every target watched and records its history entry
// inside a single transaction, so a canceled series mark rolls back to
// "nothing marked" rather than stranding a partial subset. SQLite round-trips
// are local and cheap, so this reuses the same per-row statements as the
// single-item path and buys atomicity rather than fewer queries.
//
// Unlike most of this package it honors ctx: cancellation safety is the whole
// point of the transaction, and a caller that disconnects mid-mark must leave
// the series untouched rather than half-watched.
func MarkWatchedBatch(
ctx context.Context,
db *sql.DB,
profileID string,
targets []userstore.MarkWatchedTarget,
entries []userstore.WatchHistoryEntry,
) ([]userstore.WatchHistoryEntry, error) {
if len(targets) == 0 {
return nil, nil
}
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return nil, fmt.Errorf("begin mark watched batch: %w", err)
}
defer tx.Rollback() //nolint:errcheck // rollback after commit is a no-op
now := nowUTC()
seen := make(map[string]struct{}, len(targets))
for _, target := range targets {
mediaItemID := strings.TrimSpace(target.MediaItemID)
if mediaItemID == "" {
continue
}
if _, ok := seen[mediaItemID]; ok {
continue
}
seen[mediaItemID] = struct{}{}
duration := target.DurationSeconds
if duration < 0 {
duration = 0
}
if _, err := tx.ExecContext(ctx, markWatchedBatchProgressSQL,
profileID, mediaItemID, duration, now, now, profileID, mediaItemID,
); err != nil {
return nil, fmt.Errorf("marking watched batch: %w", err)
}
}
written := make([]userstore.WatchHistoryEntry, 0, len(entries))
for _, entry := range entries {
if strings.TrimSpace(entry.MediaItemID) == "" {
continue
}
if entry.ID == "" {
entry.ID = generateUUID()
}
if entry.WatchedAt == "" {
entry.WatchedAt = now
}
if entry.Source == "" {
entry.Source = userstore.WatchHistorySourceLegacy
}
identityJSON, err := json.Marshal(entry.Identity)
if err != nil {
return nil, fmt.Errorf("marshaling watch identity: %w", err)
}
if err := tx.QueryRowContext(ctx, addVisibleHistorySQL,
entry.ID, entry.ProfileID, entry.MediaItemID, entry.WatchedAt, entry.WatchedAt,
entry.DurationSeconds, entry.Completed, entry.Source, string(identityJSON),
entry.ProfileID, entry.MediaItemID,
).Scan(&entry.WatchedAt); err != nil {
return nil, fmt.Errorf("adding visible history batch: %w", err)
}
written = append(written, entry)
}
if err := tx.Commit(); err != nil {
return nil, fmt.Errorf("commit mark watched batch: %w", err)
}
return written, nil
}
// MarkWatched creates or replaces progress with a completed entry.
func MarkWatched(db *sql.DB, profileID, mediaItemID string, duration float64) error {
if duration < 0 {
duration = 0
}
now := nowUTC()
_, err := db.Exec(markWatchedProgressSQL, profileID, mediaItemID, duration, now, now, profileID, mediaItemID)
_, err := db.Exec(query, profileID, mediaItemID, duration, now, now, profileID, mediaItemID)
if err != nil {
return fmt.Errorf("marking watched: %w", err)
}
@@ -706,11 +584,16 @@ func AddVisibleHistory(db *sql.DB, entry WatchHistoryEntry) (WatchHistoryEntry,
if err != nil {
return entry, fmt.Errorf("marshaling watch identity: %w", err)
}
if err := db.QueryRow(addVisibleHistorySQL,
entry.ID, entry.ProfileID, entry.MediaItemID, entry.WatchedAt, entry.WatchedAt,
entry.DurationSeconds, entry.Completed, entry.Source, string(identityJSON),
entry.ProfileID, entry.MediaItemID,
).Scan(&entry.WatchedAt); err != nil {
if err := db.QueryRow(`
INSERT INTO watch_history (id, profile_id, media_item_id, watched_at, duration_seconds, completed, source, watch_identity)
SELECT ?, ?, ?, `+visibleTimestampSQL+`, ?, ?, ?, ?
FROM (SELECT 1) seed
LEFT JOIN hidden_history_items hhi
ON hhi.profile_id = ?
AND hhi.media_item_id = ?
WHERE true
RETURNING watched_at
`, entry.ID, entry.ProfileID, entry.MediaItemID, entry.WatchedAt, entry.WatchedAt, entry.DurationSeconds, entry.Completed, entry.Source, string(identityJSON), entry.ProfileID, entry.MediaItemID).Scan(&entry.WatchedAt); err != nil {
return entry, fmt.Errorf("adding visible history entry: %w", err)
}
return entry, nil
-28
View File
@@ -124,18 +124,6 @@ CREATE TABLE IF NOT EXISTS personal_collections (
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS collection_sort_preferences (
profile_id TEXT NOT NULL,
collection_kind TEXT NOT NULL,
collection_id TEXT NOT NULL,
sort_field TEXT NOT NULL DEFAULT '',
sort_order TEXT NOT NULL DEFAULT '',
updated_at TEXT NOT NULL,
PRIMARY KEY (profile_id, collection_kind, collection_id),
CHECK (collection_kind IN ('library', 'user')),
CHECK (sort_order IN ('', 'asc', 'desc'))
);
CREATE TABLE IF NOT EXISTS personal_collection_items (
collection_id TEXT NOT NULL,
media_item_id TEXT NOT NULL,
@@ -507,22 +495,6 @@ CREATE INDEX user_setting_values_library_idx
return nil
}
// collectionSortPreferencesSchema is kept as its own const (rather than only
// inlined in Schema) so migrateToV19 can create the table on databases that
// predate it. collection_kind separates the 'library' and 'user' id spaces.
const collectionSortPreferencesSchema = `
CREATE TABLE IF NOT EXISTS collection_sort_preferences (
profile_id TEXT NOT NULL,
collection_kind TEXT NOT NULL,
collection_id TEXT NOT NULL,
sort_field TEXT NOT NULL DEFAULT '',
sort_order TEXT NOT NULL DEFAULT '',
updated_at TEXT NOT NULL,
PRIMARY KEY (profile_id, collection_kind, collection_id),
CHECK (collection_kind IN ('library', 'user')),
CHECK (sort_order IN ('', 'asc', 'desc'))
);`
// watchProgressSyncTriggers stamps the server-owned cursor (synced_seq) on every
// watch_progress write and owns the LWW key: event_at defaults to the write
// time, and an UPDATE that changed updated_at without explicitly changing
-19
View File
@@ -24,7 +24,6 @@ func NewSQLiteUserStore(db *sql.DB) *SQLiteUserStore {
// Compile-time interface check.
var _ userstore.UserStore = (*SQLiteUserStore)(nil)
var _ userstore.DeviceRegistry = (*SQLiteUserStore)(nil)
var _ userstore.WatchedBatchWriter = (*SQLiteUserStore)(nil)
// --- Profiles ---
@@ -82,12 +81,6 @@ func (s *SQLiteUserStore) ClearProgress(_ context.Context, profileID, mediaItemI
return ClearProgress(s.db, profileID, mediaItemID)
}
// MarkWatchedBatch is one of the few paths here that forwards ctx: the write
// is transactional precisely so a canceled request marks nothing.
func (s *SQLiteUserStore) MarkWatchedBatch(ctx context.Context, profileID string, targets []userstore.MarkWatchedTarget, entries []userstore.WatchHistoryEntry) ([]userstore.WatchHistoryEntry, error) {
return MarkWatchedBatch(ctx, s.db, profileID, targets, entries)
}
func (s *SQLiteUserStore) MarkProgressBatch(_ context.Context, profileID string, mediaItemIDs []string, updatedAt time.Time) error {
return MarkProgressBatch(s.db, profileID, mediaItemIDs, updatedAt)
}
@@ -429,18 +422,6 @@ func (s *SQLiteUserStore) DeleteSeriesPlaybackPreference(_ context.Context, prof
return DeleteSeriesPlaybackPreference(s.db, profileID, seriesID)
}
func (s *SQLiteUserStore) SetCollectionSortPreference(_ context.Context, pref userstore.CollectionSortPreference) error {
return SetCollectionSortPreference(s.db, pref)
}
func (s *SQLiteUserStore) GetCollectionSortPreference(_ context.Context, profileID, collectionKind, collectionID string) (*userstore.CollectionSortPreference, error) {
return GetCollectionSortPreference(s.db, profileID, collectionKind, collectionID)
}
func (s *SQLiteUserStore) ClearCollectionSortPreference(_ context.Context, profileID, collectionKind, collectionID string) error {
return ClearCollectionSortPreference(s.db, profileID, collectionKind, collectionID)
}
func (s *SQLiteUserStore) GetLibraryPlaybackPreference(_ context.Context, profileID string, libraryID int) (*userstore.LibraryPlaybackPreference, error) {
return GetLibraryPlaybackPreference(s.db, profileID, libraryID)
}
@@ -1,62 +0,0 @@
package userstore
import (
"context"
"log/slog"
"sort"
)
// CollectionSortPreferenceCleaner removes cross-user viewer preferences when
// a shared library collection is deleted. Library collections live in the
// central catalog while preferences may live in either shared PostgreSQL or
// per-user SQLite, so database foreign keys cannot provide this cleanup.
type CollectionSortPreferenceCleaner struct {
users UserLister
stores UserStoreProvider
logger *slog.Logger
}
// NewCollectionSortPreferenceCleaner wires the cross-user cleanup sweep.
func NewCollectionSortPreferenceCleaner(users UserLister, stores UserStoreProvider) *CollectionSortPreferenceCleaner {
return &CollectionSortPreferenceCleaner{
users: users,
stores: stores,
logger: slog.Default().With("component", "userstore.collection_sort_cleanup"),
}
}
// DeleteForCollection clears every profile's preference for one collection.
// Failures are logged per user and do not undo the owning collection delete;
// stale rows are unreachable because preference writes and reads both require
// the collection to remain accessible.
func (c *CollectionSortPreferenceCleaner) DeleteForCollection(ctx context.Context, kind, collectionID string) {
if c == nil || c.users == nil || c.stores == nil {
return
}
users, err := c.users.List(ctx)
if err != nil {
c.logger.WarnContext(ctx, "collection sort cleanup: list users failed", "kind", kind, "collection_id", collectionID, "error", err)
return
}
sort.Slice(users, func(i, j int) bool { return users[i].ID < users[j].ID })
for _, user := range users {
if ctx.Err() != nil {
return
}
store, err := c.stores.ForUser(ctx, user.ID)
if err != nil {
c.logger.WarnContext(ctx, "collection sort cleanup: open user store failed", "kind", kind, "collection_id", collectionID, "user_id", user.ID, "error", err)
continue
}
profiles, err := store.ListProfiles(ctx)
if err != nil {
c.logger.WarnContext(ctx, "collection sort cleanup: list profiles failed", "kind", kind, "collection_id", collectionID, "user_id", user.ID, "error", err)
continue
}
for _, profile := range profiles {
if err := store.ClearCollectionSortPreference(ctx, profile.ID, kind, collectionID); err != nil {
c.logger.WarnContext(ctx, "collection sort cleanup: delete failed", "kind", kind, "collection_id", collectionID, "user_id", user.ID, "profile_id", profile.ID, "error", err)
}
}
}
}
@@ -1,62 +0,0 @@
package userstore_test
import (
"context"
"path/filepath"
"testing"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/userdb"
"github.com/Silo-Server/silo-server/internal/userstore"
)
type cleanupUserLister struct {
users []*models.User
}
func (l cleanupUserLister) List(context.Context) ([]*models.User, error) { return l.users, nil }
type cleanupStoreProvider struct {
stores map[int]userstore.UserStore
}
func (p cleanupStoreProvider) ForUser(_ context.Context, userID int) (userstore.UserStore, error) {
return p.stores[userID], nil
}
func (cleanupStoreProvider) Close() error { return nil }
func TestCollectionSortPreferenceCleanerDeletesAcrossUsers(t *testing.T) {
ctx := context.Background()
provider := cleanupStoreProvider{stores: make(map[int]userstore.UserStore)}
for _, userID := range []int{1, 2} {
db, err := userdb.NewUserDB(filepath.Join(t.TempDir(), "user.db"), userID)
if err != nil {
t.Fatalf("open user %d database: %v", userID, err)
}
t.Cleanup(func() { _ = db.Close() })
store := userdb.NewSQLiteUserStore(db.DB)
profileID := "profile-" + string(rune('0'+userID))
if err := store.CreateProfile(ctx, userstore.Profile{ID: profileID, Name: profileID}); err != nil {
t.Fatalf("create profile: %v", err)
}
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: profileID, CollectionKind: userstore.CollectionKindLibrary,
CollectionID: "library-collection", SortField: "title", SortOrder: "asc",
}); err != nil {
t.Fatalf("set preference: %v", err)
}
provider.stores[userID] = store
}
cleaner := userstore.NewCollectionSortPreferenceCleaner(cleanupUserLister{users: []*models.User{{ID: 1}, {ID: 2}}}, provider)
cleaner.DeleteForCollection(ctx, userstore.CollectionKindLibrary, "library-collection")
for userID, store := range provider.stores {
profileID := "profile-" + string(rune('0'+userID))
pref, err := store.GetCollectionSortPreference(ctx, profileID, userstore.CollectionKindLibrary, "library-collection")
if err != nil || pref != nil {
t.Fatalf("user %d preference = %+v, err = %v; want nil", userID, pref, err)
}
}
}
@@ -1,85 +0,0 @@
package pgstore
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/Silo-Server/silo-server/internal/userstore"
)
func (s *PostgresUserStore) SetCollectionSortPreference(ctx context.Context, pref userstore.CollectionSortPreference) error {
if pref.UpdatedAt == "" {
pref.UpdatedAt = nowUTC()
}
_, err := s.pool.Exec(ctx, `
INSERT INTO user_collection_sort_preferences (
user_id, profile_id, collection_kind, collection_id, sort_field, sort_order, updated_at
) VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT(user_id, profile_id, collection_kind, collection_id) DO UPDATE SET
sort_field = excluded.sort_field,
sort_order = excluded.sort_order,
updated_at = excluded.updated_at`,
s.userID,
pref.ProfileID,
pref.CollectionKind,
pref.CollectionID,
pref.SortField,
pref.SortOrder,
pref.UpdatedAt,
)
if err != nil {
return fmt.Errorf("setting collection sort preference for profile %q collection %q/%q: %w",
pref.ProfileID, pref.CollectionKind, pref.CollectionID, err)
}
return nil
}
// GetCollectionSortPreference returns (nil, nil) when the profile has never
// changed this collection's sort. A returned preference with an empty SortField
// means the viewer explicitly chose the collection's own source order.
func (s *PostgresUserStore) GetCollectionSortPreference(ctx context.Context, profileID, collectionKind, collectionID string) (*userstore.CollectionSortPreference, error) {
var pref userstore.CollectionSortPreference
var updatedAt time.Time
err := s.pool.QueryRow(ctx, `
SELECT profile_id, collection_kind, collection_id, sort_field, sort_order, updated_at
FROM user_collection_sort_preferences
WHERE user_id = $1 AND profile_id = $2 AND collection_kind = $3 AND collection_id = $4`,
s.userID, profileID, collectionKind, collectionID,
).Scan(
&pref.ProfileID,
&pref.CollectionKind,
&pref.CollectionID,
&pref.SortField,
&pref.SortOrder,
&updatedAt,
)
if errors.Is(err, pgx.ErrNoRows) {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("getting collection sort preference for profile %q collection %q/%q: %w",
profileID, collectionKind, collectionID, err)
}
pref.UpdatedAt = updatedAt.UTC().Format(time.RFC3339)
return &pref, nil
}
// ClearCollectionSortPreference drops the override so the collection falls back
// to its creator-configured default. Distinct from storing an empty SortField,
// which pins the viewer to source order.
func (s *PostgresUserStore) ClearCollectionSortPreference(ctx context.Context, profileID, collectionKind, collectionID string) error {
_, err := s.pool.Exec(ctx, `
DELETE FROM user_collection_sort_preferences
WHERE user_id = $1 AND profile_id = $2 AND collection_kind = $3 AND collection_id = $4`,
s.userID, profileID, collectionKind, collectionID,
)
if err != nil {
return fmt.Errorf("clearing collection sort preference for profile %q collection %q/%q: %w",
profileID, collectionKind, collectionID, err)
}
return nil
}
@@ -365,11 +365,6 @@ func (s *PostgresUserStore) DeleteCollection(ctx context.Context, id string) err
if _, err := tx.Exec(ctx, `DELETE FROM user_personal_collection_items WHERE user_id = $1 AND collection_id = $2`, s.userID, id); err != nil {
return fmt.Errorf("deleting collection items: %w", err)
}
if _, err := tx.Exec(ctx, `
DELETE FROM user_collection_sort_preferences
WHERE user_id = $1 AND collection_kind = 'user' AND collection_id = $2`, s.userID, id); err != nil {
return fmt.Errorf("deleting collection sort preferences: %w", err)
}
if _, err := tx.Exec(ctx, `DELETE FROM user_personal_collections WHERE user_id = $1 AND id = $2`, s.userID, id); err != nil {
return fmt.Errorf("deleting collection: %w", err)
}
@@ -58,42 +58,6 @@ func TestPostgresProgressSince(t *testing.T) {
})
}
// TestPostgresMarkWatchedBatch runs the batch mark-watched conformance test
// (series/season mark-watched) against the Postgres backend. The per-user
// SQLite backend runs the same suite in internal/userdb, which is what keeps
// the two transactional implementations from drifting. Skips unless
// SILO_TEST_DATABASE_URL is set.
func TestPostgresMarkWatchedBatch(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
storetest.RunMarkWatchedBatch(t, func(t *testing.T) userstore.UserStore {
var userID int
if err := pool.QueryRow(ctx,
`INSERT INTO users (username, role) VALUES ($1, 'user') RETURNING id`,
fmt.Sprintf("conf-markwatched-%d", time.Now().UnixNano()),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, `DELETE FROM user_watch_history WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_history_hidden_items WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_watch_progress WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM user_profiles WHERE user_id = $1`, userID)
_, _ = pool.Exec(ctx, `DELETE FROM users WHERE id = $1`, userID)
})
return newStore(pool, userID)
})
}
// TestPostgresSettingValues runs the canonical settings-contract storage
// conformance tests against the Postgres backend. The per-user SQLite backend
// runs the same suite in internal/userdb, which is what keeps the two from
@@ -207,40 +171,3 @@ func TestPostgresJellycompatDisplayPrefs(t *testing.T) {
return newStore(pool, userID)
})
}
func TestPostgresCollectionSortPreferences(t *testing.T) {
dsn := os.Getenv("SILO_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("SILO_TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Fatalf("connect test database: %v", err)
}
t.Cleanup(pool.Close)
var tableName *string
if err := pool.QueryRow(ctx,
`SELECT to_regclass('public.user_collection_sort_preferences')::text`).Scan(&tableName); err != nil {
t.Fatalf("check preference table: %v", err)
}
if tableName == nil || *tableName == "" {
t.Skip("collection sort preference migration has not been applied")
}
storetest.RunCollectionSortPreferences(t, func(t *testing.T) userstore.UserStore {
var userID int
if err := pool.QueryRow(ctx,
`INSERT INTO users (username, role) VALUES ($1, 'user') RETURNING id`,
fmt.Sprintf("sort-pref-conf-%d", time.Now().UnixNano()),
).Scan(&userID); err != nil {
t.Fatalf("seed user: %v", err)
}
t.Cleanup(func() {
deleteUserAssertingCascade(t, pool, userID,
"user_collection_sort_preferences", "user_profiles")
})
return newStore(pool, userID)
})
}
-1
View File
@@ -19,7 +19,6 @@ type PostgresUserStore struct {
// Compile-time interface check.
var _ userstore.UserStore = (*PostgresUserStore)(nil)
var _ userstore.DeviceRegistry = (*PostgresUserStore)(nil)
var _ userstore.WatchedBatchWriter = (*PostgresUserStore)(nil)
// newStore creates a PostgresUserStore scoped to a user.
func newStore(pool *pgxpool.Pool, userID int) *PostgresUserStore {
-11
View File
@@ -282,16 +282,6 @@ func (s *PostgresUserStore) DeleteProfile(ctx context.Context, id string) error
}
defer tx.Rollback(ctx) //nolint:errcheck
// Preferences belong to viewers, not creators, so remove every override
// for collections owned by the profile before those collections disappear.
if _, err := tx.Exec(ctx, `
DELETE FROM user_collection_sort_preferences
WHERE user_id = $1 AND collection_kind = 'user' AND collection_id IN (
SELECT id FROM user_personal_collections WHERE user_id = $1 AND profile_id = $2
)`, s.userID, id); err != nil {
return fmt.Errorf("deleting collection sort preferences for profile %s: %w", id, err)
}
_, err = tx.Exec(ctx, `
DELETE FROM user_personal_collection_items
WHERE user_id = $1 AND collection_id IN (
@@ -309,7 +299,6 @@ func (s *PostgresUserStore) DeleteProfile(ctx context.Context, id string) error
"user_favorites",
"user_watchlist",
"user_watch_progress",
"user_collection_sort_preferences",
"user_personal_collections",
"user_series_playback_preferences",
"user_library_playback_preferences",
-203
View File
@@ -316,209 +316,6 @@ func (s *PostgresUserStore) ClearProgress(ctx context.Context, profileID, mediaI
return nil
}
// MarkWatchedBatch marks every target watched and inserts its history row in
// one transaction, so a series mark either lands whole or not at all. The
// prior per-episode loop committed each episode separately, which left a large
// series half-marked whenever the client disconnected mid-request.
//
// Two statements rather than a reuse of MarkProgressBatch: that one hardcodes
// duration_seconds = 0, and the manual mark-watched path must preserve each
// episode's real duration. Both statements carry the same
// user_history_hidden_items watermark logic as their single-row counterparts
// (MarkWatched, AddVisibleHistory) so a mark after a history removal stays
// visible.
func (s *PostgresUserStore) MarkWatchedBatch(
ctx context.Context,
profileID string,
targets []userstore.MarkWatchedTarget,
entries []userstore.WatchHistoryEntry,
) ([]userstore.WatchHistoryEntry, error) {
if len(targets) == 0 {
return nil, nil
}
mediaItemIDs := make([]string, 0, len(targets))
durations := make([]float64, 0, len(targets))
seen := make(map[string]struct{}, len(targets))
for _, target := range targets {
mediaItemID := strings.TrimSpace(target.MediaItemID)
if mediaItemID == "" {
continue
}
if _, ok := seen[mediaItemID]; ok {
continue
}
seen[mediaItemID] = struct{}{}
duration := target.DurationSeconds
if duration < 0 {
duration = 0
}
mediaItemIDs = append(mediaItemIDs, mediaItemID)
durations = append(durations, duration)
}
if len(mediaItemIDs) == 0 {
return nil, nil
}
now := time.Now().UTC()
historyIDs := make([]string, 0, len(entries))
historyItemIDs := make([]string, 0, len(entries))
historyWatchedAt := make([]time.Time, 0, len(entries))
historyDurations := make([]float64, 0, len(entries))
historyCompleted := make([]bool, 0, len(entries))
historySources := make([]string, 0, len(entries))
historyIdentities := make([]string, 0, len(entries))
// Entries with a blank media ID are dropped, so keep an explicit map back
// to the caller's slice rather than assuming positional alignment.
historySourceIndex := make([]int, 0, len(entries))
for entryIndex, entry := range entries {
mediaItemID := strings.TrimSpace(entry.MediaItemID)
if mediaItemID == "" {
continue
}
id := entry.ID
if id == "" {
id = generateUUID()
}
watchedAt := now
if entry.WatchedAt != "" {
parsed, err := time.Parse(time.RFC3339, entry.WatchedAt)
if err != nil {
return nil, fmt.Errorf("parsing watched_at for %s: %w", mediaItemID, err)
}
watchedAt = parsed.UTC()
}
source := entry.Source
if source == "" {
source = userstore.WatchHistorySourceLegacy
}
identityJSON, err := json.Marshal(entry.Identity)
if err != nil {
return nil, fmt.Errorf("marshaling watch identity: %w", err)
}
historyIDs = append(historyIDs, id)
historyItemIDs = append(historyItemIDs, mediaItemID)
historyWatchedAt = append(historyWatchedAt, watchedAt)
historyDurations = append(historyDurations, entry.DurationSeconds)
historyCompleted = append(historyCompleted, entry.Completed)
historySources = append(historySources, string(source))
historyIdentities = append(historyIdentities, string(identityJSON))
historySourceIndex = append(historySourceIndex, entryIndex)
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, fmt.Errorf("begin mark watched batch: %w", err)
}
defer tx.Rollback(ctx) //nolint:errcheck
if _, err := tx.Exec(ctx, `
WITH target(media_item_id, duration_seconds) AS (
SELECT * FROM unnest($3::text[], $4::double precision[])
),
visible AS (
SELECT
t.media_item_id,
t.duration_seconds,
CASE
WHEN hhi.hidden_before IS NOT NULL AND $5::timestamptz <= hhi.hidden_before
THEN hhi.hidden_before + interval '1 second'
ELSE $5::timestamptz
END AS updated_at
FROM target t
LEFT JOIN user_history_hidden_items hhi
ON hhi.user_id = $1
AND hhi.profile_id = $2
AND hhi.media_item_id = t.media_item_id
)
INSERT INTO user_watch_progress
(user_id, profile_id, media_item_id, position_seconds, duration_seconds, completed, updated_at)
SELECT $1, $2, media_item_id, 0, duration_seconds, TRUE, updated_at
FROM visible
ON CONFLICT (user_id, profile_id, media_item_id) DO UPDATE SET
position_seconds = 0,
-- Keep a known duration when the caller has none: jellycompat's
-- series mark-played passes 0 for every target, and its previous
-- MarkProgressBatch path left duration_seconds untouched.
duration_seconds = CASE
WHEN EXCLUDED.duration_seconds > 0 THEN EXCLUDED.duration_seconds
ELSE user_watch_progress.duration_seconds
END,
completed = TRUE,
updated_at = EXCLUDED.updated_at,
event_at = EXCLUDED.updated_at`,
s.userID, profileID, mediaItemIDs, durations, now,
); err != nil {
return nil, fmt.Errorf("marking watched batch: %w", err)
}
written := make([]userstore.WatchHistoryEntry, 0, len(historyIDs))
if len(historyIDs) > 0 {
rows, err := tx.Query(ctx, `
WITH entry(id, media_item_id, watched_at, duration_seconds, completed, source, watch_identity, ord) AS (
SELECT * FROM unnest(
$3::text[], $4::text[], $5::timestamptz[], $6::double precision[],
$7::boolean[], $8::text[], $9::jsonb[]
) WITH ORDINALITY
)
INSERT INTO user_watch_history
(id, user_id, profile_id, media_item_id, watched_at, duration_seconds, completed, source, watch_identity)
SELECT
e.id, $1, $2, e.media_item_id,
CASE
WHEN hhi.hidden_before IS NOT NULL AND e.watched_at <= hhi.hidden_before
THEN hhi.hidden_before + interval '1 second'
ELSE e.watched_at
END,
e.duration_seconds, e.completed, e.source, e.watch_identity
FROM entry e
LEFT JOIN user_history_hidden_items hhi
ON hhi.user_id = $1
AND hhi.profile_id = $2
AND hhi.media_item_id = e.media_item_id
ORDER BY e.ord
RETURNING id, media_item_id, watched_at`,
s.userID, profileID, historyIDs, historyItemIDs, historyWatchedAt,
historyDurations, historyCompleted, historySources, historyIdentities,
)
if err != nil {
return nil, fmt.Errorf("adding visible history batch: %w", err)
}
resolved := make(map[string]time.Time, len(historyIDs))
for rows.Next() {
var id, mediaItemID string
var watchedAt time.Time
if err := rows.Scan(&id, &mediaItemID, &watchedAt); err != nil {
rows.Close()
return nil, fmt.Errorf("scanning inserted history: %w", err)
}
resolved[id] = watchedAt
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterating inserted history: %w", err)
}
for i, id := range historyIDs {
entry := entries[historySourceIndex[i]]
entry.ID = id
entry.MediaItemID = historyItemIDs[i]
if entry.Source == "" {
entry.Source = userstore.WatchHistorySource(historySources[i])
}
if watchedAt, ok := resolved[id]; ok {
entry.WatchedAt = timeToString(watchedAt)
}
written = append(written, entry)
}
}
if err := tx.Commit(ctx); err != nil {
return nil, fmt.Errorf("commit mark watched batch: %w", err)
}
return written, nil
}
// MarkProgressBatch upserts a `completed = TRUE` progress row for every entry
// in mediaItemIDs in a single statement. Used by the jellycompat series
// mark-played path so a 200-episode mark collapses to one INSERT.
-50
View File
@@ -31,56 +31,6 @@ func AddVisibleHistory(ctx context.Context, store UserStore, entry WatchHistoryE
return entry, nil
}
// MarkWatchedTarget is one leaf item to mark watched, carrying the duration
// that belongs on its progress row.
type MarkWatchedTarget struct {
MediaItemID string
DurationSeconds float64
}
// WatchedBatchWriter is an optional store capability: mark every target
// watched and insert its history row in a single transaction, so a canceled
// request leaves nothing marked instead of a partial subset. Marking a large
// series is the motivating case — the per-target fallback below commits each
// episode independently, so losing the request mid-loop strands the series
// half-watched.
//
// Semantics must match the fallback: per-target duration lands on the progress
// row, each entry gets exactly one history row, and both respect the
// hidden-history watermark the single-row MarkWatched/AddVisibleHistory paths
// apply. The returned entries carry the resolved (possibly watermark-adjusted)
// WatchedAt, in the order given.
type WatchedBatchWriter interface {
MarkWatchedBatch(ctx context.Context, profileID string, targets []MarkWatchedTarget, entries []WatchHistoryEntry) ([]WatchHistoryEntry, error)
}
// MarkWatchedBatch marks every target watched and records its history entry,
// atomically where the store supports it. targets and entries are parallel:
// entries[i] is the history row for targets[i].
func MarkWatchedBatch(ctx context.Context, store UserStore, profileID string, targets []MarkWatchedTarget, entries []WatchHistoryEntry) ([]WatchHistoryEntry, error) {
if len(targets) == 0 {
return nil, nil
}
if writer, ok := store.(WatchedBatchWriter); ok {
return writer.MarkWatchedBatch(ctx, profileID, targets, entries)
}
written := make([]WatchHistoryEntry, 0, len(entries))
for i, target := range targets {
if err := store.MarkWatched(ctx, profileID, target.MediaItemID, target.DurationSeconds); err != nil {
return written, err
}
if i >= len(entries) {
continue
}
entry, err := AddVisibleHistory(ctx, store, entries[i])
if err != nil {
return written, err
}
written = append(written, entry)
}
return written, nil
}
func VisibleHistoryTimestamps(ctx context.Context, store UserStore, profileID string, mediaItemIDs []string, at time.Time) (map[string]string, error) {
mediaItemIDs = compactHistoryMediaItemIDs(mediaItemIDs)
result := make(map[string]string, len(mediaItemIDs))
-5
View File
@@ -162,11 +162,6 @@ type UserStore interface {
DeleteAudioPreference(ctx context.Context, profileID, seriesID string) error
SetSeriesPlaybackPreference(ctx context.Context, pref SeriesPlaybackPreference) error
GetSeriesPlaybackPreference(ctx context.Context, profileID, seriesID string) (*SeriesPlaybackPreference, error)
// Collection sort preferences
SetCollectionSortPreference(ctx context.Context, pref CollectionSortPreference) error
GetCollectionSortPreference(ctx context.Context, profileID, collectionKind, collectionID string) (*CollectionSortPreference, error)
ClearCollectionSortPreference(ctx context.Context, profileID, collectionKind, collectionID string) error
DeleteSeriesPlaybackPreference(ctx context.Context, profileID, seriesID string) error
GetLibraryPlaybackPreference(ctx context.Context, profileID string, libraryID int) (*LibraryPlaybackPreference, error)
ListLibraryPlaybackPreferences(ctx context.Context, profileID string) ([]LibraryPlaybackPreference, error)
-296
View File
@@ -11,15 +11,6 @@ import (
"github.com/Silo-Server/silo-server/internal/userstore"
)
// RunMarkWatchedBatch runs only the batch mark-watched conformance test. It is
// exposed separately, like RunProgressSince, so each backend can pin the
// series/season mark-watched path without the full suite.
func RunMarkWatchedBatch(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
t.Run("MarkWatchedBatch", func(t *testing.T) {
testMarkWatchedBatch(t, newStore)
})
}
// RunProgressSince runs only the offline-sync progress-reconciliation
// conformance test (invariant 1). It is exposed separately so a backend can
// exercise the offline-sync behavior without the full suite.
@@ -35,134 +26,6 @@ func RunProgressSince(t *testing.T, newStore func(t *testing.T) userstore.UserSt
})
}
// RunCollectionSortPreferences runs the preference timestamp and profile
// lifecycle conformance checks against a UserStore implementation.
func RunCollectionSortPreferences(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
t.Run("TimestampAndProfileLifecycle", func(t *testing.T) {
testCollectionSortPreferences(t, newStore)
})
}
func testCollectionSortPreferences(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
ctx := context.Background()
store := newStore(t)
const (
profileID = "sort-pref-profile"
viewerProfileID = "sort-pref-viewer"
titleSortField = "title"
ascendingOrder = "asc"
)
if err := store.CreateProfile(ctx, userstore.Profile{ID: profileID, Name: "Sort Pref"}); err != nil {
t.Fatalf("CreateProfile: %v", err)
}
if err := store.CreateProfile(ctx, userstore.Profile{ID: viewerProfileID, Name: "Sort Pref Viewer"}); err != nil {
t.Fatalf("CreateProfile(viewer): %v", err)
}
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: profileID,
CollectionKind: userstore.CollectionKindLibrary,
CollectionID: "collection-1",
SortField: titleSortField,
SortOrder: ascendingOrder,
}); err != nil {
t.Fatalf("SetCollectionSortPreference: %v", err)
}
pref, err := store.GetCollectionSortPreference(ctx, profileID, userstore.CollectionKindLibrary, "collection-1")
if err != nil || pref == nil {
t.Fatalf("GetCollectionSortPreference = %v, err = %v", pref, err)
}
if _, err := time.Parse(time.RFC3339, pref.UpdatedAt); err != nil {
t.Fatalf("UpdatedAt = %q, want RFC3339 timestamp: %v", pref.UpdatedAt, err)
}
for _, invalid := range []userstore.CollectionSortPreference{
{
ProfileID: profileID,
CollectionKind: "invalid",
CollectionID: "collection-invalid-kind",
SortField: titleSortField,
SortOrder: ascendingOrder,
},
{
ProfileID: profileID,
CollectionKind: userstore.CollectionKindLibrary,
CollectionID: "collection-invalid-order",
SortField: titleSortField,
SortOrder: "sideways",
},
} {
if err := store.SetCollectionSortPreference(ctx, invalid); err == nil {
t.Fatalf("SetCollectionSortPreference accepted invalid preference: %+v", invalid)
}
}
deletedCollection, err := store.CreateCollection(ctx, userstore.CreateCollectionInput{
CreatorProfileID: profileID,
Name: "Deleted sort preference target",
})
if err != nil {
t.Fatalf("CreateCollection(delete target): %v", err)
}
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: viewerProfileID,
CollectionKind: userstore.CollectionKindUser,
CollectionID: deletedCollection.ID,
SortField: "year",
SortOrder: "desc",
}); err != nil {
t.Fatalf("SetCollectionSortPreference(delete target): %v", err)
}
if err := store.DeleteCollection(ctx, deletedCollection.ID); err != nil {
t.Fatalf("DeleteCollection: %v", err)
}
deletedPref, err := store.GetCollectionSortPreference(ctx, viewerProfileID, userstore.CollectionKindUser, deletedCollection.ID)
if err != nil {
t.Fatalf("GetCollectionSortPreference(deleted collection): %v", err)
}
if deletedPref != nil {
t.Fatalf("preference survived collection deletion: %+v", deletedPref)
}
profileDeletedCollection, err := store.CreateCollection(ctx, userstore.CreateCollectionInput{
CreatorProfileID: profileID,
Name: "Profile-deleted sort preference target",
})
if err != nil {
t.Fatalf("CreateCollection(profile delete target): %v", err)
}
if err := store.SetCollectionSortPreference(ctx, userstore.CollectionSortPreference{
ProfileID: viewerProfileID,
CollectionKind: userstore.CollectionKindUser,
CollectionID: profileDeletedCollection.ID,
SortField: titleSortField,
SortOrder: ascendingOrder,
}); err != nil {
t.Fatalf("SetCollectionSortPreference(profile delete target): %v", err)
}
if err := store.DeleteProfile(ctx, profileID); err != nil {
t.Fatalf("DeleteProfile: %v", err)
}
deletedPref, err = store.GetCollectionSortPreference(ctx, viewerProfileID, userstore.CollectionKindUser, profileDeletedCollection.ID)
if err != nil {
t.Fatalf("GetCollectionSortPreference(profile-deleted collection): %v", err)
}
if deletedPref != nil {
t.Fatalf("preference survived creator profile deletion: %+v", deletedPref)
}
if err := store.CreateProfile(ctx, userstore.Profile{ID: profileID, Name: "Recreated"}); err != nil {
t.Fatalf("CreateProfile(recreate): %v", err)
}
pref, err = store.GetCollectionSortPreference(ctx, profileID, userstore.CollectionKindLibrary, "collection-1")
if err != nil {
t.Fatalf("GetCollectionSortPreference(recreated profile): %v", err)
}
if pref != nil {
t.Fatalf("stale preference survived profile recreation: %+v", pref)
}
}
// RunSuite runs all conformance tests against a UserStore implementation.
// The newStore function should return a fresh, empty store for each test.
func RunSuite(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
@@ -181,9 +44,6 @@ func RunSuite(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
t.Run("BatchWritesAdvanceEventAt", func(t *testing.T) {
testBatchWritesAdvanceEventAt(t, newStore)
})
t.Run("MarkWatchedBatch", func(t *testing.T) {
testMarkWatchedBatch(t, newStore)
})
t.Run("Favorites", func(t *testing.T) {
testFavorites(t, newStore)
})
@@ -1144,162 +1004,6 @@ func testOnlineWriteAdvancesEventAt(t *testing.T, newStore func(t *testing.T) us
}
}
// testMarkWatchedBatch pins the batch mark-watched path used by the series and
// season mark-watched handlers. It must be equivalent to a MarkWatched +
// AddVisibleHistory loop: per-target durations land on progress rows, exactly
// one history row appears per target, the hidden-history watermark still
// pushes a mark after a removal back into visibility, and a zero duration
// leaves a known duration alone (jellycompat's mark-played supplies none).
func testMarkWatchedBatch(t *testing.T, newStore func(t *testing.T) userstore.UserStore) {
ctx := context.Background()
store := newStore(t)
if err := store.CreateProfile(ctx, userstore.Profile{ID: "p1", Name: "Test"}); err != nil {
t.Fatalf("CreateProfile: %v", err)
}
watchedAt := time.Date(2026, 5, 2, 9, 0, 0, 0, time.UTC)
targets := []userstore.MarkWatchedTarget{
{MediaItemID: "ep-1", DurationSeconds: 1200},
{MediaItemID: "ep-2", DurationSeconds: 1500},
}
entries := []userstore.WatchHistoryEntry{
{
ProfileID: "p1",
MediaItemID: "ep-1",
WatchedAt: watchedAt.Format(time.RFC3339),
DurationSeconds: 1200,
Completed: true,
Source: userstore.WatchHistorySourceManual,
},
{
ProfileID: "p1",
MediaItemID: "ep-2",
WatchedAt: watchedAt.Format(time.RFC3339),
DurationSeconds: 1500,
Completed: true,
Source: userstore.WatchHistorySourceManual,
},
}
written, err := userstore.MarkWatchedBatch(ctx, store, "p1", targets, entries)
if err != nil {
t.Fatalf("MarkWatchedBatch: %v", err)
}
if len(written) != 2 {
t.Fatalf("MarkWatchedBatch returned %d entries, want 2", len(written))
}
for _, entry := range written {
if entry.ID == "" {
t.Fatalf("returned entry %s has no ID; callers relay these to watch providers", entry.MediaItemID)
}
if entry.WatchedAt == "" {
t.Fatalf("returned entry %s has no WatchedAt", entry.MediaItemID)
}
}
for _, want := range targets {
progress, err := store.GetProgress(ctx, "p1", want.MediaItemID)
if err != nil {
t.Fatalf("GetProgress(%s): %v", want.MediaItemID, err)
}
if progress == nil || !progress.Completed {
t.Fatalf("GetProgress(%s) = %+v, want completed", want.MediaItemID, progress)
}
if progress.DurationSeconds != want.DurationSeconds {
t.Fatalf("GetProgress(%s) duration = %v, want %v — per-target durations must survive the batch",
want.MediaItemID, progress.DurationSeconds, want.DurationSeconds)
}
if progress.PositionSeconds != 0 {
t.Fatalf("GetProgress(%s) position = %v, want 0", want.MediaItemID, progress.PositionSeconds)
}
}
history, err := store.ListHistory(ctx, "p1", 50, 0)
if err != nil {
t.Fatalf("ListHistory: %v", err)
}
counts := map[string]int{}
for _, entry := range history {
counts[entry.MediaItemID]++
}
if counts["ep-1"] != 1 || counts["ep-2"] != 1 {
t.Fatalf("history counts = %v, want exactly one row per target", counts)
}
// A zero duration must not erase a duration the store already knows.
if _, err := userstore.MarkWatchedBatch(ctx, store, "p1",
[]userstore.MarkWatchedTarget{{MediaItemID: "ep-1"}},
[]userstore.WatchHistoryEntry{{
ProfileID: "p1",
MediaItemID: "ep-1",
WatchedAt: watchedAt.Add(time.Hour).Format(time.RFC3339),
Completed: true,
Source: userstore.WatchHistorySourceJellycompat,
}},
); err != nil {
t.Fatalf("MarkWatchedBatch(zero duration): %v", err)
}
progress, err := store.GetProgress(ctx, "p1", "ep-1")
if err != nil || progress == nil {
t.Fatalf("GetProgress(ep-1 after zero-duration mark) = %+v (%v)", progress, err)
}
if progress.DurationSeconds != 1200 {
t.Fatalf("duration = %v, want 1200 — a zero-duration mark must not clear a known duration",
progress.DurationSeconds)
}
// Marking watched after a history removal must land after the hidden
// watermark, otherwise the new mark is invisible.
removedAt := time.Date(2026, 5, 3, 9, 0, 0, 0, time.UTC)
if err := store.RemoveHistoryItems(ctx, "p1", []string{"ep-2"}, removedAt); err != nil {
t.Fatalf("RemoveHistoryItems(ep-2): %v", err)
}
remarked, err := userstore.MarkWatchedBatch(ctx, store, "p1",
[]userstore.MarkWatchedTarget{{MediaItemID: "ep-2", DurationSeconds: 1500}},
[]userstore.WatchHistoryEntry{{
ProfileID: "p1",
MediaItemID: "ep-2",
WatchedAt: watchedAt.Format(time.RFC3339), // older than the removal
DurationSeconds: 1500,
Completed: true,
Source: userstore.WatchHistorySourceManual,
}},
)
if err != nil {
t.Fatalf("MarkWatchedBatch(after removal): %v", err)
}
if len(remarked) != 1 {
t.Fatalf("re-mark returned %d entries, want 1", len(remarked))
}
if remarked[0].WatchedAt <= removedAt.Format(time.RFC3339) {
t.Fatalf("re-marked WatchedAt = %q, want later than the %q removal watermark",
remarked[0].WatchedAt, removedAt.Format(time.RFC3339))
}
completed, err := store.ListCompletedHistoryItems(ctx, userstore.CompletedHistoryItemQuery{
ProfileID: "p1",
MediaItemIDs: []string{"ep-2"},
})
if err != nil {
t.Fatalf("ListCompletedHistoryItems(ep-2): %v", err)
}
if len(completed) != 1 {
t.Fatalf("ep-2 completed history = %v, want the re-mark to be visible", completed)
}
// Blank IDs are compacted out rather than reaching SQL.
if _, err := userstore.MarkWatchedBatch(ctx, store, "p1",
[]userstore.MarkWatchedTarget{{MediaItemID: " "}, {MediaItemID: ""}},
nil,
); err != nil {
t.Fatalf("MarkWatchedBatch(blank IDs): %v", err)
}
if blank, err := store.GetProgress(ctx, "p1", ""); err != nil {
t.Fatalf("GetProgress(blank): %v", err)
} else if blank != nil {
t.Fatalf("GetProgress(blank) = %+v, want nil", blank)
}
}
// testBatchWritesAdvanceEventAt extends the LWW guarantee to the batch write
// paths (jellycompat series mark-played and mark-unplayed): they advance
// updated_at without hand-setting event_at, so the stamping trigger must
-21
View File
@@ -376,27 +376,6 @@ type SeriesPlaybackPreference struct {
UpdatedAt string `json:"updated_at"`
}
// Collection kinds for CollectionSortPreference. Collection ids are unique
// within a kind but not across the two id spaces, so the kind is part of the
// preference's identity.
const (
CollectionKindLibrary = "library"
CollectionKindUser = "user"
)
// CollectionSortPreference records that a profile changed the sort order while
// browsing a collection, overriding whatever default the collection's creator
// configured. An empty SortField is a real choice — "show me this collection in
// its own source order" — and is distinct from having no preference row at all.
type CollectionSortPreference struct {
ProfileID string `json:"profile_id"`
CollectionKind string `json:"collection_kind"`
CollectionID string `json:"collection_id"`
SortField string `json:"sort_field"`
SortOrder string `json:"sort_order"`
UpdatedAt string `json:"updated_at"`
}
// LibraryPlaybackPreference stores per-library playback settings.
type LibraryPlaybackPreference struct {
ProfileID string `json:"profile_id"`
-159
View File
@@ -86,165 +86,6 @@ func (r *StableIdentityResolver) ResolveHistoryIdentity(ctx context.Context, med
}
}
// batchEpisodeLookup is an optional capability on the episode lookup: fetch
// many episodes in one query. catalog.EpisodeRepository implements it.
type batchEpisodeLookup interface {
GetByIDs(ctx context.Context, contentIDs []string) ([]*models.Episode, error)
}
// batchProviderIDLookup is the same idea for provider IDs, letting a series
// mark resolve every distinct series in one query instead of one per episode.
type batchProviderIDLookup interface {
GetByContentIDs(ctx context.Context, contentIDs []string) (map[string][]*models.MediaItemProviderID, error)
}
// ResolveHistoryIdentities resolves stable identities for many media items at
// once. Marking a series watched previously resolved each episode on its own,
// costing an episode lookup plus a series provider-ID lookup per episode; this
// collapses that to one episode query plus one provider-ID query per distinct
// series. Items that resolve to nothing are absent from the map, matching
// ResolveHistoryIdentity's zero-value return.
func (r *StableIdentityResolver) ResolveHistoryIdentities(ctx context.Context, mediaItemIDs []string) map[string]userstore.WatchIdentity {
result := make(map[string]userstore.WatchIdentity, len(mediaItemIDs))
if r == nil || r.providerIDs == nil || len(mediaItemIDs) == 0 {
return result
}
ids := make([]string, 0, len(mediaItemIDs))
seen := make(map[string]struct{}, len(mediaItemIDs))
for _, mediaItemID := range mediaItemIDs {
mediaItemID = strings.TrimSpace(mediaItemID)
if mediaItemID == "" {
continue
}
if _, ok := seen[mediaItemID]; ok {
continue
}
seen[mediaItemID] = struct{}{}
ids = append(ids, mediaItemID)
}
if len(ids) == 0 {
return result
}
episodes := r.loadEpisodes(ctx, ids)
// One provider-ID lookup per distinct series covers every episode of it.
seriesIDs := make([]string, 0, len(episodes))
seenSeries := make(map[string]struct{}, len(episodes))
for _, episode := range episodes {
seriesID := strings.TrimSpace(episode.SeriesID)
if seriesID == "" {
continue
}
if _, ok := seenSeries[seriesID]; ok {
continue
}
seenSeries[seriesID] = struct{}{}
seriesIDs = append(seriesIDs, seriesID)
}
// Anything that is not an episode may still be a movie; resolve those
// provider IDs in the same batch.
movieCandidates := make([]string, 0, len(ids))
for _, mediaItemID := range ids {
if _, ok := episodes[mediaItemID]; !ok {
movieCandidates = append(movieCandidates, mediaItemID)
}
}
providerIDsByContent := r.loadProviderIDsBatch(ctx, append(append([]string{}, seriesIDs...), movieCandidates...))
for _, mediaItemID := range ids {
if episode, ok := episodes[mediaItemID]; ok {
episodeIDs := episodeProviderIDs(episode)
seriesProviderIDs := providerIDMap(providerIDsByContent[episode.SeriesID])
// Mirrors ResolveHistoryIdentity: an episode carrying its own IDs
// is addressable without the nested show fallback.
if len(episodeIDs) == 0 && len(seriesProviderIDs) == 0 {
continue
}
seasonNumber := episode.SeasonNumber
episodeNumber := episode.EpisodeNumber
result[mediaItemID] = userstore.WatchIdentity{
StableType: "episode",
ProviderIDs: episodeIDs,
SeriesProviderIDs: seriesProviderIDs,
Season: &seasonNumber,
Episode: &episodeNumber,
}
continue
}
if r.items == nil {
continue
}
item, err := r.items.GetByID(ctx, mediaItemID)
if err != nil || item == nil || item.Type != "movie" {
continue
}
itemIDs := providerIDMap(providerIDsByContent[mediaItemID])
if len(itemIDs) == 0 {
continue
}
result[mediaItemID] = userstore.WatchIdentity{
StableType: "movie",
ProviderIDs: itemIDs,
}
}
return result
}
// loadEpisodes returns the subset of ids that are episodes, batching when the
// lookup supports it and falling back to per-ID reads otherwise.
func (r *StableIdentityResolver) loadEpisodes(ctx context.Context, ids []string) map[string]*models.Episode {
episodes := make(map[string]*models.Episode, len(ids))
if r.episodes == nil {
return episodes
}
if batch, ok := r.episodes.(batchEpisodeLookup); ok {
found, err := batch.GetByIDs(ctx, ids)
if err == nil {
for _, episode := range found {
if episode != nil {
episodes[episode.ContentID] = episode
}
}
return episodes
}
}
for _, mediaItemID := range ids {
episode, err := r.episodes.GetByID(ctx, mediaItemID)
if err == nil && episode != nil {
episodes[mediaItemID] = episode
}
}
return episodes
}
// loadProviderIDsBatch groups provider IDs by content ID, batching when the
// lookup supports it and falling back to per-ID reads otherwise.
func (r *StableIdentityResolver) loadProviderIDsBatch(ctx context.Context, contentIDs []string) map[string][]*models.MediaItemProviderID {
result := make(map[string][]*models.MediaItemProviderID, len(contentIDs))
if len(contentIDs) == 0 {
return result
}
if batch, ok := r.providerIDs.(batchProviderIDLookup); ok {
found, err := batch.GetByContentIDs(ctx, contentIDs)
if err == nil {
return found
}
}
for _, contentID := range contentIDs {
if _, ok := result[contentID]; ok {
continue
}
result[contentID] = r.loadProviderIDs(ctx, contentID)
}
return result
}
func (r *StableIdentityResolver) ResolveMovieContentID(ctx context.Context, providerIDs map[string]string) (string, error) {
if r == nil || r.providerIDs == nil {
return "", nil
+27 -49
View File
@@ -325,22 +325,12 @@ func (s *Service) recordMarkWatched(
if watchedAt.IsZero() {
watchedAt = time.Now().UTC()
}
// Resolve every identity up front: one episode query plus one provider-ID
// query per distinct series, instead of two lookups per episode.
completedIDs := make([]string, 0, len(targets))
result := ManualMarkResult{Entries: make([]userstore.WatchHistoryEntry, 0, len(targets))}
for _, target := range targets {
completedIDs = append(completedIDs, target.MediaItemID)
}
identities := s.resolveStableIdentities(ctx, completedIDs)
batchTargets := make([]userstore.MarkWatchedTarget, 0, len(targets))
entries := make([]userstore.WatchHistoryEntry, 0, len(targets))
for _, target := range targets {
batchTargets = append(batchTargets, userstore.MarkWatchedTarget{
MediaItemID: target.MediaItemID,
DurationSeconds: target.DurationSeconds,
})
entries = append(entries, userstore.WatchHistoryEntry{
if err := store.MarkWatched(ctx, profileID, target.MediaItemID, target.DurationSeconds); err != nil {
return result, err
}
histEntry := userstore.WatchHistoryEntry{
ID: uuid.NewString(),
ProfileID: profileID,
MediaItemID: target.MediaItemID,
@@ -348,16 +338,17 @@ func (s *Service) recordMarkWatched(
DurationSeconds: target.DurationSeconds,
Completed: true,
Source: source,
Identity: identities[target.MediaItemID],
})
}
s.applyStableIdentity(ctx, &histEntry)
histEntry, err = userstore.AddVisibleHistory(ctx, store, histEntry)
if err != nil {
return result, err
}
result.Entries = append(result.Entries, histEntry)
}
// One transaction for the whole set: a canceled request rolls back to
// "nothing marked" rather than leaving a series partially watched.
written, err := userstore.MarkWatchedBatch(ctx, store, profileID, batchTargets, entries)
result := ManualMarkResult{Entries: written}
if err != nil {
return result, err
completedIDs := make([]string, 0, len(targets))
for _, target := range targets {
completedIDs = append(completedIDs, target.MediaItemID)
}
s.notifyWatchedCompleted(ctx, userID, profileID, completedIDs)
return result, nil
@@ -454,28 +445,24 @@ func (s *Service) recordMarkWatchedBatch(
if watchedAt.IsZero() {
watchedAt = time.Now().UTC()
}
// Shares recordMarkWatched's transactional store path (audit 2026-05-01
// §2.7 kept the progress upsert batched; this extends the same treatment
// to history so the whole mark commits or rolls back as one). Identities
// still resolve per episode, now in a single bulk lookup. No durations are
// available here, and the store leaves a known duration in place when a
// target supplies zero.
identities := s.resolveStableIdentities(ctx, targetIDs)
batchTargets := make([]userstore.MarkWatchedTarget, 0, len(targetIDs))
entries := make([]userstore.WatchHistoryEntry, 0, len(targetIDs))
if err := store.MarkProgressBatch(ctx, profileID, targetIDs, watchedAt); err != nil {
return err
}
// Strategy A (audit 2026-05-01 §2.7): batch the progress upsert because it
// powers hot Continue-Watching queries. History inserts stay per-target so
// per-episode stable-identity resolution still applies.
for _, targetID := range targetIDs {
batchTargets = append(batchTargets, userstore.MarkWatchedTarget{MediaItemID: targetID})
entries = append(entries, userstore.WatchHistoryEntry{
histEntry := userstore.WatchHistoryEntry{
ProfileID: profileID,
MediaItemID: targetID,
WatchedAt: formatWatchedAt(watchedAt),
Completed: true,
Source: source,
Identity: identities[targetID],
})
}
if _, err := userstore.MarkWatchedBatch(ctx, store, profileID, batchTargets, entries); err != nil {
return err
}
s.applyStableIdentity(ctx, &histEntry)
if _, err := userstore.AddVisibleHistory(ctx, store, histEntry); err != nil {
return err
}
}
s.notifyWatchedCompleted(ctx, userID, profileID, targetIDs)
return nil
@@ -549,15 +536,6 @@ func (s *Service) addImportedHistoryIfMissingWithSource(
return store.AddHistoryIfMissing(ctx, entry)
}
// resolveStableIdentities resolves identities for many media items in one go,
// returning an empty map when no resolver is configured.
func (s *Service) resolveStableIdentities(ctx context.Context, mediaItemIDs []string) map[string]userstore.WatchIdentity {
if s == nil || s.identity == nil || len(mediaItemIDs) == 0 {
return nil
}
return s.identity.ResolveHistoryIdentities(ctx, mediaItemIDs)
}
func (s *Service) applyStableIdentity(ctx context.Context, entry *userstore.WatchHistoryEntry) {
if s == nil || s.identity == nil || entry == nil {
return

Some files were not shown because too many files have changed in this diff Show More