Compare commits

...
Author SHA1 Message Date
rxwatcher 17ec6646a1 fix: clean ebook embedded descriptions 2026-06-08 21:56:19 +02:00
rxwatcher c200ecc061 fix: make ebook reader build clean 2026-06-08 21:32:41 +02:00
rxwatcher bcaa6dc58d fix: include ebook reader vendor in docker build 2026-06-08 21:30:20 +02:00
rxwatcher cfc0d4ac0d fix: encode item API ids 2026-06-08 19:52:35 +02:00
rxwatcher 5adcbbd1e2 fix: encode catalog read API ids 2026-06-08 19:51:01 +02:00
rxwatcher ca55d6f146 fix: make global search prompt media neutral 2026-06-08 19:48:42 +02:00
rxwatcher 28c56a7e59 fix: label ebook search results clearly 2026-06-08 19:48:08 +02:00
rxwatcher 2908fc9a9a test: cover ebook scan format set 2026-06-08 19:47:38 +02:00
rxwatcher 60542f8768 fix: encode recommendation and search item links 2026-06-08 19:46:19 +02:00
rxwatcher f1e3a8df95 fix: encode watch tonight item links 2026-06-08 19:44:09 +02:00
rxwatcher 527ccd64e3 fix: encode hero and continue item links 2026-06-08 19:42:26 +02:00
rxwatcher 45c79f4d82 fix: encode catalog card item links 2026-06-08 19:41:22 +02:00
rxwatcher 78ac6b0551 fix: encode related rail item links 2026-06-08 19:40:11 +02:00
rxwatcher ae42516a91 fix: link ebook genres with library scope 2026-06-08 19:37:06 +02:00
rxwatcher 958abefe53 fix: preserve ebook reader library context 2026-06-08 19:35:58 +02:00
rxwatcher 7a28a8fde9 test: cover ebook unread catalog rules 2026-06-08 19:33:24 +02:00
rxwatcher 3f6105ae00 fix: remove plain text reader loader path 2026-06-08 19:32:28 +02:00
rxwatcher 998174c8f7 fix: label ebook personalized sorts as read 2026-06-08 19:31:24 +02:00
rxwatcher 48422f0fe1 fix: label ebook advanced filters as read 2026-06-08 19:29:17 +02:00
rxwatcher 463b21c683 fix: reject txt ebook reader files 2026-06-08 19:27:06 +02:00
rxwatcher d40dca9022 fix: remove txt ebook reader support 2026-06-08 19:25:11 +02:00
rxwatcher 9676e616ab fix: show ebook related rails as book covers 2026-06-08 19:22:53 +02:00
rxwatcher 544a1e1332 fix: label ebook progress filters as read 2026-06-08 19:21:10 +02:00
rxwatcher be18357469 fix: merge ebook reader progress during item repair 2026-06-08 19:17:12 +02:00
rxwatcher ecde6d6039 fix: scope narrator catalog badges to audiobooks 2026-06-08 19:15:43 +02:00
rxwatcher 9d6216da3a fix: persist non-epub ebook reader progress 2026-06-08 19:14:31 +02:00
rxwatcher 2fbc268124 fix: encode ebook hero reader links 2026-06-08 19:10:12 +02:00
rxwatcher 3c70c0aaae feat: cache embedded ebook covers during scan 2026-06-08 19:09:28 +02:00
rxwatcher 5eb34cbeda fix: detect archive ebook reader formats by filename 2026-06-08 19:04:56 +02:00
rxwatcher a7569e6d40 fix: route ebook hero actions to reader 2026-06-08 19:01:55 +02:00
rxwatcher a8afc04580 fix: route ebook watch tonight cards to reader 2026-06-08 18:59:18 +02:00
rxwatcher 8edb5c761a fix: hide watched toggle for ebooks 2026-06-08 18:57:34 +02:00
rxwatcher 6ab4458efa fix: route ebook continue cards to reader 2026-06-08 18:56:26 +02:00
rxwatcher f305e4c6a1 fix: add ebook library default sections 2026-06-08 18:53:35 +02:00
rxwatcher c406b56b98 fix: honor profile header for ebook item progress 2026-06-08 18:51:46 +02:00
rxwatcher 900f7adb77 fix: support ebook subtree scans 2026-06-08 18:51:46 +02:00
rxwatcher 783d6751dd fix: expose ebook reader progress on item detail 2026-06-08 18:49:09 +02:00
rxwatcher 5ad2fe9b9e fix: include ebook activity in recommendations 2026-06-08 18:47:13 +02:00
rxwatcher c809d8139b feat: report ebook scan progress 2026-06-08 18:41:51 +02:00
rxwatcher 9f238f2b8e feat: share ebook progress state across item surfaces 2026-06-08 18:40:08 +02:00
rxwatcher b249d53aaf feat: reflect ebook reader progress in item user state 2026-06-08 18:36:49 +02:00
rxwatcher c35fa8bbee fix: include ebooks in last watched catalog filters 2026-06-08 18:34:58 +02:00
rxwatcher 2a30d8a517 fix: filter ebook catalogs by reader progress 2026-06-08 18:33:40 +02:00
rxwatcher 45acc59196 fix: sort ebook catalogs by reader progress 2026-06-08 18:31:41 +02:00
rxwatcher 92ffbfd9ce fix: fall back from unsupported ebook reader files 2026-06-08 18:28:16 +02:00
rxwatcher d083dd44c2 fix: filter ebook asin provider aliases 2026-06-08 18:26:58 +02:00
rxwatcher ffec643a90 fix: read ebook isbn from epub metadata 2026-06-08 18:25:56 +02:00
rxwatcher 5ac4f812b2 feat: include ebooks in catalog progress metrics 2026-06-08 18:24:35 +02:00
rxwatcher 67d5bf7dfe feat: include ebooks in continue watching sections 2026-06-08 18:21:05 +02:00
rxwatcher 433535593f feat: include ebook progress in recommendation signals 2026-06-08 18:18:16 +02:00
rxwatcher 11cc37faf9 fix: include ebooks in recently added recommendations 2026-06-08 18:13:42 +02:00
rxwatcher 9b2ce9588a fix: include ebooks in recommendation media mix 2026-06-08 18:11:18 +02:00
rxwatcher 52a21e48da fix: include ebooks in embedding eligibility 2026-06-08 18:09:55 +02:00
rxwatcher fb5af3ea00 fix: build ebook recommendation text from authors 2026-06-08 18:06:14 +02:00
rxwatcher 70ba2f132d fix: reject narrator queries for ebooks 2026-06-08 18:04:15 +02:00
rxwatcher c41d8f1641 fix: scope ebook catalog facets 2026-06-08 18:02:58 +02:00
rxwatcher a3486f7671 fix: authorize fbz ebooks from compound filename 2026-06-08 17:59:37 +02:00
rxwatcher 48566514fc fix: detect fbz ebooks from compound filename 2026-06-08 17:58:39 +02:00
rxwatcher 7b599970af fix: serve fbz ebook archives with reader mime type 2026-06-08 17:57:33 +02:00
rxwatcher f014f55e6f feat: filter ebook reader file selector 2026-06-08 17:56:05 +02:00
rxwatcher 7ab24d85e7 feat: hide ebook read action for unsupported files 2026-06-08 17:54:59 +02:00
rxwatcher 3b3e1fbbde feat: sync ebook reader progress cache 2026-06-08 17:53:12 +02:00
rxwatcher 0bebbf96c7 feat: surface ebook reader progress 2026-06-08 17:49:06 +02:00
rxwatcher 3db6babfad feat: prefer epub for ebook read action 2026-06-08 17:45:11 +02:00
rxwatcher 95c1802528 feat: switch ebook reader files 2026-06-08 17:43:46 +02:00
rxwatcher 2b29356e76 feat: show ebook download summaries 2026-06-08 17:41:54 +02:00
rxwatcher f4ed50c896 feat: show ebook file page counts 2026-06-08 17:40:21 +02:00
rxwatcher 9a8babdb75 feat: count cbz ebook pages 2026-06-08 17:36:56 +02:00
rxwatcher dad9e21563 feat: extract fbz ebook metadata 2026-06-08 17:34:21 +02:00
rxwatcher 648e65bdc3 feat: favor scanner isbn during ebook enrichment 2026-06-08 17:30:23 +02:00
rxwatcher 4f139a195a feat: extract ebook pdf metadata 2026-06-08 17:24:59 +02:00
rxwatcher c53fbd3442 feat: add ebook reader controls 2026-06-08 17:02:37 +02:00
rxwatcher 72650c147f feat: persist ebook reader progress 2026-06-08 16:56:08 +02:00
rxwatcher 776c2ce034 feat: render ebooks with foliate reader 2026-06-08 15:43:38 +02:00
rxwatcher a67015a808 feat: add ebook reader file entrypoint 2026-06-08 15:39:20 +02:00
rxwatcher e34d1ff47a feat: add ebook detail related content 2026-06-08 14:22:57 +02:00
rxwatcher 82df5d063a chore: omit ebook planning docs from branch 2026-06-08 14:11:14 +02:00
rxwatcher 94f5dd8ac2 fix: force ebook people refresh for stale narrators 2026-06-08 14:09:17 +02:00
rxwatcher 46af6746b3 fix: drop asin provider ids from ebook enrichment 2026-06-08 14:03:15 +02:00
rxwatcher 53a78d1290 fix: align ebook collection filters with book media 2026-06-08 14:01:05 +02:00
rxwatcher 08d13552d7 fix: drop narrator credits from ebook scanner merges 2026-06-08 13:57:41 +02:00
rxwatcher 8ad14501cd test: cover ebook detail authors and downloads 2026-06-08 13:56:02 +02:00
rxwatcher 8161a70a09 fix: include catalog safeguards for local ebook integration 2026-06-08 13:54:44 +02:00
rxwatcher b7d05cdee9 feat: use file language in download dialog 2026-06-08 13:34:33 +02:00
rxwatcher 025009c40a feat: use file-size copy for downloads 2026-06-08 13:34:33 +02:00
rxwatcher 82f6e97e66 feat: label ebook file versions by format 2026-06-08 13:34:33 +02:00
rxwatcher f8e3f2dac8 feat: add ebook detail view 2026-06-08 13:34:33 +02:00
rxwatcher 334e2f7b06 feat: add ebook catalog scope support 2026-06-08 13:34:33 +02:00
rxwatcher cd6b0afb34 feat: expose ebook library metadata setup 2026-06-08 13:33:43 +02:00
rxwatcher f734d7a01c feat: wire ebook metadata sync task 2026-06-08 13:33:43 +02:00
rxwatcher ddd579767b fix: harden ebook poster cache 2026-06-08 13:33:43 +02:00
rxwatcher 3018622e5e feat: add ebook metadata enricher 2026-06-08 13:33:43 +02:00
rxwatcher 9e7189f074 docs: clarify ebook foundation PR scope 2026-06-08 13:31:28 +02:00
rxwatcher 21170cfcf3 fix: address ebook scanner PR review 2026-06-08 12:29:35 +02:00
rxwatcher 0e1c2995a3 test: cover ebook series persistence decisions 2026-06-08 12:03:16 +02:00
rxwatcher 98d213204a feat: persist ebook series membership 2026-06-08 11:56:29 +02:00
rxwatcher 5ea8868047 fix: refresh ebook scan metadata safely 2026-06-08 11:52:20 +02:00
rxwatcher 0b97d2bd6d fix: preserve ebook scan people credits 2026-06-08 11:47:18 +02:00
rxwatcher 293db1f255 feat: scan ebook libraries in core 2026-06-08 11:40:31 +02:00
rxwatcher 1119cef018 fix: guard ebook subtree scans 2026-06-08 11:34:14 +02:00
rxwatcher c91a1d5b29 fix: handle ebook isbn labels 2026-06-08 11:31:45 +02:00
rxwatcher f15423d505 fix: harden ebook scanner foundation 2026-06-08 11:31:45 +02:00
rxwatcher 6de2b19c07 feat: add ebook scanner parser foundation 2026-06-08 11:31:45 +02:00
rxwatcher adf595cbf9 docs: plan ebook audiobook-parity implementation 2026-06-08 11:31:45 +02:00
rxwatcher 0b44952816 docs: define ebook architecture matching audiobooks 2026-06-08 11:31:45 +02:00
359 changed files with 21896 additions and 595 deletions
+1
View File
@@ -3,6 +3,7 @@ FROM node:22-slim AS frontend
RUN corepack enable && corepack prepare pnpm@10.32.1 --activate
WORKDIR /app/web
COPY web/package.json web/pnpm-lock.yaml ./
COPY web/vendor/foliate-js ./vendor/foliate-js
RUN --mount=type=cache,target=/root/.local/share/pnpm/store \
pnpm install --frozen-lockfile
COPY web/ .
+16
View File
@@ -47,6 +47,7 @@ import (
"github.com/Silo-Server/silo-server/internal/clientip"
"github.com/Silo-Server/silo-server/internal/config"
"github.com/Silo-Server/silo-server/internal/database"
"github.com/Silo-Server/silo-server/internal/ebooks"
evt "github.com/Silo-Server/silo-server/internal/events"
"github.com/Silo-Server/silo-server/internal/historyimport"
"github.com/Silo-Server/silo-server/internal/imagecache"
@@ -888,6 +889,7 @@ func main() {
var seasonRepo *catalog.SeasonRepository
var episodeRepo *catalog.EpisodeRepository
var audiobookEnricher *audiobooks.Enricher
var ebookEnricher *ebooks.Enricher
if needsWorkers && deps.DB != nil && deps.FileRepo != nil {
chainRepo := metadata.NewChainRepository(deps.DB)
skippedRootRepo = metadata.NewSkippedRootRepository(deps.DB)
@@ -969,6 +971,14 @@ func main() {
personRepo,
providerIDRepo,
)
ebookEnricher = ebooks.NewEnricher(
deps.DB,
chainRepo,
pluginResolver,
itemRepo,
personRepo,
providerIDRepo,
)
// Always wire the image resolver so plugin-prefixed URLs (e.g.
// metadb://) can be resolved to presigned HTTP URLs in API responses.
@@ -991,6 +1001,9 @@ func main() {
audiobookEnricher.SetImageCacher(imageCacher)
audiobookEnricher.SetFFmpegPath(scanner.FFmpegPathFromFFprobe(scanner.FFprobePathFromFFmpeg(cfg.Playback.FFmpegPath)))
}
if ebookEnricher != nil {
ebookEnricher.SetImageCacher(imageCacher)
}
}
matchWorker = metadata.NewMatchWorker(metadataService, deps.FileRepo, cfg.Matcher.Workers, cfg.Matcher.BatchSize, 30*time.Second)
@@ -1534,6 +1547,9 @@ func main() {
if audiobookEnricher != nil {
taskMgr.Register(tasks.NewSyncAudiobookMetadataTask(audiobookEnricher))
}
if ebookEnricher != nil {
taskMgr.Register(tasks.NewSyncEbookMetadataTask(ebookEnricher))
}
if pluginInstallationStore != nil && pluginRuntimeConfigStore != nil && pluginService != nil {
pluginTasks, err := plugins.NewTaskRegistryWithTypedResolver(pluginInstallationStore, pluginRuntimeConfigStore, pluginService).Tasks(appCtx)
if err != nil {
+1 -1
View File
@@ -33,6 +33,7 @@ require (
github.com/zishang520/socket.io/v2 v2.5.0
go.n16f.net/thumbhash v1.1.0
golang.org/x/image v0.39.0
golang.org/x/net v0.53.0
google.golang.org/genproto/googleapis/rpc v0.0.0-20260420184626-e10c466a9529
gopkg.in/natefinch/lumberjack.v2 v2.2.1
)
@@ -71,7 +72,6 @@ require (
go.uber.org/mock v0.5.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/mod v0.34.0 // indirect
golang.org/x/net v0.53.0 // indirect
golang.org/x/tools v0.43.0 // indirect
)
+2
View File
@@ -10,6 +10,7 @@ import (
"strings"
"time"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/sections"
@@ -89,6 +90,7 @@ func (h *CatalogHandler) HandleGetCatalog(w http.ResponseWriter, r *http.Request
h.itemsH.accessFilter(r),
overlaySummaries,
store,
apimw.GetUserID(r.Context()),
profileID,
)
items := make([]itemListResponse, 0, len(result.Items))
+2 -2
View File
@@ -497,8 +497,8 @@ func (h *CatalogResourceHandler) enrichItemDetail(r *http.Request, detail *catal
detail.SeasonUserData = h.items.getAggregateUserData(r, episodes)
}
}
case "movie", "episode", "audiobook":
detail.SeasonUserData = h.items.getLeafUserData(r, detail.ContentID)
case "movie", "episode", "audiobook", "ebook":
detail.SeasonUserData = h.items.getLeafUserData(r, detail.ContentID, detail.Type)
applyEffectiveEditionPreference(detail.SeasonUserData, &detail.EffectiveVersionEditionKey)
}
+380
View File
@@ -0,0 +1,380 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
// EbookReaderProgress is the persisted reader position for one profile and ebook.
type EbookReaderProgress struct {
UserID int `json:"-"`
ProfileID string `json:"-"`
ContentID string `json:"content_id"`
FileID int `json:"file_id"`
Location string `json:"location"`
Progress float64 `json:"progress"`
UpdatedAt time.Time `json:"updated_at"`
}
type EbookReaderProgressStore interface {
Get(ctx context.Context, userID int, profileID string, contentID string) (*EbookReaderProgress, error)
Upsert(ctx context.Context, progress EbookReaderProgress) error
}
// EbookReaderHandler serves ebook files for the in-app reader.
type EbookReaderHandler struct {
FileAuthorizer *MediaFileAuthorizer
ProgressStore EbookReaderProgressStore
}
func NewEbookReaderHandler(authorizer *MediaFileAuthorizer) *EbookReaderHandler {
return &EbookReaderHandler{FileAuthorizer: authorizer}
}
// HandleReadFile serves an ebook file inline with byte-range support.
func (h *EbookReaderHandler) HandleReadFile(w http.ResponseWriter, r *http.Request) {
if apimw.GetUserID(r.Context()) == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h == nil || h.FileAuthorizer == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Ebook reader is not configured")
return
}
contentID := strings.TrimSpace(chi.URLParam(r, "content_id"))
fileID, err := strconv.Atoi(chi.URLParam(r, "file_id"))
if contentID == "" || err != nil || fileID <= 0 {
writeError(w, http.StatusBadRequest, "bad_request", "content_id and file_id are required")
return
}
file, err := h.FileAuthorizer.Authorize(r, fileID)
if err != nil {
h.writeReadError(w, err)
return
}
if file == nil || file.ContentID != contentID || !isEbookFile(file) {
writeError(w, http.StatusNotFound, "not_found", "Ebook file not found")
return
}
if err := serveEbookInline(w, r, file); err != nil {
if errors.Is(err, catalog.ErrItemNotFound) {
writeError(w, http.StatusNotFound, "not_found", "Ebook file not found")
return
}
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to serve ebook file")
}
}
type ebookReaderProgressRequest struct {
FileID int `json:"file_id"`
Location string `json:"location"`
Progress float64 `json:"progress"`
}
func (h *EbookReaderHandler) HandleGetProgress(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h == nil || h.ProgressStore == nil || h.FileAuthorizer == nil || h.FileAuthorizer.ItemAccess == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Ebook reader progress is not configured")
return
}
contentID := strings.TrimSpace(chi.URLParam(r, "content_id"))
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return
}
if err := h.FileAuthorizer.ItemAccess.EnsureAccessible(r.Context(), contentID, requestAccessFilter(r)); err != nil {
h.writeReadError(w, err)
return
}
progress, err := h.ProgressStore.Get(r.Context(), userID, profileID, contentID)
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load ebook progress")
return
}
if progress == nil {
writeJSON(w, http.StatusOK, map[string]any{})
return
}
writeJSON(w, http.StatusOK, progress)
}
func (h *EbookReaderHandler) HandleSaveProgress(w http.ResponseWriter, r *http.Request) {
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if userID == 0 {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
if h == nil || h.ProgressStore == nil || h.FileAuthorizer == nil {
writeError(w, http.StatusServiceUnavailable, "unavailable", "Ebook reader progress is not configured")
return
}
contentID := strings.TrimSpace(chi.URLParam(r, "content_id"))
if contentID == "" {
writeError(w, http.StatusBadRequest, "bad_request", "content_id is required")
return
}
var req ebookReaderProgressRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Invalid request body")
return
}
req.Location = strings.TrimSpace(req.Location)
if req.FileID <= 0 || req.Location == "" || req.Progress < 0 || req.Progress > 1 {
writeError(w, http.StatusBadRequest, "bad_request", "file_id, location, and progress are required")
return
}
file, err := h.FileAuthorizer.Authorize(r, req.FileID)
if err != nil {
h.writeReadError(w, err)
return
}
if file == nil || file.ContentID != contentID || !isEbookFile(file) {
writeError(w, http.StatusNotFound, "not_found", "Ebook file not found")
return
}
progress := EbookReaderProgress{
UserID: userID,
ProfileID: profileID,
ContentID: contentID,
FileID: req.FileID,
Location: req.Location,
Progress: req.Progress,
UpdatedAt: time.Now().UTC(),
}
if err := h.ProgressStore.Upsert(r.Context(), progress); err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save ebook progress")
return
}
writeJSON(w, http.StatusOK, progress)
}
func (h *EbookReaderHandler) writeReadError(w http.ResponseWriter, err error) {
switch {
case errors.Is(err, catalog.ErrItemNotFound), errors.Is(err, catalog.ErrEpisodeNotFound):
writeError(w, http.StatusNotFound, "not_found", "Ebook file not found")
default:
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to authorize ebook file")
}
}
func serveEbookInline(w http.ResponseWriter, r *http.Request, file *models.MediaFile) error {
f, err := os.Open(file.FilePath)
if err != nil {
if os.IsNotExist(err) {
return catalog.ErrItemNotFound
}
return fmt.Errorf("opening ebook file: %w", err)
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return fmt.Errorf("stat ebook file: %w", err)
}
filename := sanitizeInlineFilename(filepath.Base(file.FilePath))
w.Header().Set("Content-Type", ebookMimeType(file.FilePath, file.Container))
w.Header().Set("Content-Disposition", fmt.Sprintf(`inline; filename="%s"`, filename))
http.ServeContent(w, r, stat.Name(), stat.ModTime(), f)
return nil
}
func isEbookFile(file *models.MediaFile) bool {
if file == nil {
return false
}
if !strings.EqualFold(file.BaseType, "ebook") {
return false
}
if strings.HasSuffix(strings.ToLower(file.FilePath), ".fb2.zip") {
return true
}
return isEbookReaderFormat(file.Container) || isEbookReaderFormat(filepath.Ext(file.FilePath))
}
func isEbookReaderFormat(value string) bool {
format := strings.TrimPrefix(strings.ToLower(strings.TrimSpace(value)), ".")
switch format {
case "epub", "pdf", "mobi", "azw", "azw3", "cbz", "cbr", "fb2", "fbz", "md":
return true
default:
return false
}
}
func ebookMimeType(path, container string) string {
ext := strings.ToLower(filepath.Ext(path))
if strings.HasSuffix(strings.ToLower(path), ".fb2.zip") {
ext = ".fbz"
} else if ext == "" && container != "" {
ext = "." + strings.TrimPrefix(strings.ToLower(container), ".")
}
switch ext {
case ".epub":
return "application/epub+zip"
case ".pdf":
return "application/pdf"
case ".mobi":
return "application/x-mobipocket-ebook"
case ".azw":
return "application/vnd.amazon.ebook"
case ".azw3":
return "application/vnd.amazon.mobi8-ebook"
case ".cbz":
return "application/vnd.comicbook+zip"
case ".cbr":
return "application/vnd.comicbook-rar"
case ".fb2":
return "application/x-fictionbook+xml"
case ".fbz":
return "application/x-zip-compressed-fb2"
case ".md":
return "text/markdown; charset=utf-8"
default:
return "application/octet-stream"
}
}
func sanitizeInlineFilename(name string) string {
name = strings.TrimSpace(name)
if name == "" || name == "." || name == string(filepath.Separator) {
return "ebook"
}
name = strings.ReplaceAll(name, `"`, "")
name = strings.ReplaceAll(name, "\n", " ")
name = strings.ReplaceAll(name, "\r", " ")
return name
}
type PGEbookReaderProgressStore struct {
pool *pgxpool.Pool
}
func NewPGEbookReaderProgressStore(pool *pgxpool.Pool) *PGEbookReaderProgressStore {
return &PGEbookReaderProgressStore{pool: pool}
}
func (s *PGEbookReaderProgressStore) Get(ctx context.Context, userID int, profileID string, contentID string) (*EbookReaderProgress, error) {
if s == nil || s.pool == nil {
return nil, fmt.Errorf("ebook reader progress store is not configured")
}
var progress EbookReaderProgress
err := s.pool.QueryRow(ctx, `
SELECT user_id, profile_id, content_id, file_id, location, progress, updated_at
FROM ebook_reader_progress
WHERE user_id = $1 AND profile_id = $2 AND content_id = $3`,
userID, profileID, contentID,
).Scan(
&progress.UserID,
&progress.ProfileID,
&progress.ContentID,
&progress.FileID,
&progress.Location,
&progress.Progress,
&progress.UpdatedAt,
)
if errors.Is(err, pgx.ErrNoRows) {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get ebook reader progress: %w", err)
}
return &progress, nil
}
func (s *PGEbookReaderProgressStore) ListByContentIDs(ctx context.Context, userID int, profileID string, contentIDs []string) (map[string]EbookReaderProgress, error) {
if s == nil || s.pool == nil {
return nil, fmt.Errorf("ebook reader progress store is not configured")
}
if userID <= 0 || profileID == "" || len(contentIDs) == 0 {
return nil, nil
}
rows, err := s.pool.Query(ctx, `
SELECT user_id, profile_id, content_id, file_id, location, progress, updated_at
FROM ebook_reader_progress
WHERE user_id = $1 AND profile_id = $2 AND content_id = ANY($3::text[])`,
userID, profileID, contentIDs,
)
if err != nil {
return nil, fmt.Errorf("list ebook reader progress: %w", err)
}
defer rows.Close()
progressByContentID := make(map[string]EbookReaderProgress, len(contentIDs))
for rows.Next() {
var progress EbookReaderProgress
if err := rows.Scan(
&progress.UserID,
&progress.ProfileID,
&progress.ContentID,
&progress.FileID,
&progress.Location,
&progress.Progress,
&progress.UpdatedAt,
); err != nil {
return nil, fmt.Errorf("scan ebook reader progress: %w", err)
}
progressByContentID[progress.ContentID] = progress
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate ebook reader progress: %w", err)
}
return progressByContentID, nil
}
func (s *PGEbookReaderProgressStore) Upsert(ctx context.Context, progress EbookReaderProgress) error {
if s == nil || s.pool == nil {
return fmt.Errorf("ebook reader progress store is not configured")
}
if _, err := s.pool.Exec(ctx, `
INSERT INTO ebook_reader_progress
(user_id, profile_id, content_id, file_id, location, progress, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT (user_id, profile_id, content_id) DO UPDATE SET
file_id = EXCLUDED.file_id,
location = EXCLUDED.location,
progress = EXCLUDED.progress,
updated_at = EXCLUDED.updated_at`,
progress.UserID,
progress.ProfileID,
progress.ContentID,
progress.FileID,
progress.Location,
progress.Progress,
progress.UpdatedAt,
); err != nil {
return fmt.Errorf("upsert ebook reader progress: %w", err)
}
return nil
}
+328
View File
@@ -0,0 +1,328 @@
package handlers
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
)
func newEbookReaderAuthRequest(method, path string) *http.Request {
req := httptest.NewRequest(method, path, nil)
ctx := apimw.SetClaims(context.Background(), &auth.Claims{
UserID: 1,
Role: "user",
TokenType: auth.TokenTypeAccess,
})
ctx = apimw.SetProfileID(ctx, "profile-1")
return req.WithContext(ctx)
}
func withEbookReaderRouteParams(req *http.Request, contentID, fileID string) *http.Request {
routeCtx := chi.NewRouteContext()
routeCtx.URLParams.Add("content_id", contentID)
routeCtx.URLParams.Add("file_id", fileID)
return req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, routeCtx))
}
func withEbookReaderContentRouteParam(req *http.Request, contentID string) *http.Request {
routeCtx := chi.NewRouteContext()
routeCtx.URLParams.Add("content_id", contentID)
return req.WithContext(context.WithValue(req.Context(), chi.RouteCtxKey, routeCtx))
}
type fakeEbookProgressStore struct {
progress *EbookReaderProgress
upserted *EbookReaderProgress
err error
}
func (s *fakeEbookProgressStore) Get(
context.Context,
int,
string,
string,
) (*EbookReaderProgress, error) {
return s.progress, s.err
}
func (s *fakeEbookProgressStore) Upsert(_ context.Context, progress EbookReaderProgress) error {
if s.err != nil {
return s.err
}
s.upserted = &progress
return nil
}
func TestEbookReaderServesEbookInlineWithRangeSupport(t *testing.T) {
filePath := writePlaybackTestMediaFile(t, "book.epub")
if err := os.WriteFile(filePath, []byte("0123456789"), 0o644); err != nil {
t.Fatalf("write ebook: %v", err)
}
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{
file: &models.MediaFile{
ID: 42,
ContentID: "ebook-1",
FilePath: filePath,
Container: "epub",
BaseType: "ebook",
},
},
ItemAccess: stubItemAccessChecker{},
})
req := newEbookReaderAuthRequest(http.MethodGet, "/ebooks/ebook-1/files/42/read")
req.Header.Set("Range", "bytes=2-5")
req = withEbookReaderRouteParams(req, "ebook-1", "42")
rr := httptest.NewRecorder()
handler.HandleReadFile(rr, req)
if rr.Code != http.StatusPartialContent {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
if got := rr.Header().Get("Content-Type"); got != "application/epub+zip" {
t.Fatalf("content-type = %q", got)
}
if got := rr.Header().Get("Content-Disposition"); got != `inline; filename="book.epub"` {
t.Fatalf("content-disposition = %q", got)
}
if rr.Body.String() != "2345" {
t.Fatalf("body = %q", rr.Body.String())
}
}
func TestEbookReaderRejectsNonEbookFile(t *testing.T) {
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{
file: &models.MediaFile{
ID: 42,
ContentID: "movie-1",
FilePath: "/tmp/movie.mkv",
Container: "mkv",
BaseType: "movie",
},
},
ItemAccess: stubItemAccessChecker{},
})
req := newEbookReaderAuthRequest(http.MethodGet, "/ebooks/movie-1/files/42/read")
req = withEbookReaderRouteParams(req, "movie-1", "42")
rr := httptest.NewRecorder()
handler.HandleReadFile(rr, req)
if rr.Code != http.StatusNotFound {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
}
func TestEbookReaderRecognizesReadestFormats(t *testing.T) {
cases := map[string]string{
"book.epub": "application/epub+zip",
"book.pdf": "application/pdf",
"book.mobi": "application/x-mobipocket-ebook",
"book.azw": "application/vnd.amazon.ebook",
"book.azw3": "application/vnd.amazon.mobi8-ebook",
"book.cbz": "application/vnd.comicbook+zip",
"book.cbr": "application/vnd.comicbook-rar",
"book.fb2": "application/x-fictionbook+xml",
"book.fbz": "application/x-zip-compressed-fb2",
"book.fb2.zip": "application/x-zip-compressed-fb2",
"book.md": "text/markdown; charset=utf-8",
"book.unknown": "application/octet-stream",
}
for name, wantMime := range cases {
t.Run(name, func(t *testing.T) {
container := name
if name == "book.fb2.zip" {
container = "fbz"
}
file := &models.MediaFile{FilePath: "/library/" + name, Container: container}
if name == "book.unknown" {
if isEbookFile(file) {
t.Fatal("unknown extension should not be treated as an ebook reader format")
}
return
}
file.BaseType = "ebook"
if !isEbookFile(file) {
t.Fatal("expected ebook reader format")
}
if got := ebookMimeType(file.FilePath, file.Container); got != wantMime {
t.Fatalf("ebookMimeType() = %q, want %q", got, wantMime)
}
})
}
}
func TestEbookReaderRejectsPlainText(t *testing.T) {
file := &models.MediaFile{
FilePath: "/library/book.txt",
BaseType: "ebook",
}
if isEbookFile(file) {
t.Fatal("plain text should not be treated as an ebook reader format")
}
if got := ebookMimeType(file.FilePath, file.Container); got == "text/plain; charset=utf-8" {
t.Fatal("plain text should not have an ebook reader MIME type")
}
}
func TestEbookReaderRecognizesFBZFromCompoundFilenameWithoutContainer(t *testing.T) {
file := &models.MediaFile{
FilePath: "/library/book.fb2.zip",
BaseType: "ebook",
}
if !isEbookFile(file) {
t.Fatal("expected .fb2.zip path to be treated as an ebook reader format")
}
}
func TestEbookReaderMapsAccessFailureToNotFound(t *testing.T) {
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{
file: &models.MediaFile{
ID: 42,
ContentID: "ebook-1",
FilePath: "/tmp/book.epub",
Container: "epub",
BaseType: "ebook",
},
},
ItemAccess: stubItemAccessChecker{err: catalog.ErrItemNotFound},
})
req := newEbookReaderAuthRequest(http.MethodGet, "/ebooks/ebook-1/files/42/read")
req = withEbookReaderRouteParams(req, "ebook-1", "42")
rr := httptest.NewRecorder()
handler.HandleReadFile(rr, req)
if rr.Code != http.StatusNotFound {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
}
func TestEbookReaderReturnsInternalErrorForUnexpectedAuthorizeFailure(t *testing.T) {
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{err: errors.New("db down")},
ItemAccess: stubItemAccessChecker{},
})
req := newEbookReaderAuthRequest(http.MethodGet, "/ebooks/ebook-1/files/42/read")
req = withEbookReaderRouteParams(req, "ebook-1", "42")
rr := httptest.NewRecorder()
handler.HandleReadFile(rr, req)
if rr.Code != http.StatusInternalServerError {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
}
func TestEbookReaderSavesProgressForAuthorizedEbookFile(t *testing.T) {
store := &fakeEbookProgressStore{}
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{
file: &models.MediaFile{
ID: 42,
ContentID: "ebook-1",
FilePath: "/tmp/book.epub",
Container: "epub",
BaseType: "ebook",
},
},
ItemAccess: stubItemAccessChecker{},
})
handler.ProgressStore = store
req := httptest.NewRequest(
http.MethodPut,
"/ebooks/ebook-1/progress",
strings.NewReader(`{"file_id":42,"location":"epubcfi(/6/4)","progress":0.42}`),
)
ctx := apimw.SetClaims(context.Background(), &auth.Claims{
UserID: 1,
Role: "user",
TokenType: auth.TokenTypeAccess,
})
req = req.WithContext(apimw.SetProfileID(ctx, "profile-9"))
req = withEbookReaderContentRouteParam(req, "ebook-1")
rr := httptest.NewRecorder()
handler.HandleSaveProgress(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
if store.upserted == nil {
t.Fatal("expected progress to be saved")
}
if store.upserted.UserID != 1 || store.upserted.ProfileID != "profile-9" {
t.Fatalf("scope = user %d profile %q", store.upserted.UserID, store.upserted.ProfileID)
}
if store.upserted.ContentID != "ebook-1" || store.upserted.FileID != 42 {
t.Fatalf("target = content %q file %d", store.upserted.ContentID, store.upserted.FileID)
}
if store.upserted.Location != "epubcfi(/6/4)" || store.upserted.Progress != 0.42 {
t.Fatalf("progress = %+v", store.upserted)
}
}
func TestEbookReaderReturnsSavedProgress(t *testing.T) {
updatedAt := time.Date(2026, 6, 8, 12, 0, 0, 0, time.UTC)
handler := NewEbookReaderHandler(&MediaFileAuthorizer{
FileResolver: stubMediaFileResolver{},
ItemAccess: stubItemAccessChecker{},
})
handler.ProgressStore = &fakeEbookProgressStore{
progress: &EbookReaderProgress{
UserID: 1,
ProfileID: "profile-1",
ContentID: "ebook-1",
FileID: 42,
Location: "epubcfi(/6/8)",
Progress: 0.75,
UpdatedAt: updatedAt,
},
}
req := newEbookReaderAuthRequest(http.MethodGet, "/ebooks/ebook-1/progress")
req = withEbookReaderContentRouteParam(req, "ebook-1")
rr := httptest.NewRecorder()
handler.HandleGetProgress(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
body := rr.Body.String()
for _, want := range []string{
`"content_id":"ebook-1"`,
`"file_id":42`,
`"location":"epubcfi(/6/8)"`,
`"progress":0.75`,
`"updated_at":"2026-06-08T12:00:00Z"`,
} {
if !strings.Contains(body, want) {
t.Fatalf("response %s missing %s", body, want)
}
}
}
+9 -1
View File
@@ -40,6 +40,7 @@ type PersonalDataHandler struct {
localFavoriteDispatcher LocalFavoriteEventDispatcher
profileStaler ProfileStaler
profileRefreshRequester ProfileRefreshRequester
ebookProgressStore EbookReaderProgressLister
}
// NewPersonalDataHandler creates a new PersonalDataHandler.
@@ -55,6 +56,10 @@ func (h *PersonalDataHandler) SetDetailService(svc *catalog.DetailService) {
h.detailSvc = svc
}
func (h *PersonalDataHandler) SetEbookReaderProgressStore(store EbookReaderProgressLister) {
h.ebookProgressStore = store
}
// SetProfileStaler configures an optional staleness trigger for taste profiles.
func (h *PersonalDataHandler) SetProfileStaler(ps ProfileStaler) {
h.profileStaler = ps
@@ -552,7 +557,10 @@ func resolveItemsByIDs(h *PersonalDataHandler, r *http.Request, ids []string) ([
userStates := map[string]*itemUserStateResponse{}
if store, profileID, ok := h.userStoreForRequest(r); ok {
if resolvedStates, err := resolveItemUserStates(r.Context(), store, profileID, h.episodeRepo, accessibleItems); err == nil {
if resolvedStates, err := resolveItemUserStatesWithOptions(r.Context(), store, profileID, h.episodeRepo, accessibleItems, itemUserStateOptions{
UserID: apimw.GetUserID(r.Context()),
EbookProgressStore: h.ebookProgressStore,
}); err == nil {
userStates = resolvedStates
}
}
+86 -7
View File
@@ -46,6 +46,10 @@ type LocalWatchEventDispatcher interface {
HandleLocalWatchEvent(ctx context.Context, event watchsync.LocalWatchEvent) error
}
type EbookReaderProgressLister interface {
ListByContentIDs(ctx context.Context, userID int, profileID string, contentIDs []string) (map[string]EbookReaderProgress, error)
}
// ItemsHandler handles browse, search, item detail, and series endpoints.
type ItemsHandler struct {
browseRepo *catalog.BrowseRepository
@@ -62,6 +66,7 @@ type ItemsHandler struct {
profileRefreshRequester ProfileRefreshRequester
metadataRefreshRequester MetadataRefreshRequester
localWatchDispatcher LocalWatchEventDispatcher
ebookProgressStore EbookReaderProgressLister
EventsHub *evt.Hub
UserRepo *auth.UserRepository
}
@@ -114,6 +119,10 @@ func (h *ItemsHandler) SetLocalWatchEventDispatcher(dispatcher LocalWatchEventDi
h.localWatchDispatcher = dispatcher
}
func (h *ItemsHandler) SetEbookReaderProgressStore(store EbookReaderProgressLister) {
h.ebookProgressStore = store
}
func (h *ItemsHandler) maybeRequestStaleDetailMetadataRefresh(ctx context.Context, detail *catalog.ItemDetail) {
if h == nil || detail == nil || h.metadataRefreshRequester == nil {
return
@@ -373,8 +382,8 @@ func (h *ItemsHandler) HandleGetWatchDetail(w http.ResponseWriter, r *http.Reque
}
}
if detail.Type == "movie" || detail.Type == "episode" {
detail.UserData = h.getLeafUserData(r, detail.ContentID)
if detail.Type == "movie" || detail.Type == "episode" || detail.Type == "ebook" {
detail.UserData = h.getLeafUserData(r, detail.ContentID, detail.Type)
applyEffectiveEditionPreference(detail.UserData, &detail.EffectiveVersionEditionKey)
}
@@ -739,7 +748,10 @@ func (h *ItemsHandler) listItemUserStates(r *http.Request, items []*models.Media
if !ok {
return map[string]*itemUserStateResponse{}
}
states, err := resolveItemUserStates(r.Context(), store, profileID, h.episodeRepo, items)
states, err := resolveItemUserStatesWithOptions(r.Context(), store, profileID, h.episodeRepo, items, itemUserStateOptions{
UserID: apimw.GetUserID(r.Context()),
EbookProgressStore: h.ebookProgressStore,
})
if err != nil {
return map[string]*itemUserStateResponse{}
}
@@ -850,6 +862,7 @@ func (h *ItemsHandler) listSortMetrics(
filter catalog.AccessFilter,
overlaySummaries map[string]*models.OverlaySummary,
store userstore.UserStore,
userID int,
profileID string,
) map[string]*sortMetricsResponse {
metrics := make(map[string]*sortMetricsResponse, len(items))
@@ -890,7 +903,7 @@ func (h *ItemsHandler) listSortMetrics(
}
}
case "progress", "date_viewed", "plays":
h.listUserSortMetrics(ctx, items, sortField, store, profileID, metrics)
h.listUserSortMetrics(ctx, items, sortField, store, userID, profileID, metrics)
}
return metrics
}
@@ -900,6 +913,7 @@ func (h *ItemsHandler) listUserSortMetrics(
items []*models.MediaItem,
sortField string,
store userstore.UserStore,
userID int,
profileID string,
metrics map[string]*sortMetricsResponse,
) {
@@ -915,6 +929,10 @@ func (h *ItemsHandler) listUserSortMetrics(
if err != nil {
return
}
ebookProgressMap, err := h.listEbookReaderProgressByContentIDs(ctx, userID, profileID, contentIDs)
if err != nil {
return
}
switch sortField {
case "progress":
@@ -923,10 +941,16 @@ func (h *ItemsHandler) listUserSortMetrics(
continue
}
progress, ok := progressMap[item.ContentID]
if !ok || progress.Completed || progress.PositionSeconds <= 0 || progress.DurationSeconds <= 0 {
if ok && !progress.Completed && progress.PositionSeconds > 0 && progress.DurationSeconds > 0 {
ratio := progress.PositionSeconds / progress.DurationSeconds
metrics[item.ContentID] = &sortMetricsResponse{ProgressRatio: &ratio}
continue
}
ratio := progress.PositionSeconds / progress.DurationSeconds
ebookProgress, ok := ebookProgressMap[item.ContentID]
if !ok || ebookProgress.Progress <= 0 || ebookProgress.Progress >= 0.9 {
continue
}
ratio := ebookProgress.Progress
metrics[item.ContentID] = &sortMetricsResponse{ProgressRatio: &ratio}
}
case "date_viewed", "plays":
@@ -955,6 +979,12 @@ func (h *ItemsHandler) listUserSortMetrics(
if progress, ok := progressMap[item.ContentID]; ok && progress.Completed && progress.UpdatedAt > viewedAt {
viewedAt = progress.UpdatedAt
}
if progress, ok := ebookProgressMap[item.ContentID]; ok && progress.Progress >= 0.9 {
ebookViewedAt := progress.UpdatedAt.UTC().Format(time.RFC3339)
if ebookViewedAt > viewedAt {
viewedAt = ebookViewedAt
}
}
if viewedAt == "" {
continue
}
@@ -964,6 +994,9 @@ func (h *ItemsHandler) listUserSortMetrics(
if progress, ok := progressMap[item.ContentID]; ok && progress.Completed && playCount < 1 {
playCount = 1
}
if progress, ok := ebookProgressMap[item.ContentID]; ok && progress.Progress >= 0.9 && playCount < 1 {
playCount = 1
}
if playCount <= 0 {
continue
}
@@ -974,6 +1007,18 @@ func (h *ItemsHandler) listUserSortMetrics(
}
}
func (h *ItemsHandler) listEbookReaderProgressByContentIDs(
ctx context.Context,
userID int,
profileID string,
contentIDs []string,
) (map[string]EbookReaderProgress, error) {
if h == nil || h.ebookProgressStore == nil || userID <= 0 || profileID == "" || len(contentIDs) == 0 {
return nil, nil
}
return h.ebookProgressStore.ListByContentIDs(ctx, userID, profileID, contentIDs)
}
func uniqueItemContentIDs(items []*models.MediaItem) []string {
contentIDs := make([]string, 0, len(items))
seen := make(map[string]struct{}, len(items))
@@ -1110,7 +1155,11 @@ func (h *ItemsHandler) toSeasonResponse(r *http.Request, seriesID string, s *mod
return resp
}
func (h *ItemsHandler) getLeafUserData(r *http.Request, contentID string) *catalog.SeasonUserData {
func (h *ItemsHandler) getLeafUserData(r *http.Request, contentID string, itemType ...string) *catalog.SeasonUserData {
if len(itemType) > 0 && itemType[0] == "ebook" {
return h.getEbookLeafUserData(r, contentID)
}
store, profileID, ok := h.userStoreForRequest(r)
if !ok {
return nil
@@ -1134,6 +1183,36 @@ func (h *ItemsHandler) getLeafUserData(r *http.Request, contentID string) *catal
}
}
func (h *ItemsHandler) getEbookLeafUserData(r *http.Request, contentID string) *catalog.SeasonUserData {
if h == nil || h.ebookProgressStore == nil {
return nil
}
userID := apimw.GetUserID(r.Context())
profileID := apimw.GetProfileID(r.Context())
if profileID == "" {
profileID = r.Header.Get("X-Profile-Id")
}
if userID <= 0 || profileID == "" || contentID == "" {
return nil
}
progress, err := h.ebookProgressStore.ListByContentIDs(r.Context(), userID, profileID, []string{contentID})
if err != nil {
return nil
}
row, ok := progress[contentID]
if !ok || row.Progress <= 0 {
return nil
}
return &catalog.SeasonUserData{
PositionSeconds: row.Progress,
DurationSeconds: 1,
IsInProgress: row.Progress < 0.9,
Played: row.Progress >= 0.9,
}
}
func applyEffectiveEditionPreference(userData *catalog.SeasonUserData, target **string) {
if userData == nil || userData.LastEditionKey == nil || *userData.LastEditionKey == "" {
return
@@ -40,20 +40,86 @@ func TestListSortMetricsIncludesUserStateSorts(t *testing.T) {
}
}
progressMetrics := handler.listSortMetrics(ctx, items, "progress", catalog.AccessFilter{}, nil, store, "profile-1")
progressMetrics := handler.listSortMetrics(ctx, items, "progress", catalog.AccessFilter{}, nil, store, 0, "profile-1")
progress := progressMetrics["movie-progress"]
if progress == nil || progress.ProgressRatio == nil || *progress.ProgressRatio != 0.25 {
t.Fatalf("progress metrics = %#v", progress)
}
viewedMetrics := handler.listSortMetrics(ctx, items, "date_viewed", catalog.AccessFilter{}, nil, store, "profile-1")
viewedMetrics := handler.listSortMetrics(ctx, items, "date_viewed", catalog.AccessFilter{}, nil, store, 0, "profile-1")
if got := viewedMetrics["movie-viewed"]; got == nil || got.ViewedAt != "2026-05-29T18:30:00Z" {
t.Fatalf("viewed metrics = %#v", got)
}
playsMetrics := handler.listSortMetrics(ctx, items, "plays", catalog.AccessFilter{}, nil, store, "profile-1")
playsMetrics := handler.listSortMetrics(ctx, items, "plays", catalog.AccessFilter{}, nil, store, 0, "profile-1")
plays := playsMetrics["movie-plays"]
if plays == nil || plays.PlayCount == nil || *plays.PlayCount != 2 {
t.Fatalf("plays metrics = %#v", plays)
}
}
func TestListSortMetricsIncludesEbookReaderProgress(t *testing.T) {
ctx := context.Background()
store := newProfileTestStore(t)
updatedAt := time.Date(2026, 6, 3, 9, 15, 0, 0, time.UTC)
handler := &ItemsHandler{
ebookProgressStore: &fakeEbookReaderProgressLister{
progress: map[string]EbookReaderProgress{
"ebook-progress": {
UserID: 42,
ProfileID: "profile-1",
ContentID: "ebook-progress",
Progress: 0.37,
UpdatedAt: updatedAt,
},
"ebook-complete": {
UserID: 42,
ProfileID: "profile-1",
ContentID: "ebook-complete",
Progress: 0.95,
UpdatedAt: updatedAt.Add(time.Hour),
},
},
},
}
items := []*models.MediaItem{
{ContentID: "ebook-progress", Type: "ebook", Title: "Progress"},
{ContentID: "ebook-complete", Type: "ebook", Title: "Complete"},
}
progressMetrics := handler.listSortMetrics(ctx, items, "progress", catalog.AccessFilter{}, nil, store, 42, "profile-1")
progress := progressMetrics["ebook-progress"]
if progress == nil || progress.ProgressRatio == nil || *progress.ProgressRatio != 0.37 {
t.Fatalf("progress metrics = %#v", progress)
}
viewedMetrics := handler.listSortMetrics(ctx, items, "date_viewed", catalog.AccessFilter{}, nil, store, 42, "profile-1")
if got := viewedMetrics["ebook-complete"]; got == nil || got.ViewedAt != "2026-06-03T10:15:00Z" {
t.Fatalf("viewed metrics = %#v", got)
}
playsMetrics := handler.listSortMetrics(ctx, items, "plays", catalog.AccessFilter{}, nil, store, 42, "profile-1")
plays := playsMetrics["ebook-complete"]
if plays == nil || plays.PlayCount == nil || *plays.PlayCount != 1 {
t.Fatalf("plays metrics = %#v", plays)
}
}
type fakeEbookReaderProgressLister struct {
progress map[string]EbookReaderProgress
}
func (f *fakeEbookReaderProgressLister) ListByContentIDs(
_ context.Context,
_ int,
_ string,
contentIDs []string,
) (map[string]EbookReaderProgress, error) {
result := make(map[string]EbookReaderProgress, len(contentIDs))
for _, contentID := range contentIDs {
if progress, ok := f.progress[contentID]; ok {
result[contentID] = progress
}
}
return result, nil
}
@@ -0,0 +1,47 @@
package handlers
import (
"net/http/httptest"
"testing"
"time"
apimw "github.com/Silo-Server/silo-server/internal/api/middleware"
"github.com/Silo-Server/silo-server/internal/auth"
)
func TestGetLeafUserDataUsesEbookReaderProgress(t *testing.T) {
handler := &ItemsHandler{
ebookProgressStore: &fakeEbookReaderProgressLister{
progress: map[string]EbookReaderProgress{
"ebook-progress": {
UserID: 7,
ProfileID: "profile-1",
ContentID: "ebook-progress",
Progress: 0.42,
UpdatedAt: time.Date(2026, 6, 5, 10, 0, 0, 0, time.UTC),
},
"ebook-complete": {
UserID: 7,
ProfileID: "profile-1",
ContentID: "ebook-complete",
Progress: 0.93,
UpdatedAt: time.Date(2026, 6, 5, 11, 0, 0, 0, time.UTC),
},
},
},
}
req := httptest.NewRequest("GET", "/items/ebook-progress", nil)
ctx := apimw.SetClaims(req.Context(), &auth.Claims{UserID: 7})
ctx = apimw.SetProfileID(ctx, "profile-1")
req = req.WithContext(ctx)
progress := handler.getLeafUserData(req, "ebook-progress", "ebook")
if progress == nil || progress.Played || !progress.IsInProgress || progress.PositionSeconds != 0.42 || progress.DurationSeconds != 1 {
t.Fatalf("partial ebook user data = %#v", progress)
}
complete := handler.getLeafUserData(req, "ebook-complete", "ebook")
if complete == nil || !complete.Played || complete.IsInProgress {
t.Fatalf("completed ebook user data = %#v", complete)
}
}
+19 -12
View File
@@ -2029,18 +2029,8 @@ func (h *LibraryHandler) seedDefaultChain(ctx context.Context, libraryType strin
return nil
}
// Determine which content levels apply.
var levels []string
switch libraryType {
case "series":
levels = []string{"series", "season", "episode"}
case "movies", "movie":
levels = []string{"movie"}
case "audiobooks", "audiobook":
levels = []string{"audiobook"}
case "mixed":
levels = []string{"movie", "series", "season", "episode", "audiobook"}
default:
levels := metadataContentLevelsForLibraryType(libraryType)
if len(levels) == 0 {
return nil
}
@@ -2093,6 +2083,23 @@ func (h *LibraryHandler) seedDefaultChain(ctx context.Context, libraryType strin
return entries
}
func metadataContentLevelsForLibraryType(libraryType string) []string {
switch libraryType {
case "series":
return []string{"series", "season", "episode"}
case "movies", "movie":
return []string{"movie"}
case "audiobooks", "audiobook":
return []string{"audiobook"}
case "ebooks", "ebook":
return []string{"ebook"}
case "mixed":
return []string{"movie", "series", "season", "episode", "audiobook", "ebook"}
default:
return nil
}
}
// HandleListStaleIDs handles GET /libraries/stale-ids.
func (h *LibraryHandler) HandleListStaleIDs(w http.ResponseWriter, r *http.Request) {
if h.StaleIDRepo == nil {
@@ -0,0 +1,26 @@
package handlers
import (
"reflect"
"testing"
)
func TestMetadataContentLevelsForLibraryTypeIncludesEbooks(t *testing.T) {
cases := []struct {
name string
libraryType string
want []string
}{
{name: "plural ebooks", libraryType: "ebooks", want: []string{"ebook"}},
{name: "singular ebook", libraryType: "ebook", want: []string{"ebook"}},
{name: "mixed includes ebook", libraryType: "mixed", want: []string{"movie", "series", "season", "episode", "audiobook", "ebook"}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := metadataContentLevelsForLibraryType(tc.libraryType); !reflect.DeepEqual(got, tc.want) {
t.Fatalf("metadataContentLevelsForLibraryType(%q) = %#v, want %#v", tc.libraryType, got, tc.want)
}
})
}
}
+5 -1
View File
@@ -48,6 +48,7 @@ type RecommendationsHandler struct {
EpisodeRepo *catalog.EpisodeRepository
WatchTonightFetcher watchTonightSectionFetcher
CastFetcher cardsCastFetcher
EbookProgress EbookReaderProgressLister
// RecWorker enqueues asynchronous taste-profile refreshes after writes
// (taste seeding). Optional — when nil, refresh is simply skipped.
RecWorker ProfileRefreshRequester
@@ -643,7 +644,10 @@ func (h *RecommendationsHandler) loadItemEnrichment(
if h.storeProvider != nil {
store, storeErr := h.storeProvider.ForUser(ctx, userID)
if storeErr == nil && store != nil {
states, stateErr := resolveItemUserStates(ctx, store, profileID, h.EpisodeRepo, mediaItems)
states, stateErr := resolveItemUserStatesWithOptions(ctx, store, profileID, h.EpisodeRepo, mediaItems, itemUserStateOptions{
UserID: userID,
EbookProgressStore: h.EbookProgress,
})
if stateErr == nil {
out.states = states
}
+5 -1
View File
@@ -34,6 +34,7 @@ type SectionHandler struct {
DetailSvc *catalog.DetailService
Settings *catalog.ServerSettingsRepo
CollectionRepo *catalog.LibraryCollectionRepository
EbookProgress EbookReaderProgressLister
}
// NewSectionHandler creates a new SectionHandler.
@@ -1377,7 +1378,10 @@ func (h *SectionHandler) listSectionItemUserStates(r *http.Request, items []*mod
if err != nil || store == nil {
return map[string]*itemUserStateResponse{}
}
states, err := resolveItemUserStates(r.Context(), store, profileID, h.EpisodeRepo, items)
states, err := resolveItemUserStatesWithOptions(r.Context(), store, profileID, h.EpisodeRepo, items, itemUserStateOptions{
UserID: userID,
EbookProgressStore: h.EbookProgress,
})
if err != nil {
return map[string]*itemUserStateResponse{}
}
+4 -1
View File
@@ -181,7 +181,10 @@ func (h *RecommendationsHandler) resolveTasteSeedUserStates(ctx context.Context,
if err != nil || store == nil {
return nil
}
states, err := resolveItemUserStates(ctx, store, profileID, h.EpisodeRepo, mediaItems)
states, err := resolveItemUserStatesWithOptions(ctx, store, profileID, h.EpisodeRepo, mediaItems, itemUserStateOptions{
UserID: userID,
EbookProgressStore: h.EbookProgress,
})
if err != nil {
return nil
}
+34
View File
@@ -14,12 +14,28 @@ type itemUserStateResponse struct {
InWatchlist bool `json:"in_watchlist"`
}
type itemUserStateOptions struct {
UserID int
EbookProgressStore EbookReaderProgressLister
}
func resolveItemUserStates(
ctx context.Context,
store userstore.UserStore,
profileID string,
episodeRepo *catalog.EpisodeRepository,
items []*models.MediaItem,
) (map[string]*itemUserStateResponse, error) {
return resolveItemUserStatesWithOptions(ctx, store, profileID, episodeRepo, items, itemUserStateOptions{})
}
func resolveItemUserStatesWithOptions(
ctx context.Context,
store userstore.UserStore,
profileID string,
episodeRepo *catalog.EpisodeRepository,
items []*models.MediaItem,
options itemUserStateOptions,
) (map[string]*itemUserStateResponse, error) {
result := make(map[string]*itemUserStateResponse, len(items))
if store == nil || profileID == "" || len(items) == 0 {
@@ -111,6 +127,10 @@ func resolveItemUserStates(
if err != nil {
return nil, err
}
ebookProgressMap, err := resolveEbookProgressForUserStates(ctx, options, profileID, contentIDs)
if err != nil {
return nil, err
}
for _, item := range items {
if item == nil || item.ContentID == "" {
@@ -125,6 +145,8 @@ func resolveItemUserStates(
state.Played = allEpisodesCompleted(seriesEpisodes[item.ContentID], progressMap)
case "season":
state.Played = allEpisodesCompleted(seasonEpisodes[item.ContentID], progressMap)
case "ebook":
state.Played = ebookProgressMap[item.ContentID].Progress >= 0.9
default:
state.Played = progressMap[item.ContentID].Completed
}
@@ -134,6 +156,18 @@ func resolveItemUserStates(
return result, nil
}
func resolveEbookProgressForUserStates(
ctx context.Context,
options itemUserStateOptions,
profileID string,
contentIDs []string,
) (map[string]EbookReaderProgress, error) {
if options.EbookProgressStore == nil || options.UserID <= 0 || profileID == "" || len(contentIDs) == 0 {
return nil, nil
}
return options.EbookProgressStore.ListByContentIDs(ctx, options.UserID, profileID, contentIDs)
}
func allEpisodesCompleted(episodes []*models.Episode, progressMap map[string]userstore.WatchProgress) bool {
if len(episodes) == 0 {
return false
+50
View File
@@ -0,0 +1,50 @@
package handlers
import (
"context"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestResolveItemUserStatesIncludesCompletedEbookReaderProgress(t *testing.T) {
ctx := context.Background()
store := newProfileTestStore(t)
items := []*models.MediaItem{
{ContentID: "ebook-complete", Type: "ebook", Title: "Complete Ebook"},
{ContentID: "ebook-progress", Type: "ebook", Title: "Partial Ebook"},
}
states, err := resolveItemUserStatesWithOptions(ctx, store, "profile-1", nil, items, itemUserStateOptions{
UserID: 42,
EbookProgressStore: &fakeEbookReaderProgressLister{
progress: map[string]EbookReaderProgress{
"ebook-complete": {
UserID: 42,
ProfileID: "profile-1",
ContentID: "ebook-complete",
Progress: 0.95,
UpdatedAt: time.Date(2026, 6, 4, 10, 0, 0, 0, time.UTC),
},
"ebook-progress": {
UserID: 42,
ProfileID: "profile-1",
ContentID: "ebook-progress",
Progress: 0.45,
UpdatedAt: time.Date(2026, 6, 4, 9, 0, 0, 0, time.UTC),
},
},
},
})
if err != nil {
t.Fatalf("resolveItemUserStatesWithOptions: %v", err)
}
if states["ebook-complete"] == nil || !states["ebook-complete"].Played {
t.Fatalf("completed ebook state = %+v, want played", states["ebook-complete"])
}
if states["ebook-progress"] == nil || states["ebook-progress"].Played {
t.Fatalf("partial ebook state = %+v, want not played", states["ebook-progress"])
}
}
+4 -1
View File
@@ -362,7 +362,10 @@ func (h *RecommendationsHandler) enrichItems(r *http.Request, userID int, profil
if h.storeProvider != nil {
store, storeErr := h.storeProvider.ForUser(ctx, userID)
if storeErr == nil && store != nil {
states, stateErr := resolveItemUserStates(ctx, store, profileID, h.EpisodeRepo, mediaItems)
states, stateErr := resolveItemUserStatesWithOptions(ctx, store, profileID, h.EpisodeRepo, mediaItems, itemUserStateOptions{
UserID: userID,
EbookProgressStore: h.EbookProgress,
})
if stateErr == nil {
stateMap = states
}
+35
View File
@@ -368,7 +368,10 @@ func NewRouter(deps Dependencies) chi.Router {
var webhookSyncHandler *handlers.WebhookSyncHandler
var requestHandler *handlers.RequestsHandler
var autoscanHandler *handlers.AutoscanHandler
var ebookReaderHandler *handlers.EbookReaderHandler
var ebookProgressStore *handlers.PGEbookReaderProgressStore
if deps.DB != nil {
ebookProgressStore = handlers.NewPGEbookReaderProgressStore(deps.DB)
browseRepo := catalog.NewBrowseRepository(deps.DB)
itemRepo = catalog.NewItemRepository(deps.DB)
episodeRepo = catalog.NewEpisodeRepository(deps.DB)
@@ -419,6 +422,19 @@ func NewRouter(deps Dependencies) chi.Router {
if dispatcher, ok := deps.WatchProviderService.(handlers.LocalWatchEventDispatcher); ok {
itemsHandler.SetLocalWatchEventDispatcher(dispatcher)
}
if ebookProgressStore != nil {
itemsHandler.SetEbookReaderProgressStore(ebookProgressStore)
}
if deps.FileRepo != nil {
ebookReaderHandler = handlers.NewEbookReaderHandler(&handlers.MediaFileAuthorizer{
FileResolver: deps.FileRepo,
ItemAccess: itemRepo,
EpisodeLookup: episodeRepo,
})
if ebookProgressStore != nil {
ebookReaderHandler.ProgressStore = ebookProgressStore
}
}
catalogResourceHandler = handlers.NewCatalogResourceHandler(itemsHandler)
catalogHandler = handlers.NewCatalogHandler(
catalog.NewCatalogResolver(browseRepo, itemRepo).
@@ -498,6 +514,9 @@ func NewRouter(deps Dependencies) chi.Router {
if detailSvc != nil {
personalDataHandler.SetDetailService(detailSvc)
}
if ebookProgressStore != nil {
personalDataHandler.SetEbookReaderProgressStore(ebookProgressStore)
}
personalDataHandler.SetEpisodeRepo(episodeRepo)
personalDataHandler.SetSeasonRepo(seasonRepo)
personalDataHandler.EventsHub = deps.EventsHub
@@ -937,6 +956,9 @@ func NewRouter(deps Dependencies) chi.Router {
}
sectionHandler.EpisodeRepo = episodeRepo
sectionHandler.DetailSvc = detailSvc
if ebookProgressStore != nil {
sectionHandler.EbookProgress = ebookProgressStore
}
if userRepo != nil {
sectionHandler.UserRepo = userRepo
}
@@ -1117,6 +1139,9 @@ func NewRouter(deps Dependencies) chi.Router {
}
recsHandler.CalendarRepo = calendarRepo
recsHandler.EpisodeRepo = episodeRepo
if ebookProgressStore != nil {
recsHandler.EbookProgress = ebookProgressStore
}
if deps.PersonRepo != nil {
recsHandler.CastFetcher = deps.PersonRepo
}
@@ -1703,6 +1728,16 @@ func NewRouter(deps Dependencies) chi.Router {
})
}
if ebookReaderHandler != nil {
r.Route("/ebooks", func(r chi.Router) {
r.Use(apimw.RequireProfile)
r.Get("/{content_id}/files/{file_id}/read", ebookReaderHandler.HandleReadFile)
r.Head("/{content_id}/files/{file_id}/read", ebookReaderHandler.HandleReadFile)
r.Get("/{content_id}/progress", ebookReaderHandler.HandleGetProgress)
r.Put("/{content_id}/progress", ebookReaderHandler.HandleSaveProgress)
})
}
// Subtitle search + AI translation routes.
if subtitleSearchHandler != nil {
if deps.FileRepo != nil && itemRepo != nil {
+13 -6
View File
@@ -759,9 +759,9 @@ func listDistinctPeopleByKindWithSource(
}
// listDistinctAudiobookSeriesWithSource returns distinct series_name values
// from audiobook_series joined onto the scoped result set. Names are trimmed
// from the active book series table joined onto the scoped result set. Names are trimmed
// and case-folded for sort so the picker doesn't show duplicates that differ
// only by whitespace or casing — the literal trimmed series_name is still
// only by whitespace or casing; the literal trimmed series_name is still
// returned so existing rules continue to match.
//
// The DISTINCT happens in an inline subquery (rather than directly on the
@@ -783,14 +783,14 @@ func listDistinctAudiobookSeriesWithSource(
SELECT name FROM (
SELECT DISTINCT BTRIM(s.series_name) AS name
FROM %s
JOIN audiobook_series s ON s.content_id = mi.content_id
JOIN %s s ON s.content_id = mi.content_id
%s
AND s.series_name IS NOT NULL
AND BTRIM(s.series_name) <> ''
) names
ORDER BY LOWER(name) ASC
LIMIT %d
`, fromClause, browseFilterPrefix(whereClause), catalogFacetMaxValues)
`, fromClause, bookSeriesTableForMediaScope(mediaScope), browseFilterPrefix(whereClause), catalogFacetMaxValues)
return queryDistinctStrings(ctx, pool, query, args)
}
@@ -946,7 +946,7 @@ func searchDistinctAudiobookSeriesWithSource(
SELECT name FROM (
SELECT DISTINCT BTRIM(s.series_name) AS name
FROM %s
JOIN audiobook_series s ON s.content_id = mi.content_id
JOIN %s s ON s.content_id = mi.content_id
%s
AND s.series_name IS NOT NULL
AND BTRIM(s.series_name) <> ''
@@ -954,10 +954,17 @@ func searchDistinctAudiobookSeriesWithSource(
) names
ORDER BY LOWER(name) ASC
LIMIT %d
`, fromClause, browseFilterPrefix(whereClause), prefixIdx, limit+1)
`, fromClause, bookSeriesTableForMediaScope(mediaScope), browseFilterPrefix(whereClause), prefixIdx, limit+1)
return queryFacetSearchResults(ctx, pool, query, args, limit)
}
func bookSeriesTableForMediaScope(mediaScope string) string {
if mediaScope == "ebook" {
return "ebook_series"
}
return "audiobook_series"
}
// queryFacetSearchResults executes a facet search query that was built
// with LIMIT N+1 and returns the first N rows plus a hasMore flag
// indicating whether an additional row was present (i.e. the result set
+1 -1
View File
@@ -390,7 +390,7 @@ func normalizeCatalogMatch(raw string) string {
func parseCatalogMediaScope(raw string) string {
switch strings.ToLower(strings.TrimSpace(raw)) {
case "movie", "series", "episode", "audiobook":
case "movie", "series", "episode", "audiobook", "ebook":
return strings.ToLower(strings.TrimSpace(raw))
default:
return ""
+9
View File
@@ -0,0 +1,9 @@
package catalog
import "testing"
func TestParseCatalogMediaScope_AllowsEbook(t *testing.T) {
if got := parseCatalogMediaScope(" ebook "); got != "ebook" {
t.Fatalf("expected ebook media scope, got %q", got)
}
}
+23 -19
View File
@@ -39,10 +39,9 @@ type CatalogFiltersResult struct {
Resolutions []string
AudioLanguages []string
SubtitleLanguages []string
// Authors, Narrators and Series are audiobook-native facets. Populated
// whenever the scope contains audiobook items; empty for video-only
// scopes. They share the item_people / audiobook_series tables so
// callers don't need to switch on media type.
// Authors, Narrators and Series are book-native facets. Authors and
// Narrators use item_people; Series uses the active book series table.
// Video-only scopes return these empty.
Authors []string
Narrators []string
Series []string
@@ -64,8 +63,8 @@ type facetFetcher interface {
// PeopleByKind returns distinct people names for the given PersonKind
// across the scoped result set. Used for Authors / Narrators facets.
PeopleByKind(ctx context.Context, kind models.PersonKind, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error)
// AudiobookSeries returns distinct series_name values from
// audiobook_series joined onto the scoped result set.
// AudiobookSeries returns distinct series_name values from the active
// book series table joined onto the scoped result set.
AudiobookSeries(ctx context.Context, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error)
// SearchDistinctArrayColumn prefix-searches an array-column facet
// (genres, studios, networks, countries). Returns up to limit
@@ -710,7 +709,7 @@ func (r *CatalogResolver) ListFiltersWithOptions(ctx context.Context, req Catalo
return r.listFiltersForSource(ctx, filters, options, episodeCatalogBaseRelation, req.Query.MediaScope)
}
return r.listFiltersForSource(ctx, filters, options, "media_items mi", "")
return r.listFiltersForSource(ctx, filters, options, "media_items mi", req.Query.MediaScope)
}
// CatalogFacetSearchResult is the typed return for SearchFacet. Matches
@@ -798,10 +797,9 @@ func (r *CatalogResolver) SearchFacet(ctx context.Context, req CatalogRequest, a
}
baseRelation := "media_items mi"
mediaScope := ""
mediaScope := req.Query.MediaScope
if isEpisodeCatalogScope(req.Query.MediaScope) {
baseRelation = episodeCatalogBaseRelation
mediaScope = req.Query.MediaScope
}
facets := r.facets
@@ -837,6 +835,9 @@ func dispatchFacetSearch(ctx context.Context, facets facetFetcher, facet string,
case "author":
return facets.SearchPeopleByKind(ctx, models.PersonKindAuthor, filters, baseRelation, mediaScope, prefix, limit)
case "narrator":
if mediaScope == "ebook" {
return nil, false, fmt.Errorf("%w: narrator facet is not available for ebook scope", ErrInvalidCatalogRequest)
}
return facets.SearchPeopleByKind(ctx, models.PersonKindNarrator, filters, baseRelation, mediaScope, prefix, limit)
case "series":
return facets.SearchAudiobookSeries(ctx, filters, baseRelation, mediaScope, prefix, limit)
@@ -958,14 +959,16 @@ func (r *CatalogResolver) listFiltersForSource(
authors = out
return nil
}))
eg.Go(withLimit(func() error {
out, err := facets.PeopleByKind(gctx, models.PersonKindNarrator, filters, baseRelation, mediaScope)
if err != nil {
return fmt.Errorf("listing catalog narrators: %w", err)
}
narrators = out
return nil
}))
if mediaScope != "ebook" {
eg.Go(withLimit(func() error {
out, err := facets.PeopleByKind(gctx, models.PersonKindNarrator, filters, baseRelation, mediaScope)
if err != nil {
return fmt.Errorf("listing catalog narrators: %w", err)
}
narrators = out
return nil
}))
}
eg.Go(withLimit(func() error {
out, err := facets.AudiobookSeries(gctx, filters, baseRelation, mediaScope)
if err != nil {
@@ -1090,8 +1093,9 @@ func validateCatalogOverlayQuery(searchQuery string, def QueryDefinition, ruleFi
def.MediaScope != "movie" &&
def.MediaScope != "series" &&
def.MediaScope != "episode" &&
def.MediaScope != "audiobook" {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', 'episode', or 'audiobook'", ErrInvalidCatalogRequest)
def.MediaScope != "audiobook" &&
def.MediaScope != "ebook" {
return fmt.Errorf("%w: media_scope must be 'movie', 'series', 'episode', 'audiobook', or 'ebook'", ErrInvalidCatalogRequest)
}
if def.Match != "" && def.Match != "all" && def.Match != "any" {
return fmt.Errorf("%w: match must be 'all' or 'any'", ErrInvalidCatalogRequest)
+165
View File
@@ -39,6 +39,21 @@ func TestValidateCatalogQueryRequest_AllowsEpisodeMediaScope(t *testing.T) {
}
}
func TestValidateCatalogQueryRequest_AllowsEbookMediaScope(t *testing.T) {
req := CatalogRequest{
Source: CatalogSourceQuery,
Query: QueryDefinition{
MediaScope: "ebook",
Match: "all",
Sort: QuerySort{Field: "title", Order: "asc"},
},
}
if err := validateCatalogQueryRequest(req, true); err != nil {
t.Fatalf("expected ebook media scope to be accepted, got %v", err)
}
}
func TestValidateCatalogQueryRequest_AllowsAddedAtFilter(t *testing.T) {
req := CatalogRequest{
Source: CatalogSourceQuery,
@@ -244,6 +259,67 @@ func (s *stubFacetFetcher) SearchAudiobookSeries(ctx context.Context, filters Br
return nil, false, nil
}
type recordingFacetFetcher struct {
mu sync.Mutex
peopleKinds []models.PersonKind
searchPeopleKind models.PersonKind
searchMediaScope string
}
func (s *recordingFacetFetcher) DistinctArrayColumn(ctx context.Context, column string, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) DistinctScalarColumn(ctx context.Context, column string, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) Resolutions(ctx context.Context, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) JSONBLanguages(ctx context.Context, column string, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) SubtitleLanguages(ctx context.Context, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) PeopleByKind(ctx context.Context, kind models.PersonKind, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
s.mu.Lock()
s.peopleKinds = append(s.peopleKinds, kind)
s.mu.Unlock()
if kind == models.PersonKindAuthor {
return []string{"Author"}, nil
}
return []string{"Narrator"}, nil
}
func (s *recordingFacetFetcher) AudiobookSeries(ctx context.Context, filters BrowseFilters, baseRelation string, mediaScope string) ([]string, error) {
return nil, nil
}
func (s *recordingFacetFetcher) SearchDistinctArrayColumn(ctx context.Context, column string, filters BrowseFilters, baseRelation string, mediaScope string, prefix string, limit int) ([]string, bool, error) {
return nil, false, nil
}
func (s *recordingFacetFetcher) SearchDistinctScalarColumn(ctx context.Context, column string, filters BrowseFilters, baseRelation string, mediaScope string, prefix string, limit int) ([]string, bool, error) {
return nil, false, nil
}
func (s *recordingFacetFetcher) SearchPeopleByKind(ctx context.Context, kind models.PersonKind, filters BrowseFilters, baseRelation string, mediaScope string, prefix string, limit int) ([]string, bool, error) {
s.mu.Lock()
s.searchPeopleKind = kind
s.searchMediaScope = mediaScope
s.mu.Unlock()
return []string{"Author"}, false, nil
}
func (s *recordingFacetFetcher) SearchAudiobookSeries(ctx context.Context, filters BrowseFilters, baseRelation string, mediaScope string, prefix string, limit int) ([]string, bool, error) {
return nil, false, nil
}
// countingExecutor is a previewExecutor stub that records the number of
// PreviewPage calls and the AccessFilter passed in (so tests can confirm
// NamePrefix made it through). It returns an empty result set so the resolver
@@ -327,3 +403,92 @@ func TestListFiltersWithOptions_RunsFacetQueriesConcurrently(t *testing.T) {
barrier.assertConcurrent(t)
}
func TestListFiltersWithOptions_EbookScopeSkipsNarratorFacet(t *testing.T) {
facets := &recordingFacetFetcher{}
resolver := &CatalogResolver{
browseRepo: &BrowseRepository{},
facets: facets,
}
result, err := resolver.ListFiltersWithOptions(
context.Background(),
CatalogRequest{
Source: CatalogSourceQuery,
Query: QueryDefinition{MediaScope: "ebook"},
},
AccessFilter{},
CatalogFilterOptions{},
)
if err != nil {
t.Fatalf("ListFiltersWithOptions returned error: %v", err)
}
facets.mu.Lock()
defer facets.mu.Unlock()
for _, kind := range facets.peopleKinds {
if kind == models.PersonKindNarrator {
t.Fatal("ebook filters should not request narrator facets")
}
}
if len(result.Narrators) != 0 {
t.Fatalf("ebook filters returned narrators = %v, want empty", result.Narrators)
}
}
func TestSearchFacet_EbookScopeForwardsMediaScope(t *testing.T) {
facets := &recordingFacetFetcher{}
resolver := &CatalogResolver{
browseRepo: &BrowseRepository{},
facets: facets,
}
result, err := resolver.SearchFacet(
context.Background(),
CatalogRequest{
Source: CatalogSourceQuery,
Query: QueryDefinition{MediaScope: "ebook"},
},
AccessFilter{},
"author",
"Au",
20,
)
if err != nil {
t.Fatalf("SearchFacet returned error: %v", err)
}
if len(result.Matches) != 1 || result.Matches[0] != "Author" {
t.Fatalf("matches = %v, want Author", result.Matches)
}
facets.mu.Lock()
defer facets.mu.Unlock()
if facets.searchPeopleKind != models.PersonKindAuthor {
t.Fatalf("searchPeopleKind = %v, want author", facets.searchPeopleKind)
}
if facets.searchMediaScope != "ebook" {
t.Fatalf("search mediaScope = %q, want ebook", facets.searchMediaScope)
}
}
func TestSearchFacet_EbookScopeRejectsNarratorFacet(t *testing.T) {
resolver := &CatalogResolver{
browseRepo: &BrowseRepository{},
facets: &recordingFacetFetcher{},
}
_, err := resolver.SearchFacet(
context.Background(),
CatalogRequest{
Source: CatalogSourceQuery,
Query: QueryDefinition{MediaScope: "ebook"},
},
AccessFilter{},
"narrator",
"Na",
20,
)
if err == nil {
t.Fatal("expected narrator facet to be rejected for ebook scope")
}
}
+74 -12
View File
@@ -157,6 +157,9 @@ type ItemDetail struct {
// Audiobook-specific detail. Present only when Type == "audiobook".
Audiobook *AudiobookDetailExtension `json:"audiobook,omitempty"`
// Ebook-specific detail. Present only when Type == "ebook".
Ebook *EbookDetailExtension `json:"ebook,omitempty"`
}
type AudiobookDetailExtension struct {
@@ -201,6 +204,13 @@ type AudiobookNarration struct {
Narrators []string `json:"narrators"`
}
type EbookDetailExtension struct {
Authors []AudiobookPerson `json:"authors"`
Publisher string `json:"publisher,omitempty"`
Series *AudiobookSeriesGroup `json:"series,omitempty"`
Related AudiobookRelatedContent `json:"related"`
}
// ItemUserState is per-profile viewer state included in item detail responses.
type ItemUserState struct {
Played bool `json:"played"`
@@ -895,6 +905,9 @@ func (s *DetailService) buildMediaItemDetail(ctx context.Context, item *models.M
if item.Type == "audiobook" {
detail.Audiobook = s.buildAudiobookExtension(ctx, item, detail.Versions, crewCredits, filter)
}
if item.Type == "ebook" {
detail.Ebook = s.buildEbookExtension(ctx, item, crewCredits, filter)
}
// Series folder paths from confirmed claims when available, otherwise from
// the file links that currently belong to the item. This keeps provisional
@@ -1036,6 +1049,26 @@ func (s *DetailService) buildAudiobookExtension(
}
}
func (s *DetailService) buildEbookExtension(
ctx context.Context,
item *models.MediaItem,
crew []CrewCredit,
filter AccessFilter,
) *EbookDetailExtension {
if item == nil {
return nil
}
return &EbookDetailExtension{
Authors: audiobookPeopleFromCrew(crew, models.PersonKindAuthor.String()),
Publisher: firstNonEmptyString(item.Studios),
Series: s.fetchEbookSeries(ctx, item.ContentID, filter),
Related: AudiobookRelatedContent{
AlsoByAuthor: s.fetchEbookAlsoByAuthor(ctx, item.ContentID, filter),
Similar: s.fetchEbookSimilarByGenres(ctx, item.ContentID, filter),
},
}
}
func audiobookPeopleFromCrew(crew []CrewCredit, job string) []AudiobookPerson {
out := make([]AudiobookPerson, 0)
for _, credit := range crew {
@@ -1103,15 +1136,23 @@ func appendAudiobookItemAccessConditions(
}
func (s *DetailService) fetchAudiobookAlsoByAuthor(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
return s.fetchBookAlsoByAuthor(ctx, contentID, "audiobook", filter)
}
func (s *DetailService) fetchEbookAlsoByAuthor(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
return s.fetchBookAlsoByAuthor(ctx, contentID, "ebook", filter)
}
func (s *DetailService) fetchBookAlsoByAuthor(ctx context.Context, contentID string, mediaType string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
args := []any{contentID, models.PersonKindAuthor, mediaType}
argIdx := 4
conditions := []string{
"ip1.content_id = $1",
"ip1.kind = $2",
"m2.type = 'audiobook'",
"m2.type = $3",
}
if !appendAudiobookItemAccessConditions("m2", filter, &conditions, &args, &argIdx) {
return []AudiobookRelatedItem{}
@@ -1153,13 +1194,21 @@ func (s *DetailService) fetchAudiobookAlsoByAuthor(ctx context.Context, contentI
}
func (s *DetailService) fetchAudiobookSimilarByGenres(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
return s.fetchBookSimilarByGenres(ctx, contentID, "audiobook", filter)
}
func (s *DetailService) fetchEbookSimilarByGenres(ctx context.Context, contentID string, filter AccessFilter) []AudiobookRelatedItem {
return s.fetchBookSimilarByGenres(ctx, contentID, "ebook", filter)
}
func (s *DetailService) fetchBookSimilarByGenres(ctx context.Context, contentID string, mediaType string, filter AccessFilter) []AudiobookRelatedItem {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return []AudiobookRelatedItem{}
}
args := []any{contentID, models.PersonKindAuthor}
argIdx := 3
args := []any{contentID, models.PersonKindAuthor, mediaType}
argIdx := 4
conditions := []string{
"m.type = 'audiobook'",
"m.type = $3",
"m.content_id <> $1",
"m.genres && (SELECT array_agg(g) FROM this_genres)",
`NOT EXISTS (
@@ -1212,14 +1261,27 @@ func (s *DetailService) fetchAudiobookSimilarByGenres(ctx context.Context, conte
}
func (s *DetailService) fetchAudiobookSeries(ctx context.Context, contentID string, filter AccessFilter) *AudiobookSeriesGroup {
return s.fetchBookSeries(ctx, contentID, "audiobook", "audiobook_series", filter)
}
func (s *DetailService) fetchEbookSeries(ctx context.Context, contentID string, filter AccessFilter) *AudiobookSeriesGroup {
return s.fetchBookSeries(ctx, contentID, "ebook", "ebook_series", filter)
}
func (s *DetailService) fetchBookSeries(ctx context.Context, contentID string, mediaType string, tableName string, filter AccessFilter) *AudiobookSeriesGroup {
if s == nil || s.itemRepo == nil || s.itemRepo.pool == nil {
return nil
}
args := []any{contentID}
argIdx := 2
switch tableName {
case "audiobook_series", "ebook_series":
default:
return nil
}
args := []any{contentID, mediaType}
argIdx := 3
conditions := []string{
"root.content_id = $1",
"m.type = 'audiobook'",
"m.type = $2",
}
if !appendAudiobookItemAccessConditions("m", filter, &conditions, &args, &argIdx) {
return nil
@@ -1232,13 +1294,13 @@ func (s *DetailService) fetchAudiobookSeries(ctx context.Context, contentID stri
COALESCE(m.poster_path, ''),
s.series_name,
s.series_index
FROM audiobook_series root
JOIN audiobook_series s ON LOWER(s.series_name) = LOWER(root.series_name)
FROM %s root
JOIN %s s ON LOWER(s.series_name) = LOWER(root.series_name)
JOIN media_items m ON m.content_id = s.content_id
WHERE %s
ORDER BY s.series_index NULLS LAST, LOWER(m.sort_title)
LIMIT 30
`, strings.Join(conditions, " AND "))
`, tableName, tableName, strings.Join(conditions, " AND "))
rows, err := s.itemRepo.pool.Query(ctx, query, args...)
if err != nil {
return nil
+24
View File
@@ -40,3 +40,27 @@ func TestSortAudiobookMediaFilesFallsBackToPath(t *testing.T) {
}
}
}
func TestBuildEbookExtensionUsesAuthorsAndPublisherWithoutNarrators(t *testing.T) {
crew := []CrewCredit{
{Name: "Becky Chambers", Job: models.PersonKindAuthor.String(), PersonID: "10"},
{Name: "Narrator Should Not Apply", Job: models.PersonKindNarrator.String(), PersonID: "20"},
}
item := &models.MediaItem{
ContentID: "ebook-1",
Type: "ebook",
Studios: []string{"Tor"},
}
extension := (&DetailService{}).buildEbookExtension(nil, item, crew, AccessFilter{})
if extension == nil {
t.Fatal("buildEbookExtension returned nil")
}
if len(extension.Authors) != 1 || extension.Authors[0].Name != "Becky Chambers" {
t.Fatalf("Authors = %+v, want Becky Chambers", extension.Authors)
}
if extension.Publisher != "Tor" {
t.Fatalf("Publisher = %q, want Tor", extension.Publisher)
}
}
+64 -28
View File
@@ -36,10 +36,10 @@ type RatingFilter struct {
Filter AccessFilter
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
// buildRatingThresholdQuery builds the SQL statement and bind args for ListByRatingThreshold.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
@@ -60,7 +60,7 @@ func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f Ratin
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
return "", nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
@@ -85,7 +85,7 @@ func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f Ratin
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
qualifiedItemColumns("mi"),
fromClause,
strings.Join(conditions, " AND "),
)
@@ -95,6 +95,18 @@ func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f Ratin
args = append(args, f.Limit)
}
return query, args
}
// ListByRatingThreshold returns media items whose rating_imdb is >= f.Min,
// ordered by rating_imdb DESC NULLS LAST. Items without an IMDb rating are
// always excluded.
func (r *DiscoveryRepository) ListByRatingThreshold(ctx context.Context, f RatingFilter) ([]*models.MediaItem, error) {
query, args := buildRatingThresholdQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing items by rating threshold: %w", err)
@@ -122,17 +134,10 @@ type UnplayedFilter struct {
Filter AccessFilter
}
// ListUnplayedHighRated returns high-rated items that the given user/profile has
// never started watching. "Never started" means no row exists in
// user_watch_history for (user_id, profile_id, media_item_id), regardless of
// completion status. Items without an IMDb rating are excluded.
//
// Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f UnplayedFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListUnplayedHighRated: UserID and ProfileID are required")
}
// buildUnplayedHighRatedQuery builds the SQL statement and bind args for ListUnplayedHighRated.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
@@ -161,7 +166,7 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
return "", nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
@@ -186,7 +191,7 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
qualifiedItemColumns("mi"),
fromClause,
strings.Join(conditions, " AND "),
)
@@ -196,6 +201,25 @@ func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f Unpla
args = append(args, f.Limit)
}
return query, args
}
// ListUnplayedHighRated returns high-rated items that the given user/profile has
// never started watching. "Never started" means no row exists in
// user_watch_history for (user_id, profile_id, media_item_id), regardless of
// completion status. Items without an IMDb rating are excluded.
//
// Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListUnplayedHighRated(ctx context.Context, f UnplayedFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListUnplayedHighRated: UserID and ProfileID are required")
}
query, args := buildUnplayedHighRatedQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing unplayed high-rated items: %w", err)
@@ -225,13 +249,10 @@ type ForgottenFavoritesFilter struct {
Filter AccessFilter
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
// buildForgottenFavoritesQuery builds the SQL statement and bind args for ListForgottenFavorites.
// It returns an empty query string when access rules exclude every
// library, signalling the caller to skip the query and return no rows.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
@@ -261,7 +282,7 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return []*models.MediaItem{}, nil
return "", nil
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
@@ -286,7 +307,7 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
query := fmt.Sprintf(
"SELECT %s FROM %s WHERE %s ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC",
itemColumns,
qualifiedItemColumns("mi"),
fromClause,
strings.Join(conditions, " AND "),
)
@@ -296,6 +317,21 @@ func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f Forg
args = append(args, f.Limit)
}
return query, args
}
// ListForgottenFavorites returns high-rated items (rating_imdb >= 7.0) that the
// user/profile either has never watched OR last watched more than LookbackDays
// ago. Results are ordered by rating_imdb DESC NULLS LAST.
func (r *DiscoveryRepository) ListForgottenFavorites(ctx context.Context, f ForgottenFavoritesFilter) ([]*models.MediaItem, error) {
if f.UserID <= 0 || strings.TrimSpace(f.ProfileID) == "" {
return nil, fmt.Errorf("ListForgottenFavorites: UserID and ProfileID are required")
}
query, args := buildForgottenFavoritesQuery(f)
if query == "" {
return []*models.MediaItem{}, nil
}
rows, err := r.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("listing forgotten favorites: %w", err)
+38 -180
View File
@@ -13,130 +13,6 @@ import (
"testing"
)
// buildRatingThresholdQuery is the testable extraction of the SQL generation
// logic from ListByRatingThreshold so we can inspect it without a pool.
func buildRatingThresholdQuery(f RatingFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.Min)
argIdx++
fromClause := "media_items mi"
if f.LibraryID != nil {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
conditions = append(conditions, "mil.media_folder_id = $2")
args = append(args, *f.LibraryID)
argIdx++
} else if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// buildUnplayedHighRatedQuery mirrors the SQL-building portion of
// ListUnplayedHighRated for unit testing.
func buildUnplayedHighRatedQuery(f UnplayedFilter) (string, []any) {
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= $1")
args = append(args, f.MinRating)
argIdx++
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t)")
args = append(args, f.UserID, f.ProfileID)
argIdx += 2
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
// itoa is a local helper to avoid importing strconv.
func itoa(n int) string {
if n == 0 {
return "0"
}
buf := [20]byte{}
pos := len(buf)
for n > 0 {
pos--
buf[pos] = byte('0' + n%10)
n /= 10
}
return string(buf[pos:])
}
// ---------------------------------------------------------------------------
// ListByRatingThreshold SQL generation tests
// ---------------------------------------------------------------------------
@@ -370,62 +246,6 @@ func TestUnplayedHighRated_ContentRatingFilter(t *testing.T) {
// ListForgottenFavorites SQL generation tests
// ---------------------------------------------------------------------------
// buildForgottenFavoritesQuery is the testable extraction of the SQL
// generation logic from ListForgottenFavorites.
func buildForgottenFavoritesQuery(f ForgottenFavoritesFilter) (string, []any) {
if f.LookbackDays <= 0 {
f.LookbackDays = 365
}
var conditions []string
var args []any
argIdx := 1
conditions = append(conditions, "mi.rating_imdb >= 7.0")
conditions = append(conditions, "NOT EXISTS (\n\t\tSELECT 1\n\t\tFROM user_watch_history uwh\n\t\tWHERE uwh.user_id = $"+itoa(argIdx)+"\n\t\t AND uwh.profile_id = $"+itoa(argIdx+1)+"\n\t\t AND uwh.media_item_id = mi.content_id\n\t\t AND uwh.watched_at >= NOW() - ($"+itoa(argIdx+2)+" || ' days')::interval\n\t)")
args = append(args, f.UserID, f.ProfileID, f.LookbackDays)
argIdx += 3
fromClause := "media_items mi"
if f.Filter.AllowedLibraryIDs != nil || len(f.Filter.DisabledLibraryIDs) > 0 {
fromClause = "media_items mi JOIN media_item_libraries mil ON mi.content_id = mil.content_id"
if f.Filter.AllowedLibraryIDs != nil {
if len(f.Filter.AllowedLibraryIDs) == 0 {
return "", nil // early empty
}
placeholders := make([]string, len(f.Filter.AllowedLibraryIDs))
for i, id := range f.Filter.AllowedLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id IN ("+strings.Join(placeholders, ", ")+")")
}
if len(f.Filter.DisabledLibraryIDs) > 0 {
placeholders := make([]string, len(f.Filter.DisabledLibraryIDs))
for i, id := range f.Filter.DisabledLibraryIDs {
placeholders[i] = "$" + itoa(argIdx)
args = append(args, id)
argIdx++
}
conditions = append(conditions, "mil.media_folder_id NOT IN ("+strings.Join(placeholders, ", ")+")")
}
}
applyAccessFilter("mi", f.Filter, &conditions, &args, &argIdx)
query := "SELECT " + itemColumns + " FROM " + fromClause +
" WHERE " + strings.Join(conditions, " AND ") +
" ORDER BY mi.rating_imdb DESC NULLS LAST, mi.content_id ASC"
if f.Limit > 0 {
query += " LIMIT $" + itoa(argIdx)
args = append(args, f.Limit)
}
return query, args
}
func TestForgottenFavorites_BasicQuery(t *testing.T) {
query, args := buildForgottenFavoritesQuery(ForgottenFavoritesFilter{
LookbackDays: 365,
@@ -540,3 +360,41 @@ func TestForgottenFavorites_DefaultLookbackApplied(t *testing.T) {
t.Fatalf("expected lookback default 365, got %v", args[2])
}
}
// TestDiscoveryQueries_QualifyContentIDUnderLibraryJoin is a regression guard
// for the "column reference \"content_id\" is ambiguous" (SQLSTATE 42702)
// failure. When a viewer has library restrictions, each discovery query joins
// media_item_libraries (which also has a content_id column). The SELECT list
// must therefore qualify its columns with the mi alias; a bare column list
// makes content_id ambiguous and the query fails at runtime.
func TestDiscoveryQueries_QualifyContentIDUnderLibraryJoin(t *testing.T) {
restricted := AccessFilter{AllowedLibraryIDs: []int{1, 2}}
cases := []struct {
name string
query string
}{
{"rating_threshold", mustQuery(buildRatingThresholdQuery(RatingFilter{Min: 7, Filter: restricted}))},
{"unplayed_high_rated", mustQuery(buildUnplayedHighRatedQuery(UnplayedFilter{MinRating: 7, UserID: 1, ProfileID: "p", Filter: restricted}))},
{"forgotten_favorites", mustQuery(buildForgottenFavoritesQuery(ForgottenFavoritesFilter{UserID: 1, ProfileID: "p", LookbackDays: 365, Filter: restricted}))},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
// Precondition: the join that introduces the ambiguity is present.
if !strings.Contains(tc.query, "JOIN media_item_libraries mil") {
t.Fatalf("expected library join in query, got:\n%s", tc.query)
}
// The SELECT list must be alias-qualified.
if !strings.HasPrefix(tc.query, "SELECT mi.content_id,") {
t.Fatalf("SELECT list is not qualified with the mi alias (ambiguous content_id), got:\n%s", tc.query)
}
// And must not contain a bare, unqualified content_id column.
if strings.Contains(tc.query, "SELECT content_id,") {
t.Fatalf("SELECT list uses an unqualified content_id, got:\n%s", tc.query)
}
})
}
}
func mustQuery(query string, _ []any) string { return query }
+2 -2
View File
@@ -716,12 +716,12 @@ const collectOrphanedProvisionalIDsByContentIDSQL = `
SELECT mi.content_id
FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedMediaItemSafetyConditions
AND ` + orphanedProvisionalMediaItemConditions
const deleteOrphanedProvisionalIDsByContentIDSQL = `
DELETE FROM public.media_items mi
WHERE mi.content_id = ANY($1)
AND ` + orphanedMediaItemSafetyConditions
AND ` + orphanedProvisionalMediaItemConditions
func (r *FolderRepository) deleteOrphanedItemsByContentID(ctx context.Context, contentIDs []string) (int, []string, error) {
if len(contentIDs) == 0 {
+130 -9
View File
@@ -159,9 +159,10 @@ func (qb *QueryBuilder) UserHistoryCTEArgs() []any {
// UserHistoryCTESQL returns the user_last_watched CTE definition. The two
// placeholder positions (user_id, profile_id) are bound to the args returned
// by UserHistoryCTEArgs and should be the first two entries in the final
// statement's arg list. The CTE aggregates user_watch_history watch_at and
// user_watch_progress.completed=TRUE updated_at into one row per
// media_item_id, filtering out items the user has hidden via
// statement's arg list. The CTE aggregates user_watch_history watched_at,
// user_watch_progress.completed=TRUE updated_at, and completed ebook reader
// progress updated_at into one row per media_item_id, filtering out videos the
// user has hidden via
// user_history_hidden_items. The single GROUP BY replaces the two
// correlated MAX subqueries the previous lastWatchedExpr emitted (audit
// 2026-05-01 §3.1 Pattern B).
@@ -187,6 +188,10 @@ func UserHistoryCTESQL(argIdx int) string {
SELECT uwp.media_item_id, NULL::timestamptz, uwp.updated_at
FROM user_watch_progress uwp
WHERE uwp.user_id = $%d AND uwp.profile_id = $%d AND uwp.completed = TRUE
UNION ALL
SELECT erp.content_id AS media_item_id, NULL::timestamptz, erp.updated_at
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d AND erp.profile_id = $%d AND erp.progress >= 0.9
) src
WHERE NOT EXISTS (
SELECT 1 FROM user_history_hidden_items hhi
@@ -195,7 +200,7 @@ func UserHistoryCTESQL(argIdx int) string {
AND COALESCE(src.uwh_at, src.uwp_at) <= hhi.hidden_before
)
GROUP BY src.media_item_id
)`, argIdx, argIdx+1, argIdx, argIdx+1, argIdx, argIdx+1)
)`, argIdx, argIdx+1, argIdx, argIdx+1, argIdx, argIdx+1, argIdx, argIdx+1)
}
func (qb *QueryBuilder) BuildSortClause(sortConfig QuerySort) (string, []any, error) {
@@ -298,7 +303,8 @@ func (qb *QueryBuilder) BuildSortPlan(sortConfig QuerySort) (QuerySortPlan, erro
return plan, nil
case "series":
plan.Joins = []string{fmt.Sprintf(
"LEFT JOIN audiobook_series sort_series ON sort_series.content_id = %s.content_id",
"LEFT JOIN %s sort_series ON sort_series.content_id = %s.content_id",
qb.bookSeriesTable(),
qb.alias,
)}
// Sort by series name primarily, then by series_index so books
@@ -385,7 +391,7 @@ func (qb *QueryBuilder) buildRule(rule QueryRule) (string, error) {
case "narrator":
return qb.buildPersonClause(models.PersonKindNarrator, rule)
case "series":
return qb.buildAudiobookSeriesClause(rule)
return qb.buildBookSeriesClause(rule)
case "watched":
return qb.buildWatchedClause(rule)
case "favorited":
@@ -502,14 +508,21 @@ func (qb *QueryBuilder) personKindSortJoin(kind models.PersonKind, alias string)
)
}
func (qb *QueryBuilder) buildAudiobookSeriesClause(rule QueryRule) (string, error) {
func (qb *QueryBuilder) bookSeriesTable() string {
if qb.mediaScope == "ebook" {
return "ebook_series"
}
return "audiobook_series"
}
func (qb *QueryBuilder) buildBookSeriesClause(rule QueryRule) (string, error) {
qb.args = append(qb.args, rule.Value)
clause := fmt.Sprintf(`EXISTS (
SELECT 1
FROM audiobook_series s
FROM %s s
WHERE s.content_id = %s.content_id
AND LOWER(BTRIM(s.series_name)) = LOWER(BTRIM($%d))
)`, qb.alias, qb.argIdx)
)`, qb.bookSeriesTable(), qb.alias, qb.argIdx)
qb.argIdx++
if rule.Op == "is_not" {
return "NOT (" + clause + ")", nil
@@ -947,6 +960,9 @@ func (qb *QueryBuilder) buildInProgressClause(rule QueryRule) (string, error) {
if !ok {
return "", fmt.Errorf("in_progress requires a boolean value")
}
if qb.mediaScope == "ebook" {
return qb.buildEbookInProgressClause(value), nil
}
qb.args = append(qb.args, qb.userID, qb.profileID)
clause := fmt.Sprintf(`EXISTS (
SELECT 1
@@ -972,6 +988,24 @@ func (qb *QueryBuilder) buildInProgressClause(rule QueryRule) (string, error) {
return clause, nil
}
func (qb *QueryBuilder) buildEbookInProgressClause(value bool) string {
qb.args = append(qb.args, qb.userID, qb.profileID)
clause := fmt.Sprintf(`EXISTS (
SELECT 1
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d
AND erp.profile_id = $%d
AND erp.content_id = %s.content_id
AND erp.progress > 0
AND erp.progress < 0.9
)`, qb.argIdx, qb.argIdx+1, qb.alias)
qb.argIdx += 2
if !value {
return "NOT (" + clause + ")"
}
return clause
}
func (qb *QueryBuilder) buildLastWatchedClause(rule QueryRule) (string, error) {
if err := qb.ensureUserScope(rule.Field); err != nil {
return "", err
@@ -1070,6 +1104,9 @@ func (qb *QueryBuilder) buildReleaseDateClause(rule QueryRule) (string, error) {
}
func (qb *QueryBuilder) userStateCompletionClause() string {
if qb.mediaScope == "ebook" {
return qb.ebookUserStateCompletionClause()
}
qb.args = append(qb.args, qb.userID, qb.profileID)
clause := fmt.Sprintf(`(
EXISTS (
@@ -1120,6 +1157,20 @@ func (qb *QueryBuilder) userStateCompletionClause() string {
return clause
}
func (qb *QueryBuilder) ebookUserStateCompletionClause() string {
qb.args = append(qb.args, qb.userID, qb.profileID)
clause := fmt.Sprintf(`EXISTS (
SELECT 1
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d
AND erp.profile_id = $%d
AND erp.content_id = %s.content_id
AND erp.progress >= 0.9
)`, qb.argIdx, qb.argIdx+1, qb.alias)
qb.argIdx += 2
return clause
}
// lastWatchedExpr returns the SQL expression the executor binds to the
// per-user history aggregate. It signals the executor (via
// requireUserHistoryCTE) that the user_last_watched CTE and a LEFT JOIN
@@ -1343,6 +1394,9 @@ func (qb *QueryBuilder) consumeIntArgs(values []int) ([]string, []any) {
}
func (qb *QueryBuilder) progressSortPlan() (string, []string, []any) {
if qb.mediaScope == "ebook" {
return qb.ebookProgressSortPlan()
}
joinSQL := fmt.Sprintf(
`LEFT JOIN (
SELECT uwp.media_item_id AS content_id,
@@ -1371,7 +1425,31 @@ func (qb *QueryBuilder) progressSortPlan() (string, []string, []any) {
return "sort_progress.progress_ratio", []string{joinSQL}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) ebookProgressSortPlan() (string, []string, []any) {
joinSQL := fmt.Sprintf(
`LEFT JOIN (
SELECT erp.content_id,
CASE
WHEN erp.progress > 0
AND erp.progress < 0.9
THEN erp.progress::double precision
ELSE NULL
END AS progress_ratio
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d
AND erp.profile_id = $%d
) sort_progress ON sort_progress.content_id = %s.content_id`,
qb.argIdx,
qb.argIdx+1,
qb.alias,
)
return "sort_progress.progress_ratio", []string{joinSQL}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) dateViewedSortPlan() (string, []string, []any) {
if qb.mediaScope == "ebook" {
return qb.ebookDateViewedSortPlan()
}
historyJoin := fmt.Sprintf(
`LEFT JOIN (
SELECT uwh.media_item_id AS content_id, MAX(uwh.watched_at) AS viewed_at
@@ -1422,7 +1500,30 @@ func (qb *QueryBuilder) dateViewedSortPlan() (string, []string, []any) {
return expr, []string{historyJoin, progressJoin}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) ebookDateViewedSortPlan() (string, []string, []any) {
joinSQL := fmt.Sprintf(
`LEFT JOIN (
SELECT erp.content_id,
CASE
WHEN erp.progress >= 0.9
THEN erp.updated_at
ELSE NULL
END AS viewed_at
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d
AND erp.profile_id = $%d
) sort_ebook_viewed ON sort_ebook_viewed.content_id = %s.content_id`,
qb.argIdx,
qb.argIdx+1,
qb.alias,
)
return "sort_ebook_viewed.viewed_at", []string{joinSQL}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) playsSortPlan() (string, []string, []any) {
if qb.mediaScope == "ebook" {
return qb.ebookPlaysSortPlan()
}
historyJoin := fmt.Sprintf(
`LEFT JOIN (
SELECT uwh.media_item_id AS content_id, COUNT(*) AS play_count
@@ -1473,6 +1574,26 @@ func (qb *QueryBuilder) playsSortPlan() (string, []string, []any) {
return expr, []string{historyJoin, progressJoin}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) ebookPlaysSortPlan() (string, []string, []any) {
joinSQL := fmt.Sprintf(
`LEFT JOIN (
SELECT erp.content_id,
CASE
WHEN erp.progress >= 0.9
THEN 1
ELSE 0
END AS completed_play_count
FROM ebook_reader_progress erp
WHERE erp.user_id = $%d
AND erp.profile_id = $%d
) sort_ebook_plays ON sort_ebook_plays.content_id = %s.content_id`,
qb.argIdx,
qb.argIdx+1,
qb.alias,
)
return "NULLIF(sort_ebook_plays.completed_play_count, 0)", []string{joinSQL}, []any{qb.userID, qb.profileID}
}
func (qb *QueryBuilder) lastAirDateSortPlan() (string, []string) {
expr := fmt.Sprintf("%s.last_air_date_at", qb.alias)
return expr, nil // denormalized column — no JOIN required
+173
View File
@@ -390,6 +390,132 @@ func TestBuildSortClause_PersonalizedComputedSorts(t *testing.T) {
}
}
func TestBuildSortClause_EbookPersonalizedComputedSortsUseReaderProgress(t *testing.T) {
builder := NewQueryBuilder("mi").WithMediaScope("ebook").WithUserScope(42, "profile-1")
tests := []struct {
field string
expectJoin []string
expectExpr []string
}{
{
field: "progress",
expectJoin: []string{"FROM ebook_reader_progress erp", "erp.progress > 0", "erp.progress < 0.9"},
expectExpr: []string{"sort_progress.progress_ratio"},
},
{
field: "date_viewed",
expectJoin: []string{"FROM ebook_reader_progress erp", "erp.progress >= 0.9"},
expectExpr: []string{"sort_ebook_viewed.viewed_at"},
},
{
field: "plays",
expectJoin: []string{"FROM ebook_reader_progress erp", "erp.progress >= 0.9", "completed_play_count"},
expectExpr: []string{"sort_ebook_plays.completed_play_count"},
},
}
for _, tt := range tests {
plan, err := builder.BuildSortPlan(QuerySort{Field: tt.field, Order: "desc"})
if err != nil {
t.Fatalf("%s: BuildSortPlan returned error: %v", tt.field, err)
}
if len(plan.Args) != 2 || plan.Args[0] != 42 || plan.Args[1] != "profile-1" {
t.Fatalf("%s: expected user scope args, got %v", tt.field, plan.Args)
}
joined := strings.Join(plan.Joins, "\n")
for _, fragment := range tt.expectJoin {
if !strings.Contains(joined, fragment) {
t.Fatalf("%s: expected joins to contain %q, got %q", tt.field, fragment, joined)
}
}
for _, fragment := range tt.expectExpr {
if !strings.Contains(plan.OrderBy, fragment) {
t.Fatalf("%s: expected order by to contain %q, got %q", tt.field, fragment, plan.OrderBy)
}
}
}
}
func TestBuild_EbookUserStateRulesUseReaderProgress(t *testing.T) {
tests := []struct {
name string
rule QueryRule
expectSQL []string
unexpectSQL []string
}{
{
name: "watched",
rule: QueryRule{Field: "watched", Op: "is", Value: true},
expectSQL: []string{
"FROM ebook_reader_progress erp",
"erp.progress >= 0.9",
},
unexpectSQL: []string{"FROM user_watch_progress uwp", "FROM user_watch_history uwh"},
},
{
name: "unread",
rule: QueryRule{Field: "watched", Op: "is", Value: false},
expectSQL: []string{
"NOT (EXISTS",
"FROM ebook_reader_progress erp",
"erp.progress >= 0.9",
},
unexpectSQL: []string{"FROM user_watch_progress uwp", "FROM user_watch_history uwh"},
},
{
name: "in progress",
rule: QueryRule{Field: "in_progress", Op: "is", Value: true},
expectSQL: []string{
"FROM ebook_reader_progress erp",
"erp.progress > 0",
"erp.progress < 0.9",
},
unexpectSQL: []string{"FROM user_watch_progress uwp"},
},
{
name: "not in progress",
rule: QueryRule{Field: "in_progress", Op: "is", Value: false},
expectSQL: []string{
"NOT (EXISTS",
"FROM ebook_reader_progress erp",
"erp.progress > 0",
"erp.progress < 0.9",
},
unexpectSQL: []string{"FROM user_watch_progress uwp"},
},
}
for _, tt := range tests {
clause, args, err := NewQueryBuilder("mi").
WithMediaScope("ebook").
WithUserScope(42, "profile-1").
Build(QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []QueryGroup{{
Match: "all",
Rules: []QueryRule{tt.rule},
}},
})
if err != nil {
t.Fatalf("%s: Build returned error: %v", tt.name, err)
}
if len(args) != 2 || args[0] != 42 || args[1] != "profile-1" {
t.Fatalf("%s: expected user scope args, got %v", tt.name, args)
}
for _, fragment := range tt.expectSQL {
if !strings.Contains(clause, fragment) {
t.Fatalf("%s: expected clause to contain %q, got %q", tt.name, fragment, clause)
}
}
for _, fragment := range tt.unexpectSQL {
if strings.Contains(clause, fragment) {
t.Fatalf("%s: did not expect clause to contain %q, got %q", tt.name, fragment, clause)
}
}
}
}
func TestBuild_ResolutionNormalizes4KAndScopesMediaFiles(t *testing.T) {
clause, args, err := NewQueryBuilder("mi").
WithLibraryScope([]int{3}).
@@ -531,6 +657,26 @@ func TestBuild_SeriesClauseJoinsAudiobookSeries(t *testing.T) {
}
}
func TestBuild_SeriesClauseJoinsEbookSeriesForEbookScope(t *testing.T) {
clause, args, err := NewQueryBuilder("mi").
WithMediaScope("ebook").
Build(QueryDefinition{
Match: "all",
Groups: []QueryGroup{
{Match: "all", Rules: []QueryRule{{Field: "series", Op: "is", Value: "Wayfarers"}}},
},
})
if err != nil {
t.Fatalf("Build returned error: %v", err)
}
if len(args) != 1 || args[0] != "Wayfarers" {
t.Fatalf("expected series arg, got %v", args)
}
if !strings.Contains(clause, "FROM ebook_series s") {
t.Fatalf("expected ebook_series join in series clause, got %q", clause)
}
}
func TestBuildSortPlan_AuthorJoinsLateralOnPersonKindAuthor(t *testing.T) {
plan, err := NewQueryBuilder("mi").BuildSortPlan(QuerySort{Field: "author", Order: "asc"})
if err != nil {
@@ -576,6 +722,18 @@ func TestBuildSortPlan_SeriesOrdersByNameThenIndex(t *testing.T) {
}
}
func TestBuildSortPlan_SeriesUsesEbookSeriesForEbookScope(t *testing.T) {
plan, err := NewQueryBuilder("mi").
WithMediaScope("ebook").
BuildSortPlan(QuerySort{Field: "series", Order: "asc"})
if err != nil {
t.Fatalf("BuildSortPlan(series) returned error: %v", err)
}
if len(plan.Joins) != 1 || !strings.Contains(plan.Joins[0], "ebook_series sort_series") {
t.Fatalf("expected LEFT JOIN ebook_series, got %v", plan.Joins)
}
}
func TestBuild_SeriesClauseNegationWrapsInNOT(t *testing.T) {
clause, _, err := NewQueryBuilder("mi").Build(QueryDefinition{
Match: "all",
@@ -713,3 +871,18 @@ func TestQueryBuilder_LastWatched_RequiresCTE(t *testing.T) {
t.Fatalf("after lastWatchedExpr, builder should require CTE")
}
}
func TestUserHistoryCTESQLIncludesCompletedEbookReaderProgress(t *testing.T) {
sql := UserHistoryCTESQL(1)
expectedFragments := []string{
"FROM ebook_reader_progress erp",
"erp.progress >= 0.9",
"erp.content_id",
"erp.updated_at",
}
for _, fragment := range expectedFragments {
if !strings.Contains(sql, fragment) {
t.Fatalf("expected CTE to contain %q, got:\n%s", fragment, sql)
}
}
}
+9 -2
View File
@@ -186,8 +186,9 @@ func (q QueryDefinition) ValidateWithOptions(allowPersonalizedSorts, allowPerson
normalized.MediaScope != "movie" &&
normalized.MediaScope != "series" &&
normalized.MediaScope != "episode" &&
normalized.MediaScope != "audiobook" {
return fmt.Errorf("media_scope must be 'movie', 'series', 'episode', or 'audiobook'")
normalized.MediaScope != "audiobook" &&
normalized.MediaScope != "ebook" {
return fmt.Errorf("media_scope must be 'movie', 'series', 'episode', 'audiobook', or 'ebook'")
}
if normalized.Match != "all" && normalized.Match != "any" {
@@ -206,6 +207,9 @@ func (q QueryDefinition) ValidateWithOptions(allowPersonalizedSorts, allowPerson
if def.personalized && !allowPersonalizedFields {
return fmt.Errorf("groups[%d].rules[%d].field %q requires profile scope", i, j, rule.Field)
}
if normalized.MediaScope == "ebook" && rule.Field == "narrator" {
return fmt.Errorf("groups[%d].rules[%d].field %q is not supported for ebook media_scope", i, j, rule.Field)
}
if !def.validOps[rule.Op] {
return fmt.Errorf("groups[%d].rules[%d].op %q is not supported for field %q", i, j, rule.Op, rule.Field)
}
@@ -220,6 +224,9 @@ func (q QueryDefinition) ValidateWithOptions(allowPersonalizedSorts, allowPerson
if def.personalized && !allowPersonalizedSorts {
return fmt.Errorf("sort.field %q requires profile scope", normalized.Sort.Field)
}
if normalized.MediaScope == "ebook" && normalized.Sort.Field == "narrator" {
return fmt.Errorf("sort.field %q is not supported for ebook media_scope", normalized.Sort.Field)
}
}
if normalized.Sort.Order != "" && normalized.Sort.Order != "asc" && normalized.Sort.Order != "desc" {
return fmt.Errorf("sort.order must be 'asc' or 'desc'")
+39
View File
@@ -72,6 +72,45 @@ func TestValidate_AudiobookMediaScope(t *testing.T) {
}
}
func TestValidate_EbookMediaScope(t *testing.T) {
qd := QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []QueryGroup{},
Sort: QuerySort{Field: "title", Order: "asc"},
}
if err := qd.Validate(); err != nil {
t.Fatalf("expected ebook media scope to be valid, got %v", err)
}
}
func TestValidate_EbookMediaScopeRejectsNarratorRule(t *testing.T) {
qd := QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []QueryGroup{{
Match: "all",
Rules: []QueryRule{{Field: "narrator", Op: "is", Value: "Narrator"}},
}},
Sort: QuerySort{Field: "title", Order: "asc"},
}
if err := qd.Validate(); err == nil {
t.Fatal("expected ebook narrator rule to be rejected")
}
}
func TestValidate_EbookMediaScopeRejectsNarratorSort(t *testing.T) {
qd := QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []QueryGroup{},
Sort: QuerySort{Field: "narrator", Order: "asc"},
}
if err := qd.Validate(); err == nil {
t.Fatal("expected ebook narrator sort to be rejected")
}
}
func TestValidateWithSortScope_RejectsPersonalizedSortWithoutProfileScope(t *testing.T) {
qd := QueryDefinition{
Match: "all",
+642
View File
@@ -0,0 +1,642 @@
package ebooks
// Enricher periodically enriches ebook media_items that are missing metadata
// by querying the configured metadata-provider chain for each item's library
// folder.
import (
"context"
"fmt"
"log/slog"
"os"
"reflect"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/metadata"
"github.com/Silo-Server/silo-server/internal/models"
)
const (
ebookMetadataImageProviderID = "ebook-metadata"
defaultEnrichBatchSize = 50
defaultEnrichWorkers = 4
)
func ebookContentType() string {
return "ebook"
}
func ebookEnrichWorkers() int {
n := defaultEnrichWorkers
if v := os.Getenv("SILO_EBOOK_ENRICH_WORKERS"); v != "" {
if parsed, err := strconv.Atoi(v); err == nil && parsed > 0 {
n = parsed
}
}
if n > defaultEnrichBatchSize {
n = defaultEnrichBatchSize
}
return n
}
type enrichmentItemRow struct {
ContentID string
Title string
Year int
FolderID int
Language string
Author string
ProviderIDs map[string]string
}
// Enricher drives the ebook metadata enrichment sweep.
type Enricher struct {
pool *pgxpool.Pool
chainRepo *metadata.ChainRepository
resolver *metadata.PluginResolverAdapter
itemRepo *catalog.ItemRepository
personRepo *catalog.PersonRepository
providerIDs *catalog.ProviderIDRepository
imageCacher metadata.ImageCacher
batchSize int
workers int
}
func NewEnricher(
pool *pgxpool.Pool,
chainRepo *metadata.ChainRepository,
resolver *metadata.PluginResolverAdapter,
itemRepo *catalog.ItemRepository,
personRepo *catalog.PersonRepository,
providerIDs *catalog.ProviderIDRepository,
) *Enricher {
return &Enricher{
pool: pool,
chainRepo: chainRepo,
resolver: resolver,
itemRepo: itemRepo,
personRepo: personRepo,
providerIDs: providerIDs,
batchSize: defaultEnrichBatchSize,
workers: ebookEnrichWorkers(),
}
}
func (e *Enricher) SetImageCacher(cacher metadata.ImageCacher) {
if e == nil {
return
}
e.imageCacher = cacher
}
func (e *Enricher) Run(ctx context.Context) (int, error) {
if e == nil || e.pool == nil || e.chainRepo == nil {
return 0, nil
}
items, err := e.claimBatch(ctx)
if err != nil {
return 0, fmt.Errorf("ebook enrichment: claim batch: %w", err)
}
if len(items) == 0 {
return 0, nil
}
slog.Info("ebook enrichment: sweep started",
"count", len(items),
"workers", e.workers,
)
enriched := e.runBatch(ctx, items, e.enrichItem)
slog.Info("ebook enrichment: sweep complete",
"attempted", len(items),
"enriched", enriched,
)
return enriched, nil
}
func (e *Enricher) runBatch(ctx context.Context, items []enrichmentItemRow, enrichFn func(context.Context, enrichmentItemRow) error) int {
workers := e.workers
if workers <= 0 {
workers = 1
}
if workers > len(items) {
workers = len(items)
}
ch := make(chan enrichmentItemRow, workers)
var (
wg sync.WaitGroup
enriched int64
)
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for item := range ch {
if ctx.Err() != nil {
continue
}
if err := enrichFn(ctx, item); err != nil {
slog.Warn("ebook enrichment: item failed",
"content_id", item.ContentID,
"title", item.Title,
"error", err,
)
continue
}
atomic.AddInt64(&enriched, 1)
}
}()
}
for _, item := range items {
if ctx.Err() != nil {
break
}
ch <- item
}
close(ch)
wg.Wait()
return int(enriched)
}
func (e *Enricher) claimBatch(ctx context.Context) ([]enrichmentItemRow, error) {
rows, err := e.pool.Query(ctx, `
SELECT
mi.content_id,
mi.title,
mi.year,
COALESCE(mil.media_folder_id, 0) AS folder_id,
COALESCE(mf.metadata_language, 'en') AS language,
COALESCE(
(SELECT p.name
FROM item_people ip
JOIN people p ON p.id = ip.person_id
WHERE ip.content_id = mi.content_id
AND ip.kind = 7
ORDER BY ip.sort_order, ip.id
LIMIT 1),
''
) AS author
FROM media_items mi
LEFT JOIN media_item_libraries mil ON mil.content_id = mi.content_id
LEFT JOIN media_folders mf ON mf.id = mil.media_folder_id
WHERE mi.type = 'ebook'
AND (mi.poster_path IS NULL OR mi.poster_path = '')
AND mi.last_refreshed IS NULL
ORDER BY mi.created_at ASC
LIMIT $1
`, e.batchSize)
if err != nil {
return nil, fmt.Errorf("querying unenriched ebooks: %w", err)
}
defer rows.Close()
var items []enrichmentItemRow
seen := make(map[string]struct{})
for rows.Next() {
var item enrichmentItemRow
if err := rows.Scan(
&item.ContentID,
&item.Title,
&item.Year,
&item.FolderID,
&item.Language,
&item.Author,
); err != nil {
return nil, fmt.Errorf("scanning ebook enrichment row: %w", err)
}
if _, dup := seen[item.ContentID]; dup {
continue
}
seen[item.ContentID] = struct{}{}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterating ebook enrichment rows: %w", err)
}
if e.providerIDs != nil {
for i := range items {
pids, err := e.providerIDs.GetByContentID(ctx, items[i].ContentID)
if err == nil {
items[i].ProviderIDs = providerIDMapFromRows(pids)
}
}
}
return items, nil
}
func (e *Enricher) enrichItem(ctx context.Context, item enrichmentItemRow) error {
if item.FolderID == 0 {
slog.Debug("ebook enrichment: item has no library folder, skipping",
"content_id", item.ContentID,
"title", item.Title,
)
return e.stampLastRefreshed(ctx, item.ContentID)
}
providers, err := metadata.ResolveChain(ctx, item.FolderID, ebookContentType(), e.chainRepo, e.resolver)
if err != nil {
return fmt.Errorf("resolving ebook chain for folder %d: %w", item.FolderID, err)
}
if len(providers) == 0 {
slog.Debug("ebook enrichment: no providers in chain",
"content_id", item.ContentID,
"folder_id", item.FolderID,
)
return e.stampLastRefreshed(ctx, item.ContentID)
}
searchQuery, accumulatedIDs := buildEbookSearchQuery(item)
for _, p := range providers {
sp, ok := p.(metadata.SearchProvider)
if !ok {
continue
}
results, searchErr := sp.Search(ctx, searchQuery)
if searchErr != nil {
slog.Warn("ebook enrichment: search error",
"provider", p.Slug(),
"content_id", item.ContentID,
"error", searchErr,
)
continue
}
if len(results) == 0 {
continue
}
for k, v := range results[0].ProviderIDs {
if v != "" {
if _, exists := accumulatedIDs[k]; !exists {
accumulatedIDs[k] = v
}
}
}
slog.Debug("ebook enrichment: search result",
"provider", p.Slug(),
"content_id", item.ContentID,
"matched_ids", accumulatedIDs,
)
}
accumulator := &metadata.MetadataResult{
ProviderIDs: accumulatedIDs,
}
for _, p := range providers {
mp, ok := p.(metadata.MetadataProvider)
if !ok {
continue
}
result, getErr := mp.GetMetadata(ctx, buildEbookMetadataRequest(accumulatedIDs, item.Language))
if getErr != nil {
slog.Warn("ebook enrichment: GetMetadata error",
"provider", p.Slug(),
"content_id", item.ContentID,
"error", getErr,
)
continue
}
if result == nil || !result.HasMetadata {
continue
}
mergeEnrichmentProviderIDs(accumulator, result)
accumulatedIDs = accumulator.ProviderIDs
metadata.MergeMetadata(result, accumulator, nil, metadata.MergeFillEmpty)
slog.Debug("ebook enrichment: metadata received",
"provider", p.Slug(),
"content_id", item.ContentID,
"has_poster", result.PosterPath != "",
"has_overview", result.Overview != "",
)
}
if !accumulator.HasMetadata && accumulator.PosterPath == "" && accumulator.Overview == "" {
slog.Info("ebook enrichment: no metadata found",
"content_id", item.ContentID,
"title", item.Title,
)
return e.stampLastRefreshed(ctx, item.ContentID)
}
e.cacheRemotePoster(ctx, item.ContentID, accumulator)
if err := e.persist(ctx, item.ContentID, accumulatedIDs, accumulator); err != nil {
return fmt.Errorf("persisting enrichment for %s: %w", item.ContentID, err)
}
slog.Info("ebook enrichment: enriched",
"content_id", item.ContentID,
"title", item.Title,
"poster", accumulator.PosterPath != "",
"overview", accumulator.Overview != "",
"people", len(filterEbookPeople(accumulator.People)),
)
return nil
}
func (e *Enricher) cacheRemotePoster(ctx context.Context, contentID string, result *metadata.MetadataResult) {
if e == nil || result == nil || result.PosterPath == "" {
return
}
if !strings.HasPrefix(result.PosterPath, "http://") && !strings.HasPrefix(result.PosterPath, "https://") {
return
}
if isNilImageCacher(e.imageCacher) {
return
}
cached, err := e.imageCacher.CacheImage(ctx, metadata.CacheImageRequest{
SourceURL: result.PosterPath,
ProviderID: ebookMetadataImageProviderID,
ContentType: "ebooks",
ContentID: contentID,
ImageType: metadata.ImagePoster,
})
if err != nil {
slog.Warn("ebook enrichment: poster cache failed, keeping provider URL",
"content_id", contentID,
"url", result.PosterPath,
"error", err,
)
return
}
if cached == nil {
slog.Warn("ebook enrichment: poster cache returned no result, keeping provider URL",
"content_id", contentID,
"url", result.PosterPath,
)
return
}
if storedPath := cachedOriginalImagePath(cached.BasePath, cached.Ext); storedPath != "" {
result.PosterPath = storedPath
}
if cached.Thumbhash != "" {
result.PosterThumbhash = cached.Thumbhash
}
}
func isNilImageCacher(cacher metadata.ImageCacher) bool {
if cacher == nil {
return true
}
value := reflect.ValueOf(cacher)
switch value.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Ptr, reflect.Slice:
return value.IsNil()
default:
return false
}
}
func cachedOriginalImagePath(basePath, ext string) string {
if basePath == "" {
return ""
}
if strings.Contains(basePath, "/original.") {
return basePath
}
if ext == "" {
ext = ".jpg"
}
return strings.TrimRight(basePath, "/") + "/original" + ext
}
func (e *Enricher) persist(ctx context.Context, contentID string, providerIDs map[string]string, result *metadata.MetadataResult) error {
upd := &catalog.MetadataUpdate{}
if result.PosterPath != "" {
upd.PosterPath = &result.PosterPath
}
if result.PosterThumbhash != "" {
upd.PosterThumbhash = &result.PosterThumbhash
}
if result.BackdropPath != "" {
upd.BackdropPath = &result.BackdropPath
}
if result.BackdropThumbhash != "" {
upd.BackdropThumbhash = &result.BackdropThumbhash
}
if result.LogoPath != "" {
upd.LogoPath = &result.LogoPath
}
if result.Overview != "" {
upd.Overview = &result.Overview
}
if result.Tagline != "" {
upd.Tagline = &result.Tagline
}
if result.ReleaseDate != "" {
upd.ReleaseDate = &result.ReleaseDate
}
if len(result.Genres) > 0 {
genres := append([]string(nil), result.Genres...)
upd.Genres = &genres
}
if len(result.Studios) > 0 {
studios := append([]string(nil), result.Studios...)
upd.Studios = &studios
}
if result.ContentRating != "" {
upd.ContentRating = &result.ContentRating
}
if result.Runtime > 0 {
upd.Runtime = &result.Runtime
}
if result.Year > 0 {
upd.Year = &result.Year
}
providerIDs = filterEbookProviderIDs(providerIDs)
if e.providerIDs != nil && len(providerIDs) > 0 {
if err := e.providerIDs.ReplaceByContentID(ctx, contentID, providerIDs); err != nil {
slog.Warn("ebook enrichment: failed to persist provider IDs",
"content_id", contentID,
"error", err,
)
}
}
if err := e.updateMetadataAndTimestamps(ctx, contentID, upd); err != nil {
return err
}
authors := filterEbookPeople(result.People)
if len(authors) > 0 && e.personRepo != nil && e.itemRepo != nil {
if err := e.persistPeople(ctx, contentID, authors); err != nil {
slog.Warn("ebook enrichment: failed to persist people",
"content_id", contentID,
"error", err,
)
}
}
return nil
}
func (e *Enricher) updateMetadataAndTimestamps(ctx context.Context, contentID string, upd *catalog.MetadataUpdate) error {
if e.itemRepo == nil {
return nil
}
if err := e.itemRepo.UpdateMetadata(ctx, contentID, upd); err != nil {
return fmt.Errorf("UpdateMetadata: %w", err)
}
return e.stampLastRefreshed(ctx, contentID)
}
func (e *Enricher) stampLastRefreshed(ctx context.Context, contentID string) error {
if e.pool == nil {
return nil
}
now := time.Now().UTC()
_, err := e.pool.Exec(ctx, `
UPDATE media_items
SET last_refreshed = $1,
matched_at = COALESCE(matched_at, $1),
status = CASE WHEN status = 'pending' THEN 'matched' ELSE status END
WHERE content_id = $2
`, now, contentID)
return err
}
func (e *Enricher) persistPeople(ctx context.Context, contentID string, people []models.ItemPerson) error {
people = filterEbookPeople(people)
if len(people) == 0 {
return nil
}
persons := make([]models.Person, len(people))
for i := range people {
persons[i] = people[i].Person
}
personIDs, err := e.personRepo.BatchFindOrCreate(ctx, persons)
if err != nil {
return fmt.Errorf("BatchFindOrCreate people: %w", err)
}
linked := make([]models.ItemPerson, 0, len(people))
for i := range people {
if i >= len(personIDs) || personIDs[i] == 0 {
continue
}
ip := people[i]
ip.Person.ID = personIDs[i]
linked = append(linked, ip)
}
if len(linked) == 0 {
return nil
}
return e.itemRepo.ReplacePeople(ctx, contentID, linked)
}
func filterEbookPeople(people []models.ItemPerson) []models.ItemPerson {
authors := make([]models.ItemPerson, 0, len(people))
for _, person := range people {
if person.Kind != models.PersonKindAuthor {
continue
}
person.SortOrder = len(authors)
authors = append(authors, person)
}
return authors
}
func buildEbookSearchQuery(item enrichmentItemRow) (metadata.SearchQuery, map[string]string) {
accumulatedIDs := filterEbookProviderIDs(item.ProviderIDs)
if accumulatedIDs == nil {
accumulatedIDs = map[string]string{}
}
return metadata.SearchQuery{
Title: item.Title,
Year: item.Year,
ContentType: ebookContentType(),
ProviderIDs: accumulatedIDs,
Language: item.Language,
}, accumulatedIDs
}
func buildEbookMetadataRequest(providerIDs map[string]string, language string) metadata.MetadataRequest {
return metadata.MetadataRequest{
ProviderIDs: filterEbookProviderIDs(providerIDs),
ContentType: ebookContentType(),
Language: language,
}
}
func mergeEnrichmentProviderIDs(dst *metadata.MetadataResult, src *metadata.MetadataResult) {
if src == nil || len(src.ProviderIDs) == 0 {
return
}
if dst.ProviderIDs == nil {
dst.ProviderIDs = make(map[string]string, len(src.ProviderIDs))
}
for k, v := range filterEbookProviderIDs(src.ProviderIDs) {
if v != "" {
if _, exists := dst.ProviderIDs[k]; !exists {
dst.ProviderIDs[k] = v
}
}
}
}
func filterEbookProviderIDs(providerIDs map[string]string) map[string]string {
if len(providerIDs) == 0 {
return nil
}
filtered := make(map[string]string, len(providerIDs))
for provider, providerID := range providerIDs {
provider = strings.TrimSpace(provider)
providerID = strings.TrimSpace(providerID)
if provider == "" || providerID == "" {
continue
}
provider = strings.ToLower(provider)
if isEbookASINProvider(provider) {
continue
}
filtered[provider] = providerID
}
if len(filtered) == 0 {
return nil
}
return filtered
}
func isEbookASINProvider(provider string) bool {
normalized := strings.ReplaceAll(strings.ReplaceAll(provider, "_", ""), "-", "")
return normalized == "asin" || normalized == "audibleasin"
}
func providerIDMapFromRows(rows []*models.MediaItemProviderID) map[string]string {
if len(rows) == 0 {
return nil
}
m := make(map[string]string, len(rows))
for _, r := range rows {
if r != nil {
for provider, providerID := range filterEbookProviderIDs(map[string]string{
r.Provider: r.ProviderID,
}) {
m[provider] = providerID
}
}
}
return m
}
+340
View File
@@ -0,0 +1,340 @@
package ebooks
import (
"context"
"errors"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/metadata"
"github.com/Silo-Server/silo-server/internal/models"
)
func TestEbookContentType(t *testing.T) {
if got := ebookContentType(); got != "ebook" {
t.Fatalf("ebookContentType() = %q, want ebook", got)
}
}
func TestFilterEbookPeopleKeepsAuthorsOnly(t *testing.T) {
people := []models.ItemPerson{
{Person: models.Person{Name: "Author One"}, Kind: models.PersonKindAuthor, SortOrder: 7},
{Person: models.Person{Name: "Narrator One"}, Kind: models.PersonKindNarrator, SortOrder: 8},
{Person: models.Person{Name: "Writer One"}, Kind: models.PersonKindWriter, SortOrder: 9},
{Person: models.Person{Name: "Author Two"}, Kind: models.PersonKindAuthor, SortOrder: 10},
}
got := filterEbookPeople(people)
if len(got) != 2 {
t.Fatalf("filtered people len = %d, want 2: %+v", len(got), got)
}
for i, p := range got {
if p.Kind != models.PersonKindAuthor {
t.Fatalf("filtered[%d].Kind = %v, want author", i, p.Kind)
}
if p.SortOrder != i {
t.Fatalf("filtered[%d].SortOrder = %d, want %d", i, p.SortOrder, i)
}
}
if got[0].Person.Name != "Author One" || got[1].Person.Name != "Author Two" {
t.Fatalf("filtered author order = %+v", got)
}
}
func TestEbookEnrichWorkersFromEnv(t *testing.T) {
t.Setenv("SILO_EBOOK_ENRICH_WORKERS", "12")
if got := ebookEnrichWorkers(); got != 12 {
t.Fatalf("ebookEnrichWorkers() = %d, want 12", got)
}
t.Setenv("SILO_EBOOK_ENRICH_WORKERS", "0")
if got := ebookEnrichWorkers(); got != defaultEnrichWorkers {
t.Fatalf("ebookEnrichWorkers() with zero = %d, want default %d", got, defaultEnrichWorkers)
}
t.Setenv("SILO_EBOOK_ENRICH_WORKERS", "999")
if got := ebookEnrichWorkers(); got != defaultEnrichBatchSize {
t.Fatalf("ebookEnrichWorkers() capped = %d, want %d", got, defaultEnrichBatchSize)
}
}
func TestEnricherRunFansOut(t *testing.T) {
const wantWorkers = 4
const itemCount = 16
items := make([]enrichmentItemRow, itemCount)
for i := range items {
items[i] = enrichmentItemRow{ContentID: "test", Title: "t"}
}
var inFlight int32
var maxInFlight int32
var signaled int32
var wg sync.WaitGroup
wg.Add(wantWorkers)
gate := make(chan struct{})
enrich := func(ctx context.Context, item enrichmentItemRow) error {
cur := atomic.AddInt32(&inFlight, 1)
defer atomic.AddInt32(&inFlight, -1)
for {
prev := atomic.LoadInt32(&maxInFlight)
if cur <= prev || atomic.CompareAndSwapInt32(&maxInFlight, prev, cur) {
break
}
}
if atomic.AddInt32(&signaled, 1) <= wantWorkers {
wg.Done()
}
select {
case <-gate:
case <-time.After(2 * time.Second):
}
return nil
}
go func() {
wg.Wait()
close(gate)
}()
e := &Enricher{workers: wantWorkers, batchSize: itemCount}
e.runBatch(context.Background(), items, enrich)
if got := atomic.LoadInt32(&maxInFlight); got < wantWorkers {
t.Errorf("max in-flight = %d, want >= %d", got, wantWorkers)
}
}
func TestCacheRemotePosterCachesProviderURL(t *testing.T) {
cacher := &fakeEbookImageCacher{}
e := &Enricher{imageCacher: cacher}
result := &metadata.MetadataResult{
PosterPath: "https://example.test/book.jpg",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if cacher.calls != 1 {
t.Fatalf("CacheImage calls = %d, want 1", cacher.calls)
}
if cacher.req.ProviderID != ebookMetadataImageProviderID {
t.Fatalf("ProviderID = %q, want %q", cacher.req.ProviderID, ebookMetadataImageProviderID)
}
if cacher.req.ContentType != "ebooks" || cacher.req.ContentID != "content-1" {
t.Fatalf("cache target = %q/%q", cacher.req.ContentType, cacher.req.ContentID)
}
if result.PosterPath != "ebook-metadata/ebooks/content-1/poster/original.webp" {
t.Fatalf("PosterPath = %q", result.PosterPath)
}
if result.PosterThumbhash != "thumb" {
t.Fatalf("PosterThumbhash = %q", result.PosterThumbhash)
}
}
func TestCacheRemotePosterSkipsNilCacher(t *testing.T) {
e := &Enricher{}
result := &metadata.MetadataResult{
PosterPath: "https://example.test/book.jpg",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if result.PosterPath != "https://example.test/book.jpg" {
t.Fatalf("PosterPath = %q, want provider URL preserved", result.PosterPath)
}
}
func TestCacheRemotePosterSkipsTypedNilCacher(t *testing.T) {
var cacher *fakeEbookImageCacher
e := &Enricher{imageCacher: cacher}
result := &metadata.MetadataResult{
PosterPath: "https://example.test/book.jpg",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if result.PosterPath != "https://example.test/book.jpg" {
t.Fatalf("PosterPath = %q, want provider URL preserved", result.PosterPath)
}
}
func TestCacheRemotePosterSkipsAlreadyCachedPath(t *testing.T) {
cacher := &fakeEbookImageCacher{}
e := &Enricher{imageCacher: cacher}
result := &metadata.MetadataResult{
PosterPath: "local/ebooks/content-1/poster/original.webp",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if cacher.calls != 0 {
t.Fatalf("CacheImage calls = %d, want 0", cacher.calls)
}
if result.PosterPath != "local/ebooks/content-1/poster/original.webp" {
t.Fatalf("PosterPath = %q", result.PosterPath)
}
}
func TestCacheRemotePosterPreservesProviderURLOnCacheError(t *testing.T) {
cacher := &fakeEbookImageCacher{err: errors.New("cache failed")}
e := &Enricher{imageCacher: cacher}
result := &metadata.MetadataResult{
PosterPath: "https://example.test/book.jpg",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if cacher.calls != 1 {
t.Fatalf("CacheImage calls = %d, want 1", cacher.calls)
}
if result.PosterPath != "https://example.test/book.jpg" {
t.Fatalf("PosterPath = %q, want provider URL preserved", result.PosterPath)
}
}
func TestCacheRemotePosterPreservesProviderURLOnNilCacheResult(t *testing.T) {
cacher := &fakeEbookImageCacher{returnNil: true}
e := &Enricher{imageCacher: cacher}
result := &metadata.MetadataResult{
PosterPath: "https://example.test/book.jpg",
}
e.cacheRemotePoster(context.Background(), "content-1", result)
if cacher.calls != 1 {
t.Fatalf("CacheImage calls = %d, want 1", cacher.calls)
}
if result.PosterPath != "https://example.test/book.jpg" {
t.Fatalf("PosterPath = %q, want provider URL preserved", result.PosterPath)
}
}
func TestMergeEnrichmentProviderIDsKeepsExistingIDs(t *testing.T) {
dst := &metadata.MetadataResult{ProviderIDs: map[string]string{"isbn": "9780306406157"}}
src := &metadata.MetadataResult{ProviderIDs: map[string]string{"isbn": "new", "openlibrary": "OL1M", "empty": ""}}
mergeEnrichmentProviderIDs(dst, src)
if got := dst.ProviderIDs["isbn"]; got != "9780306406157" {
t.Fatalf("isbn = %q, want original", got)
}
if got := dst.ProviderIDs["openlibrary"]; got != "OL1M" {
t.Fatalf("openlibrary = %q, want OL1M", got)
}
if _, exists := dst.ProviderIDs["empty"]; exists {
t.Fatal("empty provider ID should not be merged")
}
}
func TestMergeEnrichmentProviderIDsDropsAsinIDs(t *testing.T) {
dst := &metadata.MetadataResult{ProviderIDs: map[string]string{"isbn": "9780306406157"}}
src := &metadata.MetadataResult{
ProviderIDs: map[string]string{
"ASIN": "B00TEST",
"audible_asin": "B00AUDIO",
"openlibrary": "OL1M",
},
}
mergeEnrichmentProviderIDs(dst, src)
if _, exists := dst.ProviderIDs["ASIN"]; exists {
t.Fatal("ASIN provider ID should not be merged for ebooks")
}
if _, exists := dst.ProviderIDs["audible_asin"]; exists {
t.Fatal("audible_asin provider ID should not be merged for ebooks")
}
if got := dst.ProviderIDs["openlibrary"]; got != "OL1M" {
t.Fatalf("openlibrary = %q, want OL1M", got)
}
}
func TestBuildEbookSearchQueryUsesFilteredScannerISBN(t *testing.T) {
item := enrichmentItemRow{
Title: "Tagged Ebook",
Year: 2024,
Language: "en",
ProviderIDs: map[string]string{
" ISBN ": " 9780306406157 ",
"ASIN": "B00TEST",
"audible_asin": "B00AUDIO",
},
}
query, ids := buildEbookSearchQuery(item)
if query.Title != "Tagged Ebook" || query.Year != 2024 || query.Language != "en" || query.ContentType != "ebook" {
t.Fatalf("query basics = %+v", query)
}
if got := query.ProviderIDs["isbn"]; got != "9780306406157" {
t.Fatalf("query isbn = %q, want scanner ISBN", got)
}
if _, exists := query.ProviderIDs["ASIN"]; exists {
t.Fatal("query should not include ASIN for ebooks")
}
if got := ids["isbn"]; got != "9780306406157" {
t.Fatalf("accumulated isbn = %q, want scanner ISBN", got)
}
}
func TestFilterEbookProviderIDsDropsAsinAliases(t *testing.T) {
got := filterEbookProviderIDs(map[string]string{
"ASIN": "B00TEST",
"audibleASIN": "B00AUDIO",
"audible-asin": "B00AUDIO2",
"audible_asin": "B00AUDIO3",
" ISBN ": " 9780306406157 ",
" OpenLibraryID": " OL1M ",
})
if got["asin"] != "" || got["audibleasin"] != "" || got["audible-asin"] != "" || got["audible_asin"] != "" {
t.Fatalf("ASIN aliases should be filtered, got %+v", got)
}
if got["isbn"] != "9780306406157" || got["openlibraryid"] != "OL1M" {
t.Fatalf("filtered IDs = %+v, want ISBN and OpenLibraryID only", got)
}
}
func TestBuildEbookMetadataRequestCarriesAccumulatedISBN(t *testing.T) {
req := buildEbookMetadataRequest(map[string]string{
" ISBN ": " 9780306406157 ",
"openlibrary": "OL1M",
}, "fr")
if req.ContentType != "ebook" || req.Language != "fr" {
t.Fatalf("request basics = %+v", req)
}
if got := req.ProviderIDs["isbn"]; got != "9780306406157" {
t.Fatalf("request isbn = %q, want normalized key/value", got)
}
if got := req.ProviderIDs["openlibrary"]; got != "OL1M" {
t.Fatalf("request openlibrary = %q, want OL1M", got)
}
}
type fakeEbookImageCacher struct {
calls int
req metadata.CacheImageRequest
err error
returnNil bool
}
func (f *fakeEbookImageCacher) CacheImage(_ context.Context, req metadata.CacheImageRequest) (*metadata.CacheImageResult, error) {
f.calls++
f.req = req
if f.err != nil {
return nil, f.err
}
if f.returnNil {
return nil, nil
}
return &metadata.CacheImageResult{
BasePath: req.ProviderID + "/" + req.ContentType + "/" + req.ContentID + "/poster",
Thumbhash: "thumb",
Ext: ".webp",
}, nil
}
+16
View File
@@ -114,6 +114,22 @@ func (c *Cacher) CacheAudiobookCover(ctx context.Context, data []byte, contentID
return res.BasePath, res.Ext, res.Thumbhash, nil
}
// CacheEbookCover stores an embedded ebook cover under
// "local/ebooks/{contentID}/poster/..." using the same poster variants as
// provider-hosted book artwork.
func (c *Cacher) CacheEbookCover(ctx context.Context, data []byte, contentID string) (basePath string, ext string, thumbhash string, err error) {
res, err := c.CacheBytes(ctx, data, CacheRequest{
ProviderID: "local",
ContentType: "ebooks",
ContentID: contentID,
ImageType: metadata.ImagePoster,
})
if err != nil {
return "", "", "", err
}
return res.BasePath, res.Ext, res.Thumbhash, nil
}
// CacheBytes performs the same variant generation, thumbhash, and S3 upload as
// Cache but starts from raw image bytes already in hand. Used by the
// audiobook scanner to push embedded M4B cover art into S3 without round-
@@ -722,6 +722,19 @@ var mediaItemMergeSteps = []mediaItemMergeStep{
last_codec_video = CASE WHEN EXCLUDED.updated_at >= user_watch_progress.updated_at THEN EXCLUDED.last_codec_video ELSE user_watch_progress.last_codec_video END,
last_edition_key = CASE WHEN EXCLUDED.updated_at >= user_watch_progress.updated_at THEN EXCLUDED.last_edition_key ELSE user_watch_progress.last_edition_key END`},
{"delete source progress", `DELETE FROM user_watch_progress WHERE media_item_id = $1`},
{"merge ebook reader progress", `
INSERT INTO ebook_reader_progress (
user_id, profile_id, content_id, file_id, location, progress, updated_at
)
SELECT user_id, profile_id, $2, file_id, location, progress, updated_at
FROM ebook_reader_progress
WHERE content_id = $1
ON CONFLICT (user_id, profile_id, content_id) DO UPDATE
SET file_id = CASE WHEN EXCLUDED.updated_at >= ebook_reader_progress.updated_at THEN EXCLUDED.file_id ELSE ebook_reader_progress.file_id END,
location = CASE WHEN EXCLUDED.updated_at >= ebook_reader_progress.updated_at THEN EXCLUDED.location ELSE ebook_reader_progress.location END,
progress = CASE WHEN EXCLUDED.updated_at >= ebook_reader_progress.updated_at THEN EXCLUDED.progress ELSE ebook_reader_progress.progress END,
updated_at = GREATEST(ebook_reader_progress.updated_at, EXCLUDED.updated_at)`},
{"delete source ebook reader progress", `DELETE FROM ebook_reader_progress WHERE content_id = $1`},
{"move history", `UPDATE user_watch_history SET media_item_id = $2 WHERE media_item_id = $1`},
{"merge hidden history", `
INSERT INTO user_history_hidden_items (user_id, profile_id, media_item_id, hidden_before, updated_at)
@@ -110,6 +110,25 @@ func TestMergeStepPlaceholdersAreBounded(t *testing.T) {
}
}
func TestMergeStepsPreserveEbookReaderProgress(t *testing.T) {
var hasMerge bool
var hasDelete bool
for _, step := range mediaItemMergeSteps {
if strings.Contains(step.sql, "ebook_reader_progress") && strings.Contains(step.sql, "INSERT INTO") {
hasMerge = true
}
if strings.Contains(step.sql, "ebook_reader_progress") && strings.Contains(step.sql, "DELETE FROM") {
hasDelete = true
}
}
if !hasMerge {
t.Fatal("media item merge steps should merge ebook_reader_progress rows")
}
if !hasDelete {
t.Fatal("media item merge steps should delete source ebook_reader_progress rows")
}
}
func TestBothCandidatesHaveContentDetectsRealUserData(t *testing.T) {
withFiles := canonicalCandidate{ContentID: "a", ActiveFiles: 2}
withLibraries := canonicalCandidate{ContentID: "b", LibraryCount: 1}
+26
View File
@@ -111,6 +111,32 @@ func TestApplyMediaTypeFloorNoopsWithoutSupplementalType(t *testing.T) {
}
}
func TestApplyMediaTypeFloorIncludesEbookSupplement(t *testing.T) {
items := []ScoredItem{
{MediaItemID: "s1", Score: 1.00},
{MediaItemID: "s2", Score: 0.99},
{MediaItemID: "s3", Score: 0.98},
{MediaItemID: "s4", Score: 0.97},
{MediaItemID: "s5", Score: 0.96},
}
candidates := append([]ScoredItem(nil), items...)
candidates = append(candidates, ScoredItem{MediaItemID: "e1", Score: 0.95})
mediaTypes := map[string]string{
"s1": "series",
"s2": "series",
"s3": "series",
"s4": "series",
"s5": "series",
"e1": "ebook",
}
mixed := applyMediaTypeFloor(items, candidates, mediaTypes)
if got := countMediaType(mixed, mediaTypes, "ebook"); got != 1 {
t.Fatalf("expected 1 ebook from supplemental candidates, got %d in %#v", got, mixed)
}
}
func TestApplyGenreCapDoesNotCollapseConcentratedRows(t *testing.T) {
items := []ScoredItem{
{MediaItemID: "a", Score: 1.0},
+10 -5
View File
@@ -63,17 +63,20 @@ func BuildEmbeddingText(item *models.MediaItem) string {
parts = append(parts, fmt.Sprintf(`"%s"`, item.Tagline))
}
if item.Type == "audiobook" {
// Audiobooks credit author (kind=7) and narrator (kind=8). Cast/
// director/writer don't apply. Keep the SQL canonical_text builder
// in recommendations/repo.go in sync with this shape.
if item.Type == "audiobook" || item.Type == "ebook" {
// Books credit authors (kind=7). Audiobooks also credit narrators
// (kind=8); ebooks do not. Cast/director/writer don't apply. Keep
// the SQL canonical_text builder in recommendations/repo.go in sync
// with this shape.
var authors, narrators []models.ItemPerson
for _, p := range item.People {
switch p.Kind {
case models.PersonKindAuthor:
authors = append(authors, p)
case models.PersonKindNarrator:
narrators = append(narrators, p)
if item.Type == "audiobook" {
narrators = append(narrators, p)
}
}
}
sortItemPeople(authors)
@@ -174,6 +177,8 @@ func mediaTypeLabel(t string) string {
return "TV series"
case "audiobook":
return "audiobook"
case "ebook":
return "ebook"
default:
return "movie"
}
@@ -61,6 +61,42 @@ func TestBuildEmbeddingTextAudiobookUsesAuthorAndNarrator(t *testing.T) {
}
}
func TestBuildEmbeddingTextEbookUsesAuthorWithoutNarratorOrCast(t *testing.T) {
item := &models.MediaItem{
Title: "A Wizard of Earthsea",
Type: "ebook",
Year: 1968,
Genres: []string{"Fantasy"},
Overview: "Ged learns the cost of power.",
People: []models.ItemPerson{
{Person: models.Person{ID: 1, Name: "Ursula K. Le Guin"}, Kind: models.PersonKindAuthor},
{Person: models.Person{ID: 2, Name: "Narrator Should Not Apply"}, Kind: models.PersonKindNarrator},
{Person: models.Person{ID: 3, Name: "Actor Should Not Apply"}, Kind: models.PersonKindActor, Character: "Ged"},
{Person: models.Person{ID: 4, Name: "Writer Should Not Apply"}, Kind: models.PersonKindWriter},
},
}
text := BuildEmbeddingText(item)
if !strings.Contains(text, "Fantasy ebook about Ged learns the cost of power.") {
t.Fatalf("ebook lede missing or wrong: %q", text)
}
if !strings.Contains(text, "Written by Ursula K. Le Guin") {
t.Fatalf("ebook author missing: %q", text)
}
for _, unwanted := range []string{
"Narrated by",
"Narrator Should Not Apply",
"Cast:",
"Actor Should Not Apply",
"Writer Should Not Apply",
} {
if strings.Contains(text, unwanted) {
t.Fatalf("ebook text should not contain %q: %q", unwanted, text)
}
}
}
func TestBuildEmbeddingTextSortsActorsBeforeTopFive(t *testing.T) {
item := &models.MediaItem{
Title: "Example",
+1 -1
View File
@@ -11,7 +11,7 @@ import (
const aggregateMediaTypeFloorDivisor = 5
var aggregateSupplementMediaTypes = []string{"movie", "series"}
var aggregateSupplementMediaTypes = []string{"movie", "series", "ebook"}
// ForYou returns personalised recommendations grouped by taste clusters.
// For cold-start users, non-personalized rows are returned.
+155 -131
View File
@@ -39,15 +39,53 @@ func ensureCanonicalDimensions(vec []float32) ([]float32, error) {
const embeddingLockSettingKey = "recommendations.embedding_lock"
const minHNSWEfSearch = 200
const watchedActivityCTE = `
watched_activity AS (
SELECT COALESCE(e.series_id, wp.media_item_id) AS item_id,
wp.media_item_id AS leaf_item_id,
wp.user_id,
wp.profile_id,
wp.user_id::text || ':' || COALESCE(wp.profile_id, '') AS watcher_id,
wp.updated_at,
wp.completed AS completed
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = wp.user_id
AND hhi.profile_id = wp.profile_id
AND hhi.media_item_id = wp.media_item_id
AND wp.updated_at <= hhi.hidden_before
)
UNION ALL
SELECT erp.content_id AS item_id,
erp.content_id AS leaf_item_id,
erp.user_id,
erp.profile_id,
erp.user_id::text || ':' || COALESCE(erp.profile_id, '') AS watcher_id,
erp.updated_at,
erp.progress >= 0.9 AS completed
FROM ebook_reader_progress erp
WHERE erp.progress >= 0.5
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = erp.user_id
AND hhi.profile_id = erp.profile_id
AND hhi.media_item_id = erp.content_id
AND erp.updated_at <= hhi.hidden_before
)
)`
const tasteSeedCandidateQuery = `
WITH watched_counts AS (
SELECT COALESCE(e.series_id, wp.media_item_id) AS item_id,
COUNT(DISTINCT wp.user_id::text || ':' || COALESCE(wp.profile_id, '')) AS watch_count
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND wp.updated_at > NOW() - INTERVAL '180 days'
GROUP BY 1
WITH ` + watchedActivityCTE + `,
watched_counts AS (
SELECT item_id, COUNT(DISTINCT watcher_id) AS watch_count
FROM watched_activity
WHERE updated_at > NOW() - INTERVAL '180 days'
GROUP BY item_id
)
SELECT mi.content_id
FROM media_items mi
@@ -120,6 +158,18 @@ type EmbeddingTextCandidate struct {
CanonicalText string
}
func embeddingEligibilityWhereClause() string {
return recommendationItemEligibilityWhereClause("mi")
}
func recommendationItemEligibilityWhereClause(alias string) string {
alias = strings.TrimSpace(alias)
if alias == "" {
alias = "media_items"
}
return fmt.Sprintf("(%s.status = 'matched' OR %s.type = 'audiobook' OR %s.type = 'ebook')", alias, alias, alias)
}
// UpsertEmbedding stores or updates an embedding for a media item.
func (r *Repo) UpsertEmbedding(ctx context.Context, itemID string, embedding []float32, model, canonicalText string) error {
padded, err := ensureCanonicalDimensions(embedding)
@@ -416,18 +466,18 @@ func (r *Repo) findTasteProfileCandidates(
// ItemsNeedingEmbedding returns content IDs of media items that either have no
// embedding or whose embedding was generated with a different model.
// Audiobooks bypass the status='matched' gate because they don't go through
// the external-provider match pipeline; their tag-derived metadata is
// authoritative as soon as the scan completes.
// Books bypass the status='matched' gate because their scanner/plugin-derived
// metadata is authoritative as soon as the scan/enrichment completes.
func (r *Repo) ItemsNeedingEmbedding(ctx context.Context, currentModel string, limit int) ([]string, error) {
rows, err := r.pool.Query(ctx, `
query := fmt.Sprintf(`
SELECT mi.content_id
FROM media_items mi
LEFT JOIN media_item_embeddings e ON e.media_item_id = mi.content_id
WHERE (mi.status = 'matched' OR mi.type = 'audiobook')
WHERE %s
AND (e.media_item_id IS NULL OR e.model != $1)
LIMIT $2
`, currentModel, limit)
`, embeddingEligibilityWhereClause())
rows, err := r.pool.Query(ctx, query, currentModel, limit)
if err != nil {
return nil, fmt.Errorf("items needing embedding: %w", err)
}
@@ -448,11 +498,11 @@ func (r *Repo) ItemsNeedingEmbedding(ctx context.Context, currentModel string, l
}
func (r *Repo) ListEmbeddingTextCandidates(ctx context.Context, afterID, currentModel string, limit int) ([]EmbeddingTextCandidate, error) {
rows, err := r.pool.Query(ctx, `
query := fmt.Sprintf(`
-- Keep current_text in sync with embeddings.BuildEmbeddingText. This lets
-- the embedding job page over only missing, model-stale, or text-stale rows.
-- Audiobook lines map to embeddings.mediaTypeLabel + the author/narrator
-- branch in BuildEmbeddingText; any divergence here forces every audiobook
-- Book lines map to embeddings.mediaTypeLabel + the author/narrator
-- branch in BuildEmbeddingText; any divergence here forces book items
-- to re-embed every job run.
WITH text_candidates AS (
SELECT mi.content_id,
@@ -461,11 +511,11 @@ func (r *Repo) ListEmbeddingTextCandidates(ctx context.Context, afterID, current
array_to_string(array_remove(ARRAY[
CASE
WHEN cardinality(COALESCE(mi.genres, ARRAY[]::text[])) > 0 AND COALESCE(mi.overview, '') <> ''
THEN array_to_string(mi.genres, ', ') || ' ' || CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' ELSE 'movie' END || ' about ' || substr(mi.overview, 1, 1000)
THEN array_to_string(mi.genres, ', ') || ' ' || CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' WHEN mi.type = 'ebook' THEN 'ebook' ELSE 'movie' END || ' about ' || substr(mi.overview, 1, 1000)
WHEN cardinality(COALESCE(mi.genres, ARRAY[]::text[])) > 0
THEN array_to_string(mi.genres, ', ') || ' ' || CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' ELSE 'movie' END
THEN array_to_string(mi.genres, ', ') || ' ' || CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' WHEN mi.type = 'ebook' THEN 'ebook' ELSE 'movie' END
WHEN COALESCE(mi.overview, '') <> ''
THEN CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' ELSE 'movie' END || '. ' || substr(mi.overview, 1, 1000)
THEN CASE WHEN mi.type = 'series' THEN 'TV series' WHEN mi.type = 'audiobook' THEN 'audiobook' WHEN mi.type = 'ebook' THEN 'ebook' ELSE 'movie' END || '. ' || substr(mi.overview, 1, 1000)
ELSE NULL
END,
CASE WHEN COALESCE(mi.year, 0) > 0
@@ -474,11 +524,11 @@ func (r *Repo) ListEmbeddingTextCandidates(ctx context.Context, afterID, current
END,
CASE WHEN COALESCE(mi.content_rating, '') <> '' THEN 'Rated ' || mi.content_rating ELSE NULL END,
CASE WHEN COALESCE(mi.tagline, '') <> '' THEN '"' || mi.tagline || '"' ELSE NULL END,
CASE WHEN mi.type <> 'audiobook' AND actors.names <> '' THEN 'Cast: ' || actors.names ELSE NULL END,
CASE WHEN mi.type <> 'audiobook' AND directors.names <> '' THEN 'Directed by ' || directors.names ELSE NULL END,
CASE WHEN mi.type NOT IN ('audiobook', 'ebook') AND actors.names <> '' THEN 'Cast: ' || actors.names ELSE NULL END,
CASE WHEN mi.type NOT IN ('audiobook', 'ebook') AND directors.names <> '' THEN 'Directed by ' || directors.names ELSE NULL END,
CASE
WHEN mi.type = 'audiobook' AND authors.names <> '' THEN 'Written by ' || authors.names
WHEN mi.type <> 'audiobook' AND writers.names <> '' THEN 'Written by ' || writers.names
WHEN mi.type IN ('audiobook', 'ebook') AND authors.names <> '' THEN 'Written by ' || authors.names
WHEN mi.type NOT IN ('audiobook', 'ebook') AND writers.names <> '' THEN 'Written by ' || writers.names
ELSE NULL
END,
CASE WHEN mi.type = 'audiobook' AND narrators.names <> '' THEN 'Narrated by ' || narrators.names ELSE NULL END,
@@ -536,7 +586,7 @@ func (r *Repo) ListEmbeddingTextCandidates(ctx context.Context, afterID, current
WHERE ip.content_id = mi.content_id
AND ip.kind = 8
) narrators ON TRUE
WHERE (mi.status = 'matched' OR mi.type = 'audiobook')
WHERE %s
AND ($1 = '' OR mi.content_id > $1)
)
SELECT mi.content_id,
@@ -548,7 +598,8 @@ func (r *Repo) ListEmbeddingTextCandidates(ctx context.Context, afterID, current
OR mi.canonical_text IS DISTINCT FROM mi.current_text
ORDER BY mi.content_id
LIMIT $3
`, afterID, currentModel, limit)
`, embeddingEligibilityWhereClause())
rows, err := r.pool.Query(ctx, query, afterID, currentModel, limit)
if err != nil {
return nil, fmt.Errorf("embedding text candidates: %w", err)
}
@@ -583,7 +634,8 @@ func (r *Repo) EmbeddingCount(ctx context.Context) (int, error) {
// (used by the admin worker UI) lines up with what the job actually does.
func (r *Repo) TotalMediaItemCount(ctx context.Context) (int, error) {
var count int
err := r.pool.QueryRow(ctx, `SELECT COUNT(*) FROM media_items WHERE status = 'matched' OR type = 'audiobook'`).Scan(&count)
query := fmt.Sprintf(`SELECT COUNT(*) FROM media_items mi WHERE %s`, embeddingEligibilityWhereClause())
err := r.pool.QueryRow(ctx, query).Scan(&count)
if err != nil {
return 0, fmt.Errorf("total media item count: %w", err)
}
@@ -1137,6 +1189,32 @@ func (r *Repo) GetWatchProgressForUser(ctx context.Context, userID int, profileI
return result, rows.Err()
}
func (r *Repo) GetEbookReaderProgressForUser(ctx context.Context, userID int, profileID string) ([]WatchProgressRow, error) {
rows, err := r.pool.Query(ctx, `
SELECT content_id,
progress::double precision AS position_seconds,
1::double precision AS duration_seconds,
progress >= 0.9 AS completed,
updated_at
FROM ebook_reader_progress
WHERE user_id = $1 AND profile_id = $2`,
userID, profileID)
if err != nil {
return nil, fmt.Errorf("get ebook reader progress: %w", err)
}
defer rows.Close()
var result []WatchProgressRow
for rows.Next() {
var wp WatchProgressRow
if err := rows.Scan(&wp.MediaItemID, &wp.PositionSeconds, &wp.DurationSeconds, &wp.Completed, &wp.UpdatedAt); err != nil {
return nil, fmt.Errorf("scan ebook reader progress: %w", err)
}
result = append(result, wp)
}
return result, rows.Err()
}
// RewatchCount holds the number of completed watches for an item.
type RewatchCount struct {
MediaItemID string
@@ -1181,29 +1259,20 @@ func (r *Repo) GetRewatchCounts(ctx context.Context, userID int, profileID strin
// Used for co-watch matrix computation.
func (r *Repo) GetItemWatchers(ctx context.Context, minWatchers int, maxPerUser int) (map[string][]string, error) {
// Get all watch progress entries where progress >= 50% or completed
rows, err := r.pool.Query(ctx, `
WITH user_watches AS (
query := fmt.Sprintf(`
WITH %s,
user_watches AS (
SELECT user_id::text || ':' || COALESCE(profile_id, '') AS watcher_id,
media_item_id,
item_id AS media_item_id,
ROW_NUMBER() OVER (PARTITION BY user_id, profile_id ORDER BY updated_at DESC) AS rn
FROM user_watch_progress
WHERE (completed = true
OR (duration_seconds > 0 AND position_seconds / duration_seconds >= 0.5))
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = user_watch_progress.user_id
AND hhi.profile_id = user_watch_progress.profile_id
AND hhi.media_item_id = user_watch_progress.media_item_id
AND user_watch_progress.updated_at <= hhi.hidden_before
)
FROM watched_activity
)
SELECT media_item_id, ARRAY_AGG(watcher_id) AS watchers
FROM user_watches
WHERE rn <= $1
GROUP BY media_item_id
HAVING COUNT(*) >= $2`,
maxPerUser, minWatchers)
HAVING COUNT(*) >= $2`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, maxPerUser, minWatchers)
if err != nil {
return nil, fmt.Errorf("get item watchers: %w", err)
}
@@ -1226,30 +1295,20 @@ func (r *Repo) GetItemWatchers(ctx context.Context, minWatchers int, maxPerUser
// GetPopularItems returns the most-watched series/movies over the given number of days.
// Episodes are resolved to their parent series.
func (r *Repo) GetPopularItems(ctx context.Context, days, limit int) ([]ScoredItem, error) {
rows, err := r.pool.Query(ctx, `
WITH watched_items AS (
SELECT COALESCE(e.series_id, wp.media_item_id) AS item_id,
wp.user_id::text || ':' || COALESCE(wp.profile_id, '') AS watcher_id
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND wp.updated_at > NOW() - ($1 || ' days')::interval
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = wp.user_id
AND hhi.profile_id = wp.profile_id
AND hhi.media_item_id = wp.media_item_id
AND wp.updated_at <= hhi.hidden_before
)
query := fmt.Sprintf(`
WITH %s,
watched_items AS (
SELECT item_id, watcher_id
FROM watched_activity
WHERE updated_at > NOW() - ($1 || ' days')::interval
)
SELECT wi.item_id, COUNT(DISTINCT wi.watcher_id) AS watch_count
FROM watched_items wi
JOIN media_items mi ON mi.content_id = wi.item_id
GROUP BY wi.item_id
ORDER BY watch_count DESC
LIMIT $2`,
fmt.Sprintf("%d", days), limit)
LIMIT $2`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, fmt.Sprintf("%d", days), limit)
if err != nil {
return nil, fmt.Errorf("get popular items: %w", err)
}
@@ -1271,14 +1330,14 @@ func (r *Repo) GetPopularItems(ctx context.Context, days, limit int) ([]ScoredIt
// GetRecentlyAddedItems returns items added within the given number of days.
func (r *Repo) GetRecentlyAddedItems(ctx context.Context, days, limit int) ([]ScoredItem, error) {
rows, err := r.pool.Query(ctx, `
SELECT content_id, created_at
FROM media_items
WHERE status = 'matched'
AND created_at > NOW() - ($1 || ' days')::interval
ORDER BY created_at DESC
LIMIT $2`,
fmt.Sprintf("%d", days), limit)
query := fmt.Sprintf(`
SELECT mi.content_id, mi.created_at
FROM media_items mi
WHERE %s
AND mi.created_at > NOW() - ($1 || ' days')::interval
ORDER BY mi.created_at DESC
LIMIT $2`, recommendationItemEligibilityWhereClause("mi"))
rows, err := r.pool.Query(ctx, query, fmt.Sprintf("%d", days), limit)
if err != nil {
return nil, fmt.Errorf("get recently added: %w", err)
}
@@ -1352,21 +1411,11 @@ func (r *Repo) GetTasteSeedCandidates(ctx context.Context, limit, offset int) ([
// GetTopGenres returns the most popular genres by watch count.
// Episodes are resolved to their parent series for genre lookup.
func (r *Repo) GetTopGenres(ctx context.Context, limit int) ([]string, error) {
rows, err := r.pool.Query(ctx, `
WITH watched_items AS (
SELECT COALESCE(e.series_id, wp.media_item_id) AS item_id,
wp.user_id::text || ':' || COALESCE(wp.profile_id, '') AS watcher_id
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = wp.user_id
AND hhi.profile_id = wp.profile_id
AND hhi.media_item_id = wp.media_item_id
AND wp.updated_at <= hhi.hidden_before
)
query := fmt.Sprintf(`
WITH %s,
watched_items AS (
SELECT item_id, watcher_id
FROM watched_activity
)
SELECT g.genre, COUNT(DISTINCT wi.watcher_id) AS watchers
FROM watched_items wi
@@ -1374,8 +1423,8 @@ func (r *Repo) GetTopGenres(ctx context.Context, limit int) ([]string, error) {
CROSS JOIN LATERAL UNNEST(mi.genres) AS g(genre)
GROUP BY g.genre
ORDER BY watchers DESC
LIMIT $1`,
limit)
LIMIT $1`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, limit)
if err != nil {
return nil, fmt.Errorf("get top genres: %w", err)
}
@@ -1395,21 +1444,11 @@ func (r *Repo) GetTopGenres(ctx context.Context, limit int) ([]string, error) {
// GetGenreSamplerItems returns the most-watched series/movies in a specific genre.
func (r *Repo) GetGenreSamplerItems(ctx context.Context, genre string, limit int) ([]ScoredItem, error) {
rows, err := r.pool.Query(ctx, `
WITH watched_items AS (
SELECT COALESCE(e.series_id, wp.media_item_id) AS item_id,
wp.user_id::text || ':' || COALESCE(wp.profile_id, '') AS watcher_id
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = wp.user_id
AND hhi.profile_id = wp.profile_id
AND hhi.media_item_id = wp.media_item_id
AND wp.updated_at <= hhi.hidden_before
)
query := fmt.Sprintf(`
WITH %s,
watched_items AS (
SELECT item_id, watcher_id
FROM watched_activity
)
SELECT wi.item_id, COUNT(DISTINCT wi.watcher_id) AS watch_count
FROM watched_items wi
@@ -1417,8 +1456,8 @@ func (r *Repo) GetGenreSamplerItems(ctx context.Context, genre string, limit int
WHERE $1 = ANY(mi.genres)
GROUP BY wi.item_id
ORDER BY watch_count DESC
LIMIT $2`,
genre, limit)
LIMIT $2`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, genre, limit)
if err != nil {
return nil, fmt.Errorf("get genre sampler items: %w", err)
}
@@ -1617,21 +1656,12 @@ func (r *Repo) GetAllUsersWithTasteProfiles(ctx context.Context) ([]StaleProfile
// GetWatchedItemIDs returns content IDs of series/movies the user has watched
// (>= 50% progress or completed). Episodes are resolved to their parent series.
func (r *Repo) GetWatchedItemIDs(ctx context.Context, userID int, profileID string) ([]string, error) {
rows, err := r.pool.Query(ctx, `
SELECT DISTINCT COALESCE(e.series_id, wp.media_item_id) AS item_id
FROM user_watch_progress wp
LEFT JOIN episodes e ON e.content_id = wp.media_item_id
WHERE wp.user_id = $1 AND wp.profile_id = $2
AND (wp.completed = true OR (wp.duration_seconds > 0 AND wp.position_seconds / wp.duration_seconds >= 0.5))
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = wp.user_id
AND hhi.profile_id = wp.profile_id
AND hhi.media_item_id = wp.media_item_id
AND wp.updated_at <= hhi.hidden_before
)`,
userID, profileID)
query := fmt.Sprintf(`
WITH %s
SELECT DISTINCT item_id
FROM watched_activity
WHERE user_id = $1 AND profile_id = $2`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, userID, profileID)
if err != nil {
return nil, fmt.Errorf("get watched item IDs: %w", err)
}
@@ -1695,21 +1725,15 @@ func (r *Repo) GetRecentCompletedItemIDs(ctx context.Context, userID int, profil
return []string{}, nil
}
rows, err := r.pool.Query(ctx, `
SELECT media_item_id
FROM user_watch_progress
query := fmt.Sprintf(`
WITH %s
SELECT leaf_item_id
FROM watched_activity
WHERE user_id = $1 AND profile_id = $2 AND completed = true
AND NOT EXISTS (
SELECT 1
FROM user_history_hidden_items hhi
WHERE hhi.user_id = user_watch_progress.user_id
AND hhi.profile_id = user_watch_progress.profile_id
AND hhi.media_item_id = user_watch_progress.media_item_id
AND user_watch_progress.updated_at <= hhi.hidden_before
)
ORDER BY updated_at DESC, media_item_id ASC
ORDER BY updated_at DESC, leaf_item_id ASC
LIMIT $3
`, userID, profileID, limit)
`, watchedActivityCTE)
rows, err := r.pool.Query(ctx, query, userID, profileID, limit)
if err != nil {
return nil, fmt.Errorf("get recent completed item IDs: %w", err)
}
+61
View File
@@ -24,6 +24,67 @@ func TestTasteSeedCandidateQueryOrdersByReliableColdStartSignals(t *testing.T) {
}
}
func TestWatchedActivityCTEIncludesEbookReaderProgress(t *testing.T) {
query := strings.Join(strings.Fields(watchedActivityCTE), " ")
for _, term := range []string{
"FROM ebook_reader_progress erp",
"erp.progress >= 0.5",
"erp.progress >= 0.9 AS completed",
"hhi.media_item_id = erp.content_id",
} {
if !strings.Contains(query, term) {
t.Fatalf("watched activity CTE missing %q: %s", term, query)
}
}
}
func TestEmbeddingEligibilityWhereClauseIncludesBookTypes(t *testing.T) {
clause := embeddingEligibilityWhereClause()
for _, term := range []string{"mi.status = 'matched'", "mi.type = 'audiobook'", "mi.type = 'ebook'"} {
if !strings.Contains(clause, term) {
t.Fatalf("embedding eligibility clause missing %q: %s", term, clause)
}
}
}
func TestEmbeddingEligibilityWhereClauseUsesMediaItemsAlias(t *testing.T) {
clause := embeddingEligibilityWhereClause()
if strings.Contains(clause, " type =") || strings.Contains(clause, " status =") {
t.Fatalf("embedding eligibility clause should qualify media_items columns: %s", clause)
}
}
func TestRecommendationItemEligibilityWhereClauseIncludesBookTypes(t *testing.T) {
clause := recommendationItemEligibilityWhereClause("mi")
for _, term := range []string{"mi.status = 'matched'", "mi.type = 'audiobook'", "mi.type = 'ebook'"} {
if !strings.Contains(clause, term) {
t.Fatalf("recommendation item eligibility clause missing %q: %s", term, clause)
}
}
}
func TestRecommendationItemEligibilityWhereClauseDefaultsAlias(t *testing.T) {
clause := recommendationItemEligibilityWhereClause("")
if !strings.Contains(clause, "media_items.status = 'matched'") {
t.Fatalf("default eligibility alias missing media_items qualification: %s", clause)
}
}
func TestTasteProfileRefreshSubjectsQueryIncludesEbookReaderProgress(t *testing.T) {
query := strings.Join(strings.Fields(tasteProfileRefreshSubjectsQuery), " ")
assertQueryTermsInOrder(t, query,
"SELECT DISTINCT user_id, profile_id FROM user_watch_progress",
"UNION",
"SELECT DISTINCT user_id, profile_id FROM ebook_reader_progress",
)
}
func assertQueryTermsInOrder(t *testing.T, query string, terms ...string) {
t.Helper()
+43 -10
View File
@@ -14,6 +14,7 @@ const signalPageSize = 1000
type signalRepo interface {
GetWatchedItemIDSet(ctx context.Context, userID int, profileID string) (map[string]struct{}, error)
GetWatchProgressForUser(ctx context.Context, userID int, profileID string) ([]WatchProgressRow, error)
GetEbookReaderProgressForUser(ctx context.Context, userID int, profileID string) ([]WatchProgressRow, error)
GetRecentCompletedItemIDs(ctx context.Context, userID int, profileID string, limit int) ([]string, error)
GetRewatchCounts(ctx context.Context, userID int, profileID string) ([]RewatchCount, error)
ResolveCanonicalItemIDSet(ctx context.Context, contentIDs []string) (map[string]struct{}, error)
@@ -69,6 +70,15 @@ func (s *SignalReader) WatchedItemIDSet(ctx context.Context, userID int, profile
}); err != nil {
return nil, err
}
ebookProgress, err := s.repo.GetEbookReaderProgressForUser(ctx, userID, profileID)
if err != nil {
return nil, err
}
for _, wp := range ebookProgress {
if wp.Completed || watchedProgressThresholdMet(wp.PositionSeconds, wp.DurationSeconds) {
rawIDs = append(rawIDs, wp.MediaItemID)
}
}
return s.repo.ResolveCanonicalItemIDSet(ctx, rawIDs)
}
@@ -97,6 +107,11 @@ func (s *SignalReader) WatchProgressForUser(ctx context.Context, userID int, pro
}); err != nil {
return nil, err
}
ebookProgress, err := s.repo.GetEbookReaderProgressForUser(ctx, userID, profileID)
if err != nil {
return nil, err
}
rows = append(rows, ebookProgress...)
return rows, nil
}
@@ -118,20 +133,38 @@ func (s *SignalReader) RecentCompletedItemIDs(ctx context.Context, userID int, p
if err != nil {
return nil, fmt.Errorf("list completed progress from store: %w", err)
}
sort.SliceStable(progress, func(i, j int) bool {
left := parseSignalTime(progress[i].UpdatedAt, time.Time{})
right := parseSignalTime(progress[j].UpdatedAt, time.Time{})
if !left.Equal(right) {
return left.After(right)
}
return progress[i].MediaItemID < progress[j].MediaItemID
})
ids := make([]string, 0, min(limit, len(progress)))
completed := make([]WatchProgressRow, 0, len(progress))
for _, wp := range progress {
if !wp.Completed {
continue
}
completed = append(completed, WatchProgressRow{
MediaItemID: wp.MediaItemID,
Completed: true,
UpdatedAt: parseSignalTime(wp.UpdatedAt, time.Time{}),
})
}
ebookProgress, err := s.repo.GetEbookReaderProgressForUser(ctx, userID, profileID)
if err != nil {
return nil, err
}
for _, wp := range ebookProgress {
if !wp.Completed {
continue
}
completed = append(completed, wp)
}
sort.SliceStable(completed, func(i, j int) bool {
left := completed[i].UpdatedAt
right := completed[j].UpdatedAt
if !left.Equal(right) {
return left.After(right)
}
return completed[i].MediaItemID < completed[j].MediaItemID
})
ids := make([]string, 0, min(limit, len(completed)))
for _, wp := range completed {
ids = append(ids, wp.MediaItemID)
if len(ids) == limit {
break
+74
View File
@@ -14,6 +14,7 @@ type fakeSignalRepo struct {
fallbackWatched map[string]struct{}
fallbackProgress []WatchProgressRow
ebookProgress []WatchProgressRow
fallbackRecentCompleted []string
fallbackRewatches []RewatchCount
}
@@ -26,6 +27,10 @@ func (r *fakeSignalRepo) GetWatchProgressForUser(context.Context, int, string) (
return r.fallbackProgress, nil
}
func (r *fakeSignalRepo) GetEbookReaderProgressForUser(context.Context, int, string) ([]WatchProgressRow, error) {
return r.ebookProgress, nil
}
func (r *fakeSignalRepo) GetRecentCompletedItemIDs(context.Context, int, string, int) ([]string, error) {
return r.fallbackRecentCompleted, nil
}
@@ -161,6 +166,52 @@ func TestSignalReaderWatchedSetCanonicalizesStoreProgress(t *testing.T) {
}
}
func TestSignalReaderWatchedSetIncludesEbookReaderProgress(t *testing.T) {
store := &fakeSignalStore{}
repo := &fakeSignalRepo{
ebookProgress: []WatchProgressRow{
{MediaItemID: "ebook-half", PositionSeconds: 0.6, DurationSeconds: 1, UpdatedAt: time.Date(2026, 6, 1, 10, 0, 0, 0, time.UTC)},
{MediaItemID: "ebook-low", PositionSeconds: 0.2, DurationSeconds: 1, UpdatedAt: time.Date(2026, 6, 1, 11, 0, 0, 0, time.UTC)},
},
}
reader := NewSignalReader(repo, fakeSignalProvider{store: store})
watched, err := reader.WatchedItemIDSet(context.Background(), 7, "p1")
if err != nil {
t.Fatalf("WatchedItemIDSet returned error: %v", err)
}
if _, ok := watched["ebook-half"]; !ok {
t.Fatalf("expected ebook-half in watched set, got %#v", watched)
}
if _, ok := watched["ebook-low"]; ok {
t.Fatalf("did not expect low-progress ebook in watched set: %#v", watched)
}
}
func TestSignalReaderWatchProgressIncludesEbookReaderProgress(t *testing.T) {
store := &fakeSignalStore{progress: []userstore.WatchProgress{
{ProfileID: "p1", MediaItemID: "movie", PositionSeconds: 60, DurationSeconds: 100, UpdatedAt: "2026-06-01T10:00:00Z"},
}}
repo := &fakeSignalRepo{
ebookProgress: []WatchProgressRow{
{MediaItemID: "ebook", PositionSeconds: 0.42, DurationSeconds: 1, Completed: false, UpdatedAt: time.Date(2026, 6, 1, 11, 0, 0, 0, time.UTC)},
},
}
reader := NewSignalReader(repo, fakeSignalProvider{store: store})
progress, err := reader.WatchProgressForUser(context.Background(), 7, "p1")
if err != nil {
t.Fatalf("WatchProgressForUser returned error: %v", err)
}
if !slices.ContainsFunc(progress, func(row WatchProgressRow) bool {
return row.MediaItemID == "ebook" && row.PositionSeconds == 0.42 && row.DurationSeconds == 1
}) {
t.Fatalf("ebook progress missing from rows: %#v", progress)
}
}
func TestSignalReaderRecentCompletedUsesStoreUpdatedOrder(t *testing.T) {
store := &fakeSignalStore{progress: []userstore.WatchProgress{
{ProfileID: "p1", MediaItemID: "older", Completed: true, UpdatedAt: "2026-05-01T10:00:00Z"},
@@ -181,6 +232,29 @@ func TestSignalReaderRecentCompletedUsesStoreUpdatedOrder(t *testing.T) {
}
}
func TestSignalReaderRecentCompletedIncludesEbookReaderProgress(t *testing.T) {
store := &fakeSignalStore{progress: []userstore.WatchProgress{
{ProfileID: "p1", MediaItemID: "movie", Completed: true, UpdatedAt: "2026-06-01T10:00:00Z"},
}}
repo := &fakeSignalRepo{
ebookProgress: []WatchProgressRow{
{MediaItemID: "ebook-done", PositionSeconds: 0.95, DurationSeconds: 1, Completed: true, UpdatedAt: time.Date(2026, 6, 3, 10, 0, 0, 0, time.UTC)},
{MediaItemID: "ebook-open", PositionSeconds: 0.4, DurationSeconds: 1, Completed: false, UpdatedAt: time.Date(2026, 6, 4, 10, 0, 0, 0, time.UTC)},
},
}
reader := NewSignalReader(repo, fakeSignalProvider{store: store})
ids, err := reader.RecentCompletedItemIDs(context.Background(), 7, "p1", 3)
if err != nil {
t.Fatalf("RecentCompletedItemIDs returned error: %v", err)
}
want := []string{"ebook-done", "movie"}
if !slices.Equal(ids, want) {
t.Fatalf("recent completed = %#v, want %#v", ids, want)
}
}
func TestSignalReaderRewatchCountsAggregatesCompletedHistory(t *testing.T) {
store := &fakeSignalStore{history: []userstore.WatchHistoryEntry{
{ProfileID: "p1", MediaItemID: "rewatched", Completed: true, WatchedAt: "2026-05-01T10:00:00Z"},
+14 -10
View File
@@ -31,6 +31,19 @@ type Worker struct {
cancelFunc context.CancelFunc
}
const tasteProfileRefreshSubjectsQuery = `
SELECT DISTINCT user_id, profile_id FROM user_ratings
UNION
SELECT DISTINCT user_id, profile_id FROM user_taste_profiles
UNION
SELECT DISTINCT user_id, profile_id FROM user_watch_progress
UNION
SELECT DISTINCT user_id, profile_id FROM ebook_reader_progress
UNION
SELECT DISTINCT user_id, profile_id FROM user_favorites
UNION
SELECT DISTINCT user_id, profile_id FROM user_watchlist`
// NewWorker creates a new recommendation Worker.
func NewWorker(engine *Engine, embeddingsCron, tasteProfilesCron, cowatchCron, recommendationsCron string) (*Worker, error) {
w := &Worker{
@@ -266,16 +279,7 @@ func (w *Worker) doTasteProfiles() {
defer cancel()
slog.Info("starting taste profile refresh")
rows, err := w.engine.pool.Query(ctx, `
SELECT DISTINCT user_id, profile_id FROM user_ratings
UNION
SELECT DISTINCT user_id, profile_id FROM user_taste_profiles
UNION
SELECT DISTINCT user_id, profile_id FROM user_watch_progress
UNION
SELECT DISTINCT user_id, profile_id FROM user_favorites
UNION
SELECT DISTINCT user_id, profile_id FROM user_watchlist`)
rows, err := w.engine.pool.Query(ctx, tasteProfileRefreshSubjectsQuery)
if err != nil {
slog.Error("taste profile query failed", "error", err)
return
+4
View File
@@ -17,6 +17,10 @@ type audiobookCoverCacher interface {
CacheAudiobookCover(ctx context.Context, data []byte, contentID string) (basePath string, ext string, thumbhash string, err error)
}
type ebookCoverCacher interface {
CacheEbookCover(ctx context.Context, data []byte, contentID string) (basePath string, ext string, thumbhash string, err error)
}
// FFmpegPathFromFFprobe derives the ffmpeg binary path from a configured
// ffprobe path. They live side by side in every silo deployment.
func FFmpegPathFromFFprobe(ffprobePath string) string {
+679
View File
@@ -0,0 +1,679 @@
package scanner
import (
"archive/zip"
"bytes"
"encoding/xml"
"fmt"
"html"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"time"
xhtml "golang.org/x/net/html"
)
const maxEPUBMetadataEntrySize = 8 * 1024 * 1024
const maxPDFMetadataScanSize = 2 * 1024 * 1024
var ebookExtensions = map[string]bool{
".epub": true,
".pdf": true,
".mobi": true,
".azw": true,
".azw3": true,
".fb2": true,
".fbz": true,
".cbz": true,
".cbr": true,
".md": true,
}
type parsedEbook struct {
Format string
Title string
Authors []string
Description string
Publisher string
PublishedAt time.Time
Year int
Language string
ISBN string
Series string
SeriesIndex string
Genres []string
PageCount int
Cover *parsedEbookCover
}
type parsedEbookCover struct {
ContentType string
Bytes []byte
}
func SupportsEbookFile(filePath string) bool {
if strings.HasSuffix(strings.ToLower(filePath), ".fb2.zip") {
return true
}
return ebookExtensions[strings.ToLower(filepath.Ext(filePath))]
}
func parseEbookFile(path string) (book parsedEbook, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("panic parsing ebook %s: %v", path, r)
}
}()
format := ebookFileFormat(path)
switch format {
case ".epub":
book, err = parseEbookEPUB(path)
case ".fb2":
book, err = parseEbookFB2(path)
case ".fbz":
book, err = parseEbookFBZ(path)
case ".cbz":
book, err = parseEbookCBZ(path)
case ".pdf":
book, err = parseEbookPDF(path)
case ".mobi", ".azw", ".azw3", ".cbr", ".md":
book = parsedEbook{Format: strings.TrimPrefix(format, ".")}
default:
err = fmt.Errorf("unsupported ebook format: %s", filepath.Ext(path))
}
book.sanitize()
return book, err
}
func parseEbookPDF(path string) (parsedEbook, error) {
book := parsedEbook{Format: "pdf"}
file, err := os.Open(path)
if err != nil {
return book, err
}
defer file.Close()
data, err := io.ReadAll(io.LimitReader(file, maxPDFMetadataScanSize))
if err != nil {
return book, err
}
info := parsePDFInfoFields(data)
book.Title = info["Title"]
book.Authors = splitEbookAuthors(info["Author"])
book.Description = info["Subject"]
book.Genres = splitPDFKeywords(info["Keywords"])
for _, value := range []string{
info["ISBN"],
info["Subject"],
info["Keywords"],
info["Title"],
} {
if isbn := normalizeEbookISBN(value); isbn != "" {
book.ISBN = isbn
break
}
}
if t, ok := parsePDFDate(info["CreationDate"]); ok {
book.PublishedAt = t
book.Year = t.Year()
}
return book, nil
}
func ebookFileFormat(path string) string {
if strings.HasSuffix(strings.ToLower(path), ".fb2.zip") {
return ".fbz"
}
return strings.ToLower(filepath.Ext(path))
}
func (b *parsedEbook) sanitize() {
b.Format = strings.TrimPrefix(strings.ToLower(strings.TrimSpace(b.Format)), ".")
b.Title = strings.TrimSpace(b.Title)
b.Description = cleanEbookDescription(b.Description)
b.Publisher = strings.TrimSpace(b.Publisher)
b.Language = strings.TrimSpace(b.Language)
b.ISBN = normalizeEbookISBN(b.ISBN)
b.Series = strings.TrimSpace(b.Series)
b.SeriesIndex = strings.TrimSpace(b.SeriesIndex)
b.Authors = uniqueTrimmedStrings(b.Authors)
b.Genres = uniqueTrimmedStrings(b.Genres)
if b.PageCount < 0 {
b.PageCount = 0
}
if b.Year == 0 && !b.PublishedAt.IsZero() {
b.Year = b.PublishedAt.Year()
}
}
func cleanEbookDescription(value string) string {
value = strings.TrimSpace(html.UnescapeString(value))
if value == "" {
return ""
}
if !strings.Contains(value, "<") || !strings.Contains(value, ">") {
return strings.Join(strings.Fields(value), " ")
}
tokenizer := xhtml.NewTokenizer(strings.NewReader(value))
var out strings.Builder
needsSpace := false
writeSpace := func() {
if out.Len() > 0 && !needsSpace {
needsSpace = true
}
}
for {
switch tokenizer.Next() {
case xhtml.ErrorToken:
return strings.Join(strings.Fields(out.String()), " ")
case xhtml.TextToken:
text := strings.TrimSpace(html.UnescapeString(string(tokenizer.Text())))
if text == "" {
continue
}
if out.Len() > 0 && (needsSpace || !startsWithClosingPunctuation(text)) {
out.WriteByte(' ')
}
out.WriteString(text)
needsSpace = false
case xhtml.StartTagToken, xhtml.SelfClosingTagToken, xhtml.EndTagToken:
name, _ := tokenizer.TagName()
switch strings.ToLower(string(name)) {
case "br", "p", "div", "section", "article", "li", "ul", "ol", "blockquote", "tr":
writeSpace()
}
}
}
}
func startsWithClosingPunctuation(value string) bool {
for _, r := range value {
switch r {
case '.', ',', ';', ':', '!', '?', ')', ']', '}':
return true
default:
return false
}
}
return false
}
func looksLikeHTML(value string) bool {
value = strings.TrimSpace(value)
return strings.Contains(value, "<") && strings.Contains(value, ">")
}
func uniqueTrimmedStrings(values []string) []string {
seen := map[string]struct{}{}
var out []string
for _, value := range values {
trimmed := strings.TrimSpace(value)
key := strings.ToLower(trimmed)
if key == "" {
continue
}
if _, ok := seen[key]; ok {
continue
}
seen[key] = struct{}{}
out = append(out, trimmed)
}
return out
}
func normalizeEbookISBN(value string) string {
value = strings.ToUpper(strings.TrimSpace(value))
for _, prefix := range []string{"ISBN-13", "ISBN-10", "ISBN"} {
if strings.HasPrefix(value, prefix) {
value = strings.TrimSpace(strings.TrimPrefix(value, prefix))
value = strings.TrimLeft(value, ": ")
break
}
}
var out strings.Builder
for _, r := range value {
if r >= '0' && r <= '9' {
out.WriteRune(r)
continue
}
if r == 'X' {
out.WriteRune(r)
}
}
candidate := out.String()
switch len(candidate) {
case 10:
if validISBN10(candidate) {
return candidate
}
case 13:
if validISBN13(candidate) {
return candidate
}
}
return ""
}
func validISBN10(candidate string) bool {
if len(candidate) != 10 {
return false
}
sum := 0
for i, r := range candidate {
value := 0
switch {
case r >= '0' && r <= '9':
value = int(r - '0')
case r == 'X' && i == 9:
value = 10
default:
return false
}
sum += value * (10 - i)
}
return sum%11 == 0
}
func validISBN13(candidate string) bool {
if len(candidate) != 13 {
return false
}
sum := 0
for i, r := range candidate {
if r < '0' || r > '9' {
return false
}
value := int(r - '0')
if i%2 == 1 {
value *= 3
}
sum += value
}
return sum%10 == 0
}
func parseEbookEPUB(path string) (parsedEbook, error) {
book := parsedEbook{Format: "epub"}
reader, err := zip.OpenReader(path)
if err != nil {
return book, err
}
defer reader.Close()
container, err := readEPUBZipEntry(&reader.Reader, "META-INF/container.xml")
if err != nil {
return book, err
}
opfPath, err := epubOPFPath(container)
if err != nil {
return book, err
}
opf, err := readEPUBZipEntry(&reader.Reader, opfPath)
if err != nil {
return book, err
}
if err := parseEPUBOPFMetadata(opf, &book); err != nil {
return book, err
}
return book, nil
}
func parseEbookFB2(path string) (parsedEbook, error) {
file, err := os.Open(path)
if err != nil {
return parsedEbook{Format: "fb2"}, err
}
defer file.Close()
return parseEbookFB2Reader(file, "fb2")
}
func parseEbookFBZ(path string) (parsedEbook, error) {
book := parsedEbook{Format: "fbz"}
reader, err := zip.OpenReader(path)
if err != nil {
return book, err
}
defer reader.Close()
for _, file := range reader.File {
if !strings.HasSuffix(strings.ToLower(file.Name), ".fb2") {
continue
}
if file.UncompressedSize64 > maxEPUBMetadataEntrySize {
return book, fmt.Errorf("fbz entry too large: %s", file.Name)
}
entry, err := file.Open()
if err != nil {
return book, err
}
defer entry.Close()
return parseEbookFB2Reader(io.LimitReader(entry, maxEPUBMetadataEntrySize+1), "fbz")
}
return book, fmt.Errorf("fbz archive has no fb2 entry")
}
func parseEbookCBZ(path string) (parsedEbook, error) {
book := parsedEbook{Format: "cbz"}
reader, err := zip.OpenReader(path)
if err != nil {
return book, err
}
defer reader.Close()
for _, file := range reader.File {
if isComicArchivePage(file.Name) {
book.PageCount++
}
}
return book, nil
}
func isComicArchivePage(name string) bool {
clean := strings.TrimSpace(strings.ReplaceAll(name, "\\", "/"))
if clean == "" || strings.HasSuffix(clean, "/") {
return false
}
base := strings.ToLower(filepath.Base(clean))
if strings.HasPrefix(base, "._") {
return false
}
parts := strings.Split(strings.ToLower(clean), "/")
for _, part := range parts {
if part == "__macosx" {
return false
}
}
switch filepath.Ext(base) {
case ".jpg", ".jpeg", ".png", ".webp", ".gif", ".avif", ".bmp":
return true
default:
return false
}
}
func parseEbookFB2Reader(reader io.Reader, format string) (parsedEbook, error) {
book := parsedEbook{Format: format}
var fb2 struct {
Description struct {
TitleInfo struct {
Genres []string `xml:"genre"`
Authors []struct {
FirstName string `xml:"first-name"`
MiddleName string `xml:"middle-name"`
LastName string `xml:"last-name"`
Nickname string `xml:"nickname"`
} `xml:"author"`
BookTitle string `xml:"book-title"`
Lang string `xml:"lang"`
Date struct {
Value string `xml:"value,attr"`
Text string `xml:",chardata"`
} `xml:"date"`
Sequences []struct {
Name string `xml:"name,attr"`
Number string `xml:"number,attr"`
} `xml:"sequence"`
} `xml:"title-info"`
PublishInfo struct {
ISBN string `xml:"isbn"`
Publisher string `xml:"publisher"`
Year string `xml:"year"`
} `xml:"publish-info"`
} `xml:"description"`
}
if err := xml.NewDecoder(reader).Decode(&fb2); err != nil {
return book, err
}
book.Title = fb2.Description.TitleInfo.BookTitle
book.Language = fb2.Description.TitleInfo.Lang
book.Genres = fb2.Description.TitleInfo.Genres
if t, ok := parseEbookDate(firstNonEmpty(fb2.Description.TitleInfo.Date.Value, fb2.Description.TitleInfo.Date.Text)); ok {
book.PublishedAt = t
}
for _, author := range fb2.Description.TitleInfo.Authors {
name := strings.Join(uniqueTrimmedStrings([]string{
author.FirstName,
author.MiddleName,
author.LastName,
}), " ")
if name == "" {
name = author.Nickname
}
book.Authors = append(book.Authors, name)
}
if len(fb2.Description.TitleInfo.Sequences) > 0 {
book.Series = fb2.Description.TitleInfo.Sequences[0].Name
book.SeriesIndex = fb2.Description.TitleInfo.Sequences[0].Number
}
book.ISBN = fb2.Description.PublishInfo.ISBN
book.Publisher = fb2.Description.PublishInfo.Publisher
if year, err := strconv.Atoi(strings.TrimSpace(fb2.Description.PublishInfo.Year)); err == nil {
book.Year = year
}
return book, nil
}
func parsePDFInfoFields(data []byte) map[string]string {
fields := map[string]string{}
for _, key := range []string{"Title", "Author", "Subject", "Keywords", "CreationDate", "ISBN"} {
token := []byte("/" + key)
idx := bytes.Index(data, token)
if idx < 0 {
continue
}
rest := bytes.TrimLeft(data[idx+len(token):], " \t\r\n")
if len(rest) == 0 || rest[0] != '(' {
continue
}
value, ok := readPDFLiteralString(rest)
if ok {
fields[key] = value
}
}
return fields
}
func readPDFLiteralString(data []byte) (string, bool) {
if len(data) == 0 || data[0] != '(' {
return "", false
}
var out strings.Builder
depth := 1
escaped := false
for _, b := range data[1:] {
if escaped {
switch b {
case 'n':
out.WriteByte('\n')
case 'r':
out.WriteByte('\r')
case 't':
out.WriteByte('\t')
case 'b':
out.WriteByte('\b')
case 'f':
out.WriteByte('\f')
default:
out.WriteByte(b)
}
escaped = false
continue
}
switch b {
case '\\':
escaped = true
case '(':
depth++
out.WriteByte(b)
case ')':
depth--
if depth == 0 {
return strings.TrimSpace(out.String()), true
}
out.WriteByte(b)
default:
out.WriteByte(b)
}
}
return "", false
}
func splitEbookAuthors(value string) []string {
value = strings.TrimSpace(value)
if value == "" {
return nil
}
parts := strings.FieldsFunc(value, func(r rune) bool {
return r == ';' || r == '|'
})
if len(parts) == 1 {
parts = strings.Split(value, " and ")
}
return uniqueTrimmedStrings(parts)
}
func splitPDFKeywords(value string) []string {
if strings.TrimSpace(value) == "" {
return nil
}
return uniqueTrimmedStrings(strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == ';'
}))
}
func parsePDFDate(value string) (time.Time, bool) {
value = strings.TrimSpace(value)
if strings.HasPrefix(value, "D:") {
value = strings.TrimPrefix(value, "D:")
}
value = strings.TrimSuffix(value, "Z")
if len(value) >= 14 {
if t, err := time.Parse("20060102150405", value[:14]); err == nil {
return t, true
}
}
if len(value) >= 8 {
if t, err := time.Parse("20060102", value[:8]); err == nil {
return t, true
}
}
if len(value) >= 4 {
if t, err := time.Parse("2006", value[:4]); err == nil {
return t, true
}
}
return time.Time{}, false
}
func readEPUBZipEntry(reader *zip.Reader, name string) ([]byte, error) {
for _, file := range reader.File {
if file.Name != name {
continue
}
entry, err := file.Open()
if err != nil {
return nil, err
}
defer entry.Close()
limited := io.LimitReader(entry, maxEPUBMetadataEntrySize+1)
data, err := io.ReadAll(limited)
if err != nil {
return nil, err
}
if len(data) > maxEPUBMetadataEntrySize {
return nil, fmt.Errorf("epub entry too large: %s", name)
}
return data, nil
}
return nil, fmt.Errorf("epub entry not found: %s", name)
}
func epubOPFPath(container []byte) (string, error) {
var parsed struct {
Rootfiles []struct {
FullPath string `xml:"full-path,attr"`
} `xml:"rootfiles>rootfile"`
}
if err := xml.Unmarshal(container, &parsed); err != nil {
return "", err
}
for _, rootfile := range parsed.Rootfiles {
if strings.TrimSpace(rootfile.FullPath) != "" {
return rootfile.FullPath, nil
}
}
return "", fmt.Errorf("epub container has no rootfile")
}
func parseEPUBOPFMetadata(opf []byte, book *parsedEbook) error {
var parsed struct {
Metadata struct {
Titles []string `xml:"title"`
Creators []string `xml:"creator"`
Identifiers []string `xml:"identifier"`
Publisher string `xml:"publisher"`
Dates []string `xml:"date"`
Language string `xml:"language"`
Subjects []string `xml:"subject"`
Descriptions []string `xml:"description"`
Meta []struct {
Name string `xml:"name,attr"`
Property string `xml:"property,attr"`
Content string `xml:"content,attr"`
Value string `xml:",chardata"`
} `xml:"meta"`
} `xml:"metadata"`
}
if err := xml.Unmarshal(opf, &parsed); err != nil {
return err
}
book.Title = firstNonEmpty(parsed.Metadata.Titles...)
book.Authors = append(book.Authors, parsed.Metadata.Creators...)
book.Publisher = parsed.Metadata.Publisher
book.Language = parsed.Metadata.Language
book.Genres = append(book.Genres, parsed.Metadata.Subjects...)
book.Description = firstNonEmpty(parsed.Metadata.Descriptions...)
for _, identifier := range parsed.Metadata.Identifiers {
if isbn := normalizeEbookISBN(identifier); isbn != "" {
book.ISBN = isbn
break
}
}
for _, date := range parsed.Metadata.Dates {
if t, ok := parseEbookDate(date); ok {
book.PublishedAt = t
book.Year = t.Year()
break
}
}
for _, meta := range parsed.Metadata.Meta {
name := strings.ToLower(strings.TrimSpace(firstNonEmpty(meta.Name, meta.Property)))
value := strings.TrimSpace(firstNonEmpty(meta.Content, meta.Value))
switch name {
case "calibre:series", "belongs-to-collection":
book.Series = value
case "calibre:series_index", "group-position":
book.SeriesIndex = value
case "calibre:isbn", "isbn", "schema:isbn":
if book.ISBN == "" {
book.ISBN = normalizeEbookISBN(value)
}
}
}
return nil
}
func parseEbookDate(value string) (time.Time, bool) {
value = strings.TrimSpace(value)
for _, layout := range []string{"2006-01-02", "2006-01", "2006"} {
if t, err := time.Parse(layout, value); err == nil {
return t, true
}
}
return time.Time{}, false
}
+750
View File
@@ -0,0 +1,750 @@
package scanner
import (
"context"
"errors"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/idgen"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/Silo-Server/silo-server/internal/titleutil"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type ebookSQLExecutor interface {
Exec(ctx context.Context, sql string, arguments ...any) (pgconn.CommandTag, error)
}
type filesystemMediaItemReader interface {
GetByIDs(ctx context.Context, ids []string) ([]*models.MediaItem, error)
}
func ebookScanWorkers() int {
if v := os.Getenv("SILO_EBOOK_SCAN_WORKERS"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
return n
}
}
return audiobookScanWorkers()
}
func (s *Scanner) ScanEbookFolder(ctx context.Context, folder *models.MediaFolder) error {
if s == nil || folder == nil {
return fmt.Errorf("ScanEbookFolder: nil scanner or folder")
}
return s.scanEbookPaths(ctx, folder, folder.Paths)
}
func (s *Scanner) scanEbookPaths(ctx context.Context, folder *models.MediaFolder, roots []string) error {
if s == nil || folder == nil {
return fmt.Errorf("scanEbookPaths: nil scanner or folder")
}
var candidates []string
for _, root := range roots {
if err := ctx.Err(); err != nil {
return err
}
walkErr := filepath.WalkDir(root, func(path string, d fs.DirEntry, walkErr error) error {
if err := ctx.Err(); err != nil {
return err
}
if walkErr != nil {
slog.Warn("ebook scan: walk error", "path", path, "error", walkErr)
return nil
}
if d.IsDir() {
return nil
}
if SupportsEbookFile(path) {
candidates = append(candidates, path)
}
return nil
})
if walkErr != nil {
slog.Warn("ebook scan: walk root failed", "root", root, "error", walkErr)
}
}
if len(candidates) == 0 {
return nil
}
workers := ebookScanWorkers()
slog.Info("ebook scan: starting",
"folder_id", folder.ID,
"candidates", len(candidates),
"workers", workers,
)
reportEbookScanProgress(ctx, folder.ID, len(candidates), 0, 0, 0)
ch := make(chan string, workers*2)
var (
wg sync.WaitGroup
processed int64
failed int64
skipped int64
failMu sync.Mutex
failures []error
cancelErr error
)
start := time.Now()
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for path := range ch {
if ctx.Err() != nil {
return
}
if err := s.reconcileEbookFile(ctx, folder, path, &skipped); err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
failMu.Lock()
if cancelErr == nil {
cancelErr = err
}
failMu.Unlock()
return
}
atomic.AddInt64(&failed, 1)
failMu.Lock()
failures = append(failures, fmt.Errorf("%s: %w", path, err))
failMu.Unlock()
slog.Warn("ebook scan: file failed",
"folder_id", folder.ID,
"path", path,
"error", err,
)
}
n := atomic.AddInt64(&processed, 1)
if n%500 == 0 || n == int64(len(candidates)) {
failedCount := atomic.LoadInt64(&failed)
skippedCount := atomic.LoadInt64(&skipped)
slog.Info("ebook scan: progress",
"folder_id", folder.ID,
"processed", n,
"failed", failedCount,
"skipped", skippedCount,
"total", len(candidates),
"elapsed_sec", int(time.Since(start).Seconds()),
)
reportEbookScanProgress(ctx, folder.ID, len(candidates), int(n), int(failedCount), int(skippedCount))
}
}
}()
}
for _, p := range candidates {
select {
case ch <- p:
case <-ctx.Done():
close(ch)
wg.Wait()
return ctx.Err()
}
}
close(ch)
wg.Wait()
if err := ctx.Err(); err != nil {
return err
}
if cancelErr != nil {
return cancelErr
}
slog.Info("ebook scan: completed",
"folder_id", folder.ID,
"processed", atomic.LoadInt64(&processed),
"failed", atomic.LoadInt64(&failed),
"skipped", atomic.LoadInt64(&skipped),
"elapsed_sec", int(time.Since(start).Seconds()),
)
if processedCount := atomic.LoadInt64(&processed); processedCount > 0 {
failedCount := atomic.LoadInt64(&failed)
skippedCount := atomic.LoadInt64(&skipped)
if failedCount > 0 && skippedCount == 0 && failedCount == processedCount {
return fmt.Errorf("ebook scan failed for every attempted folder_id=%d: %w", folder.ID, errors.Join(failures...))
}
}
return nil
}
func reportEbookScanProgress(ctx context.Context, folderID int, total, processed, failed, skipped int) {
reportProgress(ctx, ProgressUpdate{
Phase: "ebook_scan",
Message: fmt.Sprintf("Scanning ebooks in folder %d", folderID),
CurrentScope: strconv.Itoa(folderID),
TotalFiles: total,
FilesDiscovered: total,
FilesProcessed: processed,
Errors: failed,
Unchanged: skipped,
})
}
func (s *Scanner) reconcileEbookFile(ctx context.Context, folder *models.MediaFolder, filePath string, skipped *int64) error {
info, err := os.Stat(filePath)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return fmt.Errorf("stat ebook file %s: %w", filePath, err)
}
size := info.Size()
modifiedAt := normalizeFileModifiedAt(info.ModTime())
isUnchanged, skipErr := s.ebookFileShouldSkip(ctx, folder, filePath, size, modifiedAt)
if skipErr != nil {
slog.Warn("ebook scan: skip-check failed, falling through",
"folder_id", folder.ID,
"path", filePath,
"error", skipErr,
)
} else if isUnchanged {
atomic.AddInt64(skipped, 1)
return nil
}
parsed, err := parseEbookFile(filePath)
if err != nil {
return fmt.Errorf("parse ebook file %s: %w", filePath, err)
}
if parsed.Title == "" {
parsed.Title = strings.TrimSuffix(filepath.Base(filePath), filepath.Ext(filePath))
}
contentID, err := s.upsertEbookMediaItem(ctx, folder.ID, filePath, &parsed)
if err != nil {
return fmt.Errorf("upsert ebook item: %w", err)
}
if err := s.upsertEbookMediaFile(ctx, folder, contentID, filePath, size, modifiedAt, &parsed); err != nil {
return fmt.Errorf("upsert ebook file: %w", err)
}
if err := applyEbookEmbeddedCover(ctx, s.itemRepo, s.imageCacher, contentID, &parsed); err != nil {
slog.Warn("ebook scan: embedded cover upload failed",
"folder_id", folder.ID,
"content_id", contentID,
"path", filePath,
"error", err,
)
}
if err := s.upsertEbookPeople(ctx, contentID, &parsed); err != nil {
return fmt.Errorf("upsert ebook people: %w", err)
}
if err := s.upsertEbookSeries(ctx, contentID, &parsed); err != nil {
return fmt.Errorf("upsert ebook series: %w", err)
}
if err := insertEbookLibraryMembership(ctx, s.fileRepo.Pool(), contentID, folder.ID); err != nil {
return fmt.Errorf("upsert ebook library membership: %w", err)
}
if parsed.ISBN != "" {
if err := insertEbookISBNProviderID(ctx, s.fileRepo.Pool(), contentID, parsed.ISBN); err != nil {
return fmt.Errorf("upsert ebook ISBN provider id: %w", err)
}
}
slog.Info("ebook scan: indexed",
"folder_id", folder.ID,
"content_id", contentID,
"title", parsed.Title,
"authors", parsed.Authors,
"path", filePath,
)
return nil
}
func (s *Scanner) ebookFileShouldSkip(ctx context.Context, folder *models.MediaFolder, filePath string, size int64, modifiedAt time.Time) (bool, error) {
if s.fileRepo == nil || s.itemRepo == nil {
return false, nil
}
existing, err := s.fileRepo.ListByObservedRootPath(ctx, folder.ID, filePath)
if err != nil {
return false, fmt.Errorf("list existing files: %w", err)
}
if len(existing) != 1 {
return false, nil
}
mf := existing[0]
if mf.FilePath != filePath || mf.FileSize != size || mf.FileModifiedAt == nil || !sameFileModifiedAt(mf.FileModifiedAt, modifiedAt) {
return false, nil
}
if mf.ContentID == "" {
return false, nil
}
statuses, err := s.itemRepo.GetStatusByIDs(ctx, []string{mf.ContentID})
if err != nil {
return false, fmt.Errorf("get item status: %w", err)
}
return !strings.EqualFold(strings.TrimSpace(statuses[mf.ContentID]), "unmatched"), nil
}
func (s *Scanner) upsertEbookMediaItem(ctx context.Context, folderID int, filePath string, book *parsedEbook) (string, error) {
if s.itemRepo == nil {
return "", fmt.Errorf("itemRepo not configured on Scanner")
}
if s.fileRepo == nil {
return "", fmt.Errorf("fileRepo not configured on Scanner")
}
existingID, err := s.fileRepo.FindContentIDByRootPath(ctx, folderID, filePath, "ebook")
if err != nil {
return "", fmt.Errorf("find ebook by root path: %w", err)
}
if existingID != "" {
if err := updateExistingEbookMediaItem(ctx, s.itemRepo, s.itemRepo, existingID, book); err != nil {
return "", err
}
return existingID, nil
}
if existing := s.findEbookByFilePath(ctx, filePath); existing != nil {
applyEbookToMediaItem(existing, book)
if existing.SortTitle == "" {
existing.SortTitle = titleutil.DeriveDefaultSortTitle(existing.Title)
}
if err := s.itemRepo.Upsert(ctx, existing); err != nil {
return "", err
}
return existing.ContentID, nil
}
return resolveEbookMediaItem(ctx, s.fileRepo, s.itemRepo, folderID, filePath, book)
}
func resolveEbookMediaItem(
ctx context.Context,
rootFinder filesystemRootContentFinder,
itemWriter filesystemMediaItemWriter,
folderID int,
filePath string,
book *parsedEbook,
) (string, error) {
if rootFinder == nil {
return "", fmt.Errorf("root content finder not configured")
}
if itemWriter == nil {
return "", fmt.Errorf("media item writer not configured")
}
existingID, err := rootFinder.FindContentIDByRootPath(ctx, folderID, filePath, "ebook")
if err != nil {
return "", fmt.Errorf("find ebook by root path: %w", err)
}
if existingID != "" {
return existingID, nil
}
cleanTitle := strings.TrimSpace(book.Title)
if cleanTitle == "" {
cleanTitle = strings.TrimSuffix(filepath.Base(filePath), filepath.Ext(filePath))
}
return createEbookMediaItem(ctx, itemWriter, book, cleanTitle)
}
func createEbookMediaItem(ctx context.Context, itemWriter filesystemMediaItemWriter, book *parsedEbook, cleanTitle string) (string, error) {
id, err := idgen.NextID()
if err != nil {
return "", fmt.Errorf("generate content_id: %w", err)
}
item := &models.MediaItem{
ContentID: id,
Type: "ebook",
Title: cleanTitle,
SortTitle: titleutil.DeriveDefaultSortTitle(cleanTitle),
Year: book.Year,
}
applyEbookToMediaItem(item, book)
if item.SortTitle == "" {
item.SortTitle = titleutil.DeriveDefaultSortTitle(item.Title)
}
if err := itemWriter.Upsert(ctx, item); err != nil {
return "", err
}
return id, nil
}
func updateExistingEbookMediaItem(ctx context.Context, itemReader filesystemMediaItemReader, itemWriter filesystemMediaItemWriter, contentID string, book *parsedEbook) error {
if itemReader == nil {
return fmt.Errorf("media item reader not configured")
}
if itemWriter == nil {
return fmt.Errorf("media item writer not configured")
}
items, err := itemReader.GetByIDs(ctx, []string{contentID})
if err != nil {
return fmt.Errorf("get ebook media item %s: %w", contentID, err)
}
if len(items) == 0 || items[0] == nil {
return fmt.Errorf("ebook media item %s not found", contentID)
}
item := items[0]
applyEbookToMediaItem(item, book)
if item.SortTitle == "" {
item.SortTitle = titleutil.DeriveDefaultSortTitle(item.Title)
}
return itemWriter.Upsert(ctx, item)
}
func applyEbookToMediaItem(item *models.MediaItem, book *parsedEbook) {
item.Type = "ebook"
if title := strings.TrimSpace(book.Title); title != "" {
item.Title = title
}
if book.Year > 0 {
item.Year = book.Year
}
if book.Description != "" && (item.Overview == "" || looksLikeHTML(item.Overview)) {
item.Overview = book.Description
}
if book.Publisher != "" {
item.Studios = mergeUniqueStrings(item.Studios, []string{book.Publisher})
}
if len(book.Genres) > 0 {
item.Genres = mergeUniqueStrings(item.Genres, book.Genres)
}
if !book.PublishedAt.IsZero() && item.ReleaseDate == nil {
rd := book.PublishedAt.UTC().Format("2006-01-02")
item.ReleaseDate = &rd
}
if book.Language != "" && item.OriginalLanguage == "" {
item.OriginalLanguage = book.Language
}
}
func (s *Scanner) findEbookByFilePath(ctx context.Context, filePath string) *models.MediaItem {
if s.fileRepo == nil {
return nil
}
var existingID string
err := s.fileRepo.Pool().QueryRow(ctx, `
SELECT mf.content_id
FROM media_files mf
JOIN media_items mi ON mi.content_id = mf.content_id
WHERE mf.file_path = $1
AND mi.type = 'ebook'
LIMIT 1
`, filePath).Scan(&existingID)
if err != nil || existingID == "" {
return nil
}
items, err := s.itemRepo.GetByIDs(ctx, []string{existingID})
if err != nil || len(items) == 0 {
return nil
}
return items[0]
}
func (s *Scanner) upsertEbookMediaFile(ctx context.Context, folder *models.MediaFolder, contentID string, filePath string, size int64, modifiedAt time.Time, book *parsedEbook) error {
mf := buildEbookMediaFile(folder, contentID, filePath, size, modifiedAt, book)
if _, err := s.fileRepo.Upsert(ctx, mf); err != nil {
return fmt.Errorf("upsert media file %s: %w", filePath, err)
}
return nil
}
func buildEbookMediaFile(folder *models.MediaFolder, contentID string, filePath string, size int64, modifiedAt time.Time, book *parsedEbook) models.MediaFile {
return models.MediaFile{
ContentID: contentID,
MediaFolderID: folder.ID,
CanonicalRootPath: filePath,
ObservedRootPath: filePath,
ContentGroupKey: contentID,
GroupKeyVersion: 1,
BaseTitle: book.Title,
BaseYear: book.Year,
BaseType: "ebook",
IdentityConfidence: ebookIdentityConfidence(book),
FilePath: filePath,
FileSize: size,
FileModifiedAt: &modifiedAt,
Container: book.Format,
Duration: book.PageCount,
ProbeSource: "local",
}
}
type ebookCoverMetadataStore interface {
GetByIDs(ctx context.Context, ids []string) ([]*models.MediaItem, error)
UpdateMetadata(ctx context.Context, contentID string, upd *catalog.MetadataUpdate) error
}
func applyEbookEmbeddedCover(ctx context.Context, store ebookCoverMetadataStore, cacher ebookCoverCacher, contentID string, book *parsedEbook) error {
if store == nil || cacher == nil || contentID == "" || book == nil || book.Cover == nil || len(book.Cover.Bytes) == 0 {
return nil
}
items, err := store.GetByIDs(ctx, []string{contentID})
if err != nil {
return fmt.Errorf("get ebook media item for cover: %w", err)
}
if len(items) > 0 && items[0] != nil && strings.TrimSpace(items[0].PosterPath) != "" {
return nil
}
basePath, ext, thumbhash, err := cacher.CacheEbookCover(ctx, book.Cover.Bytes, contentID)
if err != nil {
return err
}
posterPath := strings.TrimRight(basePath, "/") + "/original" + ext
update := &catalog.MetadataUpdate{PosterPath: &posterPath}
if thumbhash != "" {
update.PosterThumbhash = &thumbhash
}
return store.UpdateMetadata(ctx, contentID, update)
}
func insertEbookLibraryMembership(ctx context.Context, exec ebookSQLExecutor, contentID string, folderID int) error {
_, err := exec.Exec(ctx, `
INSERT INTO media_item_libraries (content_id, media_folder_id, first_seen_at)
VALUES ($1, $2, NOW())
ON CONFLICT (content_id, media_folder_id) DO NOTHING
`, contentID, folderID)
return err
}
func insertEbookISBNProviderID(ctx context.Context, exec ebookSQLExecutor, contentID string, isbn string) error {
_, err := exec.Exec(ctx, `
INSERT INTO media_item_provider_ids (content_id, provider, provider_id, item_type)
VALUES ($1, 'isbn', $2, 'ebook')
ON CONFLICT DO NOTHING
`, contentID, isbn)
return err
}
type ebookCredit struct {
Name string
Kind models.PersonKind
}
func ebookPeopleCreditsEqual(existing []models.ItemPerson, desired []ebookCredit) bool {
existingAuthors := make([]models.ItemPerson, 0, len(existing))
for _, p := range existing {
if p.Kind == models.PersonKindNarrator {
return false
}
if p.Kind == models.PersonKindAuthor {
existingAuthors = append(existingAuthors, p)
}
}
if len(existingAuthors) != len(desired) {
return false
}
type key struct {
name string
kind models.PersonKind
}
have := make(map[key]struct{}, len(existingAuthors))
for _, p := range existingAuthors {
have[key{strings.ToLower(strings.TrimSpace(p.Person.Name)), p.Kind}] = struct{}{}
}
for _, d := range desired {
k := key{strings.ToLower(strings.TrimSpace(d.Name)), d.Kind}
if _, ok := have[k]; !ok {
return false
}
}
return true
}
type ebookResolvedAuthor struct {
ID int64
Name string
}
func mergeEbookPeople(existing []models.ItemPerson, authors []ebookResolvedAuthor) []models.ItemPerson {
people := make([]models.ItemPerson, 0, len(existing)+len(authors))
for _, p := range existing {
if p.Kind == models.PersonKindAuthor || p.Kind == models.PersonKindNarrator {
continue
}
p.SortOrder = len(people)
people = append(people, p)
}
for _, author := range authors {
people = append(people, models.ItemPerson{
Person: models.Person{ID: author.ID, Name: author.Name},
Kind: models.PersonKindAuthor,
SortOrder: len(people),
})
}
return people
}
func ebookPeopleForReplace(existing []models.ItemPerson, getErr error, authors []ebookResolvedAuthor) ([]models.ItemPerson, error) {
if getErr != nil {
return nil, fmt.Errorf("get ebook people: %w", getErr)
}
return mergeEbookPeople(existing, authors), nil
}
func (s *Scanner) upsertEbookPeople(ctx context.Context, contentID string, book *parsedEbook) error {
if s.personRepo == nil {
return fmt.Errorf("personRepo not configured on Scanner")
}
if s.itemRepo == nil {
return fmt.Errorf("itemRepo not configured on Scanner")
}
var desired []ebookCredit
for _, author := range book.Authors {
if name := strings.TrimSpace(author); name != "" {
desired = append(desired, ebookCredit{Name: name, Kind: models.PersonKindAuthor})
}
}
if len(desired) == 0 {
return nil
}
existing, err := s.itemRepo.GetPeople(ctx, contentID)
if err != nil {
return fmt.Errorf("get ebook people: %w", err)
}
if ebookPeopleCreditsEqual(existing, desired) {
return nil
}
authors := make([]ebookResolvedAuthor, 0, len(desired))
for _, c := range desired {
personID, err := s.personRepo.FindOrCreate(ctx, models.Person{Name: c.Name})
if err != nil {
return fmt.Errorf("find-or-create person %q: %w", c.Name, err)
}
authors = append(authors, ebookResolvedAuthor{ID: personID, Name: c.Name})
}
people, err := ebookPeopleForReplace(existing, nil, authors)
if err != nil {
return err
}
return s.itemRepo.ReplacePeople(ctx, contentID, people)
}
func (s *Scanner) upsertEbookSeries(ctx context.Context, contentID string, book *parsedEbook) error {
if s == nil {
return fmt.Errorf("Scanner not configured")
}
if s.fileRepo == nil {
return fmt.Errorf("fileRepo not configured on Scanner")
}
var currentName *string
var currentIdx *float64
err := s.fileRepo.Pool().QueryRow(ctx, `
SELECT series_name, series_index FROM ebook_series WHERE content_id = $1
`, contentID).Scan(&currentName, &currentIdx)
plan, err := planEbookSeriesWrite(book, currentName, currentIdx, err)
if err != nil {
return err
}
switch plan.Kind {
case ebookSeriesWriteNone:
return nil
case ebookSeriesWriteDelete:
if _, delErr := s.fileRepo.Pool().Exec(ctx,
`DELETE FROM ebook_series WHERE content_id = $1`, contentID); delErr != nil {
return fmt.Errorf("delete ebook_series row: %w", delErr)
}
return nil
case ebookSeriesWriteUpsert:
var idx any
if plan.Index != nil {
idx = *plan.Index
}
if _, err := s.fileRepo.Pool().Exec(ctx, `
INSERT INTO ebook_series (content_id, series_name, series_index, updated_at)
VALUES ($1, $2, $3, NOW())
ON CONFLICT (content_id) DO UPDATE SET
series_name = EXCLUDED.series_name,
series_index = EXCLUDED.series_index,
updated_at = NOW()
`, contentID, plan.Name, idx); err != nil {
return fmt.Errorf("upsert ebook_series row: %w", err)
}
return nil
default:
return fmt.Errorf("unknown ebook_series write kind: %d", plan.Kind)
}
}
func ebookSeriesDesired(book *parsedEbook) (string, *float64) {
if book == nil {
return "", nil
}
return strings.TrimSpace(book.Series), parseSeriesIndex(book.SeriesIndex)
}
type ebookSeriesWriteKind int
const (
ebookSeriesWriteNone ebookSeriesWriteKind = iota
ebookSeriesWriteDelete
ebookSeriesWriteUpsert
)
type ebookSeriesWritePlan struct {
Kind ebookSeriesWriteKind
Name string
Index *float64
}
func planEbookSeriesWrite(book *parsedEbook, currentName *string, currentIdx *float64, queryErr error) (ebookSeriesWritePlan, error) {
if queryErr != nil {
if !errors.Is(queryErr, pgx.ErrNoRows) {
return ebookSeriesWritePlan{}, fmt.Errorf("query ebook_series: %w", queryErr)
}
currentName = nil
currentIdx = nil
}
desiredName, desiredIdx := ebookSeriesDesired(book)
if desiredName == "" {
if currentName == nil {
return ebookSeriesWritePlan{Kind: ebookSeriesWriteNone}, nil
}
return ebookSeriesWritePlan{Kind: ebookSeriesWriteDelete}, nil
}
if currentName != nil && *currentName == desiredName && floatPtrEqual(currentIdx, desiredIdx) {
return ebookSeriesWritePlan{Kind: ebookSeriesWriteNone}, nil
}
return ebookSeriesWritePlan{
Kind: ebookSeriesWriteUpsert,
Name: desiredName,
Index: desiredIdx,
}, nil
}
func ebookIdentityConfidence(book *parsedEbook) string {
if book == nil {
return "low"
}
score := 0
if book.Title != "" {
score++
}
if len(book.Authors) > 0 {
score++
}
if book.Year > 0 {
score++
}
if book.ISBN != "" {
score++
}
switch {
case score >= 4:
return "high"
case score > 0:
return "medium"
default:
return "low"
}
}
+952
View File
@@ -0,0 +1,952 @@
package scanner
import (
"archive/zip"
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/Silo-Server/silo-server/internal/catalog"
"github.com/Silo-Server/silo-server/internal/models"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type recordingEbookExecutor struct {
queries []string
args [][]any
}
func (r *recordingEbookExecutor) Exec(_ context.Context, query string, args ...any) (pgconn.CommandTag, error) {
r.queries = append(r.queries, query)
r.args = append(r.args, append([]any(nil), args...))
return pgconn.CommandTag{}, nil
}
type fakeFilesystemItemReader struct {
items []*models.MediaItem
err error
}
func (f *fakeFilesystemItemReader) GetByIDs(_ context.Context, _ []string) ([]*models.MediaItem, error) {
return f.items, f.err
}
type fakeEbookCoverCacher struct {
calls int
data []byte
contentID string
err error
}
func (f *fakeEbookCoverCacher) CacheEbookCover(_ context.Context, data []byte, contentID string) (string, string, string, error) {
f.calls++
f.data = append([]byte(nil), data...)
f.contentID = contentID
if f.err != nil {
return "", "", "", f.err
}
return "local/ebooks/" + contentID + "/poster", ".webp", "thumb", nil
}
type fakeEbookMetadataUpdater struct {
items []*models.MediaItem
getErr error
contentID string
update *catalog.MetadataUpdate
err error
}
func (f *fakeEbookMetadataUpdater) GetByIDs(_ context.Context, _ []string) ([]*models.MediaItem, error) {
return f.items, f.getErr
}
func (f *fakeEbookMetadataUpdater) UpdateMetadata(_ context.Context, contentID string, update *catalog.MetadataUpdate) error {
f.contentID = contentID
f.update = update
return f.err
}
func TestSupportsEbookFile(t *testing.T) {
cases := []struct {
path string
want bool
}{
{"book.epub", true},
{"book.PDF", true},
{"book.mobi", true},
{"book.azw", true},
{"book.azw3", true},
{"book.FB2", true},
{"book.fb2.zip", true},
{"book.fbz", true},
{"book.cbz", true},
{"book.cbr", true},
{"book.txt", false},
{"book.md", true},
{"book.mp3", false},
{"movie.mkv", false},
}
for _, tc := range cases {
if got := SupportsEbookFile(tc.path); got != tc.want {
t.Errorf("SupportsEbookFile(%q) = %v, want %v", tc.path, got, tc.want)
}
}
}
func TestParseEbookFileSupportsReaderFormatsWithoutEmbeddedMetadata(t *testing.T) {
for _, tc := range []struct {
name string
wantFormat string
}{
{"book.pdf", "pdf"},
{"book.mobi", "mobi"},
{"book.azw", "azw"},
{"book.azw3", "azw3"},
{"book.cbr", "cbr"},
{"book.md", "md"},
} {
t.Run(tc.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), tc.name)
if err := os.WriteFile(path, []byte("placeholder"), 0o644); err != nil {
t.Fatalf("write file: %v", err)
}
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != tc.wantFormat {
t.Fatalf("Format = %q, want %q", got.Format, tc.wantFormat)
}
})
}
}
func TestNormalizeEbookISBNHandlesCommonLabels(t *testing.T) {
cases := []struct {
in string
want string
}{
{"ISBN-13: 978-0-306-40615-7", "9780306406157"},
{"ISBN-10: 0-9752298-0-x", "097522980X"},
}
for _, tc := range cases {
if got := normalizeEbookISBN(tc.in); got != tc.want {
t.Errorf("normalizeEbookISBN(%q) = %q, want %q", tc.in, got, tc.want)
}
}
}
func TestParseEbookEPUBMetadata(t *testing.T) {
path := writeTestEPUB(t, []string{"ISBN: 978-0-306-40615-7"})
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != "epub" {
t.Fatalf("Format = %q, want epub", got.Format)
}
if got.Title != "The Test Ebook" {
t.Fatalf("Title = %q, want The Test Ebook", got.Title)
}
if strings.Join(got.Authors, ", ") != "Ada Writer, Ben Author" {
t.Fatalf("Authors = %v, want Ada Writer and Ben Author", got.Authors)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN", got.ISBN)
}
if got.Year != 2024 {
t.Fatalf("Year = %d, want 2024", got.Year)
}
if got.Publisher != "Silo Press" {
t.Fatalf("Publisher = %q, want Silo Press", got.Publisher)
}
wantPublishedAt := time.Date(2024, 3, 10, 0, 0, 0, 0, time.UTC)
if !got.PublishedAt.Equal(wantPublishedAt) {
t.Fatalf("PublishedAt = %v, want %v", got.PublishedAt, wantPublishedAt)
}
if got.Language != "en" {
t.Fatalf("Language = %q, want en", got.Language)
}
if strings.Join(got.Genres, ", ") != "Fiction, Adventure" {
t.Fatalf("Genres = %v, want Fiction and Adventure", got.Genres)
}
if got.Series != "Test Series" || got.SeriesIndex != "2" {
t.Fatalf("Series = %q/%q, want Test Series/2", got.Series, got.SeriesIndex)
}
}
func TestParseEbookEPUBMetadataStripsHTMLDescription(t *testing.T) {
path := writeTestEPUBWithDescription(t, `&lt;div&gt;&lt;p&gt;&lt;strong&gt;Shanghai Dreams ...&lt;/strong&gt; Magnus is stationed at Shanghai.&lt;br&gt;Published by The Electronic Book Company&lt;/p&gt;&lt;/div&gt;`)
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
want := "Shanghai Dreams ... Magnus is stationed at Shanghai. Published by The Electronic Book Company"
if got.Description != want {
t.Fatalf("Description = %q, want %q", got.Description, want)
}
}
func TestParseEbookEPUBSkipsUUIDIdentifierBeforeISBN(t *testing.T) {
path := writeTestEPUB(t, []string{
"550e8400-e29b-41d4-a716-446655440000",
"ISBN: 978-0-306-40615-7",
})
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN after skipping UUID", got.ISBN)
}
}
func TestParseEbookEPUBMetadataReadsISBNFromMetaTags(t *testing.T) {
path := writeTestEPUBWithMeta(t, nil, ` <meta name="calibre:isbn" content="978-0-306-40615-7"/>`)
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN from calibre meta tag", got.ISBN)
}
}
func TestReadEPUBZipEntryRejectsOversizedEntry(t *testing.T) {
path := filepath.Join(t.TempDir(), "large.epub")
file, err := os.Create(path)
if err != nil {
t.Fatalf("create epub: %v", err)
}
zw := zip.NewWriter(file)
w, err := zw.Create("OPS/content.opf")
if err != nil {
t.Fatalf("create zip entry: %v", err)
}
if _, err := w.Write([]byte(strings.Repeat("x", maxEPUBMetadataEntrySize+1))); err != nil {
t.Fatalf("write zip entry: %v", err)
}
if err := zw.Close(); err != nil {
t.Fatalf("close zip: %v", err)
}
if err := file.Close(); err != nil {
t.Fatalf("close file: %v", err)
}
reader, err := zip.OpenReader(path)
if err != nil {
t.Fatalf("open zip: %v", err)
}
defer reader.Close()
_, err = readEPUBZipEntry(&reader.Reader, "OPS/content.opf")
if err == nil || !strings.Contains(err.Error(), "epub entry too large") {
t.Fatalf("readEPUBZipEntry error = %v, want size limit error", err)
}
}
func TestParseEbookFB2Metadata(t *testing.T) {
path := filepath.Join(t.TempDir(), "book.fb2")
if err := os.WriteFile(path, []byte(`<?xml version="1.0" encoding="UTF-8"?>
<FictionBook>
<description>
<title-info>
<genre>science fiction</genre>
<author><first-name>Ada</first-name><last-name>Writer</last-name></author>
<book-title>FB2 Test Ebook</book-title>
<date value="2022-04-05">5 April 2022</date>
<lang>en</lang>
<sequence name="FB2 Series" number="3"/>
</title-info>
<publish-info>
<publisher>FB2 Press</publisher>
<year>2022</year>
<isbn>978-0-306-40615-7</isbn>
</publish-info>
</description>
</FictionBook>`), 0o644); err != nil {
t.Fatalf("write fb2: %v", err)
}
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != "fb2" || got.Title != "FB2 Test Ebook" {
t.Fatalf("Format/Title = %q/%q, want fb2/FB2 Test Ebook", got.Format, got.Title)
}
if strings.Join(got.Authors, ", ") != "Ada Writer" {
t.Fatalf("Authors = %v, want Ada Writer", got.Authors)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN", got.ISBN)
}
if got.Publisher != "FB2 Press" {
t.Fatalf("Publisher = %q, want FB2 Press", got.Publisher)
}
if got.Year != 2022 {
t.Fatalf("Year = %d, want 2022", got.Year)
}
wantPublishedAt := time.Date(2022, 4, 5, 0, 0, 0, 0, time.UTC)
if !got.PublishedAt.Equal(wantPublishedAt) {
t.Fatalf("PublishedAt = %v, want %v", got.PublishedAt, wantPublishedAt)
}
if got.Language != "en" {
t.Fatalf("Language = %q, want en", got.Language)
}
if strings.Join(got.Genres, ", ") != "science fiction" {
t.Fatalf("Genres = %v, want science fiction", got.Genres)
}
if got.Series != "FB2 Series" || got.SeriesIndex != "3" {
t.Fatalf("Series = %q/%q, want FB2 Series/3", got.Series, got.SeriesIndex)
}
}
func TestParseEbookFBZMetadata(t *testing.T) {
path := filepath.Join(t.TempDir(), "book.fb2.zip")
file, err := os.Create(path)
if err != nil {
t.Fatalf("create fbz: %v", err)
}
zw := zip.NewWriter(file)
w, err := zw.Create("nested/book.fb2")
if err != nil {
t.Fatalf("create fb2 entry: %v", err)
}
if _, err := w.Write([]byte(`<?xml version="1.0" encoding="UTF-8"?>
<FictionBook>
<description>
<title-info>
<genre>fantasy</genre>
<author><first-name>Ada</first-name><last-name>Writer</last-name></author>
<book-title>FBZ Test Ebook</book-title>
<date value="2021-02-03"/>
<lang>en</lang>
<sequence name="Archive Series" number="4"/>
</title-info>
<publish-info>
<publisher>Archive Press</publisher>
<year>2021</year>
<isbn>978-0-306-40615-7</isbn>
</publish-info>
</description>
</FictionBook>`)); err != nil {
t.Fatalf("write fb2 entry: %v", err)
}
if err := zw.Close(); err != nil {
t.Fatalf("close zip: %v", err)
}
if err := file.Close(); err != nil {
t.Fatalf("close fbz: %v", err)
}
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != "fbz" || got.Title != "FBZ Test Ebook" {
t.Fatalf("Format/Title = %q/%q, want fbz/FBZ Test Ebook", got.Format, got.Title)
}
if strings.Join(got.Authors, ", ") != "Ada Writer" {
t.Fatalf("Authors = %v, want Ada Writer", got.Authors)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN", got.ISBN)
}
if got.Publisher != "Archive Press" || got.Year != 2021 {
t.Fatalf("Publisher/Year = %q/%d, want Archive Press/2021", got.Publisher, got.Year)
}
if got.Series != "Archive Series" || got.SeriesIndex != "4" {
t.Fatalf("Series = %q/%q, want Archive Series/4", got.Series, got.SeriesIndex)
}
}
func TestParseEbookPDFInfoMetadata(t *testing.T) {
path := filepath.Join(t.TempDir(), "book.pdf")
if err := os.WriteFile(path, []byte(`%PDF-1.7
1 0 obj
<< /Title (PDF Test Ebook)
/Author (Ada Writer; Ben Author)
/Subject (ISBN 978-0-306-40615-7)
/Keywords (science fiction, adventure)
/CreationDate (D:20240102030405Z)
>>
endobj
trailer
<< /Info 1 0 R >>
%%EOF`), 0o644); err != nil {
t.Fatalf("write pdf: %v", err)
}
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != "pdf" || got.Title != "PDF Test Ebook" {
t.Fatalf("Format/Title = %q/%q, want pdf/PDF Test Ebook", got.Format, got.Title)
}
if strings.Join(got.Authors, ", ") != "Ada Writer, Ben Author" {
t.Fatalf("Authors = %v, want Ada Writer and Ben Author", got.Authors)
}
if got.ISBN != "9780306406157" {
t.Fatalf("ISBN = %q, want normalized ISBN", got.ISBN)
}
if got.Year != 2024 {
t.Fatalf("Year = %d, want 2024", got.Year)
}
if strings.Join(got.Genres, ", ") != "science fiction, adventure" {
t.Fatalf("Genres = %v, want science fiction and adventure", got.Genres)
}
}
func TestParseEbookCBZPageCount(t *testing.T) {
path := filepath.Join(t.TempDir(), "comic.cbz")
file, err := os.Create(path)
if err != nil {
t.Fatalf("create cbz: %v", err)
}
zw := zip.NewWriter(file)
for _, name := range []string{
"Comic/001.jpg",
"Comic/002.PNG",
"Comic/003.webp",
"Comic/notes.txt",
"__MACOSX/._001.jpg",
} {
w, err := zw.Create(name)
if err != nil {
t.Fatalf("create zip entry %s: %v", name, err)
}
if _, err := w.Write([]byte("placeholder")); err != nil {
t.Fatalf("write zip entry %s: %v", name, err)
}
}
if err := zw.Close(); err != nil {
t.Fatalf("close zip: %v", err)
}
if err := file.Close(); err != nil {
t.Fatalf("close cbz: %v", err)
}
got, err := parseEbookFile(path)
if err != nil {
t.Fatalf("parseEbookFile: %v", err)
}
if got.Format != "cbz" {
t.Fatalf("Format = %q, want cbz", got.Format)
}
if got.PageCount != 3 {
t.Fatalf("PageCount = %d, want 3", got.PageCount)
}
}
func TestEbookSeriesDesiredParsesIndex(t *testing.T) {
name, idx := ebookSeriesDesired(&parsedEbook{
Series: " The Expanse ",
SeriesIndex: "2 of 9",
})
if name != "The Expanse" {
t.Fatalf("series name = %q, want The Expanse", name)
}
if idx == nil || *idx != 2 {
t.Fatalf("series index = %v, want 2", idx)
}
}
func TestUpsertEbookSeriesNilScannerReturnsError(t *testing.T) {
var s *Scanner
if err := s.upsertEbookSeries(context.Background(), "content-1", &parsedEbook{Series: "Series"}); err == nil {
t.Fatal("upsertEbookSeries nil scanner error = nil, want error")
}
}
func TestUpsertEbookSeriesNilFileRepoReturnsError(t *testing.T) {
s := &Scanner{}
if err := s.upsertEbookSeries(context.Background(), "content-1", &parsedEbook{Series: "Series"}); err == nil {
t.Fatal("upsertEbookSeries nil fileRepo error = nil, want error")
}
}
func TestPlanEbookSeriesWriteInsertsWhenRowAbsent(t *testing.T) {
plan, err := planEbookSeriesWrite(&parsedEbook{Series: " Series ", SeriesIndex: "2 of 9"}, nil, nil, pgx.ErrNoRows)
if err != nil {
t.Fatalf("planEbookSeriesWrite: %v", err)
}
if plan.Kind != ebookSeriesWriteUpsert || plan.Name != "Series" {
t.Fatalf("plan = %+v, want upsert Series", plan)
}
if plan.Index == nil || *plan.Index != 2 {
t.Fatalf("index = %v, want 2", plan.Index)
}
}
func TestPlanEbookSeriesWriteBlankSeriesDeletesExistingAndSkipsAbsent(t *testing.T) {
currentName := "Series"
plan, err := planEbookSeriesWrite(&parsedEbook{Series: " "}, &currentName, nil, nil)
if err != nil {
t.Fatalf("planEbookSeriesWrite delete: %v", err)
}
if plan.Kind != ebookSeriesWriteDelete {
t.Fatalf("blank existing plan = %+v, want delete", plan)
}
plan, err = planEbookSeriesWrite(&parsedEbook{Series: ""}, nil, nil, pgx.ErrNoRows)
if err != nil {
t.Fatalf("planEbookSeriesWrite absent: %v", err)
}
if plan.Kind != ebookSeriesWriteNone {
t.Fatalf("blank absent plan = %+v, want none", plan)
}
}
func TestPlanEbookSeriesWriteSkipsIdenticalRow(t *testing.T) {
currentName := "Series"
currentIdx := 3.5
plan, err := planEbookSeriesWrite(&parsedEbook{Series: "Series", SeriesIndex: "3.5"}, &currentName, &currentIdx, nil)
if err != nil {
t.Fatalf("planEbookSeriesWrite: %v", err)
}
if plan.Kind != ebookSeriesWriteNone {
t.Fatalf("identical plan = %+v, want none", plan)
}
}
func TestPlanEbookSeriesWriteAllowsNullNumericIndex(t *testing.T) {
plan, err := planEbookSeriesWrite(&parsedEbook{Series: "Series", SeriesIndex: "appendix"}, nil, nil, pgx.ErrNoRows)
if err != nil {
t.Fatalf("planEbookSeriesWrite: %v", err)
}
if plan.Kind != ebookSeriesWriteUpsert || plan.Index != nil {
t.Fatalf("plan = %+v, want upsert with nil index", plan)
}
}
func TestPlanEbookSeriesWriteReturnsQueryError(t *testing.T) {
queryErr := errors.New("query failed")
if _, err := planEbookSeriesWrite(&parsedEbook{Series: "Series"}, nil, nil, queryErr); !errors.Is(err, queryErr) {
t.Fatalf("planEbookSeriesWrite error = %v, want query error", err)
}
}
func TestEbookIdentityConfidenceReflectsMetadataCompleteness(t *testing.T) {
book := &parsedEbook{Title: "Tagged Ebook", Authors: []string{"Author"}, Year: 2024, ISBN: "9780306406157"}
if got := ebookIdentityConfidence(book); got != "high" {
t.Fatalf("complete metadata confidence = %q, want high", got)
}
book = &parsedEbook{Title: "Tagged Ebook", Authors: []string{"Author"}, Year: 2024}
if got := ebookIdentityConfidence(book); got != "medium" {
t.Fatalf("partial metadata confidence = %q, want medium", got)
}
book = &parsedEbook{}
if got := ebookIdentityConfidence(book); got != "low" {
t.Fatalf("empty metadata confidence = %q, want low", got)
}
}
func TestResolveEbookMediaItemCreatesNewWhenRootHasNoClaim(t *testing.T) {
finder := &fakeRootContentFinder{}
writer := &fakeFilesystemItemWriter{}
got, err := resolveEbookMediaItem(context.Background(), finder, writer, 7, "/library/Author/Book.epub", &parsedEbook{
Title: "Book", Year: 2024, Authors: []string{"Author"}, Description: "Overview", Publisher: "Publisher", Genres: []string{"Fiction"}, Language: "en",
})
if err != nil {
t.Fatalf("resolveEbookMediaItem: %v", err)
}
if got == "" || len(writer.upserts) != 1 {
t.Fatalf("contentID/upserts = %q/%d, want id and one upsert", got, len(writer.upserts))
}
item := writer.upserts[0]
if item.Type != "ebook" || item.Title != "Book" || item.Year != 2024 || item.Overview != "Overview" || item.OriginalLanguage != "en" {
t.Fatalf("upserted ebook item = %+v", item)
}
}
func TestResolveEbookMediaItemReusesRootScopedContentID(t *testing.T) {
finder := &fakeRootContentFinder{contentID: "ebook-root-id"}
writer := &fakeFilesystemItemWriter{}
got, err := resolveEbookMediaItem(context.Background(), finder, writer, 7, "/library/Author/Book.epub", &parsedEbook{Title: "Book"})
if err != nil {
t.Fatalf("resolveEbookMediaItem: %v", err)
}
if got != "ebook-root-id" {
t.Fatalf("contentID = %q, want root-scoped id", got)
}
if len(writer.upserts) != 0 {
t.Fatalf("unexpected item upsert for existing root: %d", len(writer.upserts))
}
}
func TestResolveEbookExistingRootAppliesParsedMetadata(t *testing.T) {
reader := &fakeFilesystemItemReader{items: []*models.MediaItem{{
ContentID: "ebook-root-id",
Type: "ebook",
Title: "Old Title",
Year: 2020,
OriginalLanguage: "",
}}}
writer := &fakeFilesystemItemWriter{}
err := updateExistingEbookMediaItem(context.Background(), reader, writer, "ebook-root-id", &parsedEbook{
Title: "New Title",
Year: 2026,
Publisher: "New Press",
Genres: []string{"Fiction"},
Language: "en",
})
if err != nil {
t.Fatalf("updateExistingEbookMediaItem: %v", err)
}
if len(writer.upserts) != 1 {
t.Fatalf("upserts = %d, want 1", len(writer.upserts))
}
item := writer.upserts[0]
if item.ContentID != "ebook-root-id" || item.Type != "ebook" || item.Title != "New Title" || item.Year != 2026 || item.OriginalLanguage != "en" {
t.Fatalf("updated item = %+v", item)
}
if strings.Join(item.Studios, ",") != "New Press" || strings.Join(item.Genres, ",") != "Fiction" {
t.Fatalf("updated item studios/genres = %+v/%+v", item.Studios, item.Genres)
}
}
func TestResolveEbookExistingRootReplacesHTMLOverview(t *testing.T) {
reader := &fakeFilesystemItemReader{items: []*models.MediaItem{{
ContentID: "ebook-root-id",
Type: "ebook",
Title: "Old Title",
Overview: "<div><p>Raw overview</p></div>",
}}}
writer := &fakeFilesystemItemWriter{}
err := updateExistingEbookMediaItem(context.Background(), reader, writer, "ebook-root-id", &parsedEbook{
Title: "New Title",
Description: "Clean overview",
})
if err != nil {
t.Fatalf("updateExistingEbookMediaItem: %v", err)
}
if len(writer.upserts) != 1 {
t.Fatalf("upserts = %d, want 1", len(writer.upserts))
}
if writer.upserts[0].Overview != "Clean overview" {
t.Fatalf("Overview = %q, want Clean overview", writer.upserts[0].Overview)
}
}
func TestApplyEbookToMediaItemPreservesExistingYearWhenParsedYearMissing(t *testing.T) {
item := &models.MediaItem{
Type: "ebook",
Title: "Existing",
Year: 1999,
}
applyEbookToMediaItem(item, &parsedEbook{Title: "Updated", Year: 0})
if item.Title != "Updated" {
t.Fatalf("Title = %q, want Updated", item.Title)
}
if item.Year != 1999 {
t.Fatalf("Year = %d, want preserved 1999", item.Year)
}
}
func TestEbookPeopleCreditsEqualAuthorsOnly(t *testing.T) {
existing := []models.ItemPerson{
{Person: models.Person{Name: "Ada Writer"}, Kind: models.PersonKindAuthor, SortOrder: 0},
{Person: models.Person{Name: "Ben Author"}, Kind: models.PersonKindAuthor, SortOrder: 1},
{Person: models.Person{Name: "Manual Contributor"}, Kind: models.PersonKindWriter, SortOrder: 2},
}
desired := []ebookCredit{
{Name: "Ada Writer", Kind: models.PersonKindAuthor},
{Name: "Ben Author", Kind: models.PersonKindAuthor},
}
if !ebookPeopleCreditsEqual(existing, desired) {
t.Fatal("expected matching author credits to compare equal while ignoring non-authors")
}
existing[1] = models.ItemPerson{Person: models.Person{Name: "Other Author"}, Kind: models.PersonKindAuthor, SortOrder: 1}
if ebookPeopleCreditsEqual(existing, desired) {
t.Fatal("expected different author credit to make ebook author set differ")
}
}
func TestEbookPeopleCreditsEqualRejectsStaleNarratorCredit(t *testing.T) {
existing := []models.ItemPerson{
{Person: models.Person{Name: "Ada Writer"}, Kind: models.PersonKindAuthor, SortOrder: 0},
{Person: models.Person{Name: "Legacy Narrator"}, Kind: models.PersonKindNarrator, SortOrder: 1},
}
desired := []ebookCredit{
{Name: "Ada Writer", Kind: models.PersonKindAuthor},
}
if ebookPeopleCreditsEqual(existing, desired) {
t.Fatal("expected stale narrator credit to force ebook people replacement")
}
}
func TestEbookPeopleMergePreservesExistingNonAuthorCreditsExceptNarrators(t *testing.T) {
existing := []models.ItemPerson{
{Person: models.Person{ID: 10, Name: "Old Author"}, Kind: models.PersonKindAuthor, SortOrder: 0},
{Person: models.Person{ID: 20, Name: "Manual Writer"}, Kind: models.PersonKindWriter, SortOrder: 1, Character: "essay"},
{Person: models.Person{ID: 30, Name: "Manual Narrator"}, Kind: models.PersonKindNarrator, SortOrder: 2},
}
authors := []ebookResolvedAuthor{
{ID: 40, Name: "New Author"},
}
got := mergeEbookPeople(existing, authors)
if len(got) != 2 {
t.Fatalf("merged people len = %d, want 2: %+v", len(got), got)
}
if got[0].Person.ID != 20 || got[0].Kind != models.PersonKindWriter || got[0].Character != "essay" {
t.Fatalf("first preserved non-author = %+v", got[0])
}
if got[1].Person.ID != 40 || got[1].Kind != models.PersonKindAuthor || got[1].SortOrder != 1 {
t.Fatalf("new author credit = %+v", got[1])
}
}
func TestEbookPeopleReplacePlanReturnsGetPeopleError(t *testing.T) {
wantErr := errors.New("get people failed")
_, err := ebookPeopleForReplace(nil, wantErr, []ebookResolvedAuthor{{ID: 40, Name: "New Author"}})
if !errors.Is(err, wantErr) {
t.Fatalf("error = %v, want %v", err, wantErr)
}
}
func TestScanEbookBuildMediaFileSetsCorePersistenceFields(t *testing.T) {
modifiedAt := time.Date(2026, 6, 8, 12, 0, 0, 0, time.UTC)
book := &parsedEbook{Format: "epub", Title: "Book", Authors: []string{"Author"}, Year: 2024, ISBN: "9780306406157", PageCount: 321}
got := buildEbookMediaFile(&models.MediaFolder{ID: 44}, "content-1", "/library/Book.epub", 1234, modifiedAt, book)
if got.ContentID != "content-1" || got.MediaFolderID != 44 {
t.Fatalf("ids = %q/%d, want content-1/44", got.ContentID, got.MediaFolderID)
}
if got.BaseType != "ebook" || got.BaseTitle != "Book" || got.BaseYear != 2024 || got.Container != "epub" || got.ProbeSource != "local" {
t.Fatalf("ebook file metadata = %+v", got)
}
if got.Duration != 321 {
t.Fatalf("Duration = %d, want ebook page count", got.Duration)
}
if got.CanonicalRootPath != "/library/Book.epub" || got.ObservedRootPath != "/library/Book.epub" || got.FilePath != "/library/Book.epub" {
t.Fatalf("ebook paths = canonical %q observed %q file %q", got.CanonicalRootPath, got.ObservedRootPath, got.FilePath)
}
if got.FileSize != 1234 || got.FileModifiedAt == nil || !got.FileModifiedAt.Equal(modifiedAt) {
t.Fatalf("file facts = size %d modified %v", got.FileSize, got.FileModifiedAt)
}
}
func TestApplyEbookEmbeddedCoverCachesAndUpdatesPoster(t *testing.T) {
cacher := &fakeEbookCoverCacher{}
updater := &fakeEbookMetadataUpdater{}
err := applyEbookEmbeddedCover(context.Background(), updater, cacher, "content-1", &parsedEbook{
Cover: &parsedEbookCover{
ContentType: "image/jpeg",
Bytes: []byte("cover-bytes"),
},
})
if err != nil {
t.Fatalf("applyEbookEmbeddedCover: %v", err)
}
if cacher.calls != 1 || cacher.contentID != "content-1" || string(cacher.data) != "cover-bytes" {
t.Fatalf("cache call = calls %d content %q data %q", cacher.calls, cacher.contentID, string(cacher.data))
}
if updater.contentID != "content-1" || updater.update == nil {
t.Fatalf("update call = content %q update %#v", updater.contentID, updater.update)
}
if updater.update.PosterPath == nil || *updater.update.PosterPath != "local/ebooks/content-1/poster/original.webp" {
t.Fatalf("poster path = %#v", updater.update.PosterPath)
}
if updater.update.PosterThumbhash == nil || *updater.update.PosterThumbhash != "thumb" {
t.Fatalf("poster thumbhash = %#v", updater.update.PosterThumbhash)
}
}
func TestApplyEbookEmbeddedCoverPreservesExistingPoster(t *testing.T) {
cacher := &fakeEbookCoverCacher{}
updater := &fakeEbookMetadataUpdater{items: []*models.MediaItem{{ContentID: "content-1", PosterPath: "provider/poster.webp"}}}
err := applyEbookEmbeddedCover(context.Background(), updater, cacher, "content-1", &parsedEbook{
Cover: &parsedEbookCover{Bytes: []byte("cover-bytes")},
})
if err != nil {
t.Fatalf("applyEbookEmbeddedCover: %v", err)
}
if cacher.calls != 0 {
t.Fatalf("cache calls = %d, want 0", cacher.calls)
}
if updater.update != nil {
t.Fatalf("unexpected update = %#v", updater.update)
}
}
func TestScanEbookPersistenceSQLWritesLibraryMembershipAndISBNOnly(t *testing.T) {
ctx := context.Background()
exec := &recordingEbookExecutor{}
if err := insertEbookLibraryMembership(ctx, exec, "content-1", 44); err != nil {
t.Fatalf("insertEbookLibraryMembership: %v", err)
}
if err := insertEbookISBNProviderID(ctx, exec, "content-1", "9780306406157"); err != nil {
t.Fatalf("insertEbookISBNProviderID: %v", err)
}
if len(exec.queries) != 2 {
t.Fatalf("queries = %d, want 2", len(exec.queries))
}
if !strings.Contains(exec.queries[0], "media_item_libraries") || exec.args[0][0] != "content-1" || exec.args[0][1] != 44 {
t.Fatalf("library membership write = query %q args %+v", exec.queries[0], exec.args[0])
}
if !strings.Contains(exec.queries[1], "media_item_provider_ids") || !strings.Contains(exec.queries[1], "'isbn'") || !strings.Contains(exec.queries[1], "'ebook'") {
t.Fatalf("ISBN provider write query = %q", exec.queries[1])
}
if strings.Contains(exec.queries[1], "asin") || exec.args[1][0] != "content-1" || exec.args[1][1] != "9780306406157" {
t.Fatalf("ISBN provider write args/query = query %q args %+v", exec.queries[1], exec.args[1])
}
}
func TestScanEbookFolderReturnsErrorWhenEveryReconcileFails(t *testing.T) {
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "bad.epub"), []byte("not a real epub"), 0o644); err != nil {
t.Fatalf("write fake ebook: %v", err)
}
s := &Scanner{}
err := s.ScanEbookFolder(context.Background(), &models.MediaFolder{ID: 44, Paths: []string{root}})
if err == nil {
t.Fatal("ScanEbookFolder returned nil, want aggregate failure")
}
if !strings.Contains(err.Error(), "folder_id=44") {
t.Fatalf("error = %q, want folder id", err)
}
}
func TestScanEbookFolderReturnsCanceledContext(t *testing.T) {
root := t.TempDir()
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := &Scanner{}
err := s.ScanEbookFolder(ctx, &models.MediaFolder{ID: 44, Paths: []string{root}})
if !errors.Is(err, context.Canceled) {
t.Fatalf("ScanEbookFolder error = %v, want context.Canceled", err)
}
}
func TestScanEbookFolderReportsProgress(t *testing.T) {
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "broken.epub"), []byte("not a zip"), 0o644); err != nil {
t.Fatalf("write ebook fixture: %v", err)
}
var updates []ProgressUpdate
ctx := WithProgressReporter(context.Background(), func(update ProgressUpdate) {
updates = append(updates, update)
})
err := (&Scanner{}).ScanEbookFolder(ctx, &models.MediaFolder{ID: 44, Paths: []string{root}})
if err == nil {
t.Fatal("ScanEbookFolder returned nil, want aggregate failure")
}
if len(updates) == 0 {
t.Fatal("expected ebook scanner progress updates")
}
if updates[0].Phase != "ebook_scan" || updates[0].TotalFiles != 1 {
t.Fatalf("first progress update = %+v, want ebook_scan discovery total", updates[0])
}
last := updates[len(updates)-1]
if last.FilesProcessed != 1 || last.Errors != 1 {
t.Fatalf("last progress update = %+v, want processed/error counts", last)
}
}
func writeTestEPUB(t *testing.T, identifiers []string) string {
return writeTestEPUBWithMeta(t, identifiers, "")
}
func writeTestEPUBWithMeta(t *testing.T, identifiers []string, extraMetaXML string) string {
return writeTestEPUBWithDescriptionAndMeta(t, identifiers, "Back cover copy", extraMetaXML)
}
func writeTestEPUBWithDescription(t *testing.T, description string) string {
return writeTestEPUBWithDescriptionAndMeta(t, []string{"ISBN: 978-0-306-40615-7"}, description, "")
}
func writeTestEPUBWithDescriptionAndMeta(t *testing.T, identifiers []string, description string, extraMetaXML string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "book.epub")
file, err := os.Create(path)
if err != nil {
t.Fatalf("create epub: %v", err)
}
zw := zip.NewWriter(file)
add := func(name, body string) {
t.Helper()
w, err := zw.Create(name)
if err != nil {
t.Fatalf("create zip entry %s: %v", name, err)
}
if _, err := w.Write([]byte(body)); err != nil {
t.Fatalf("write zip entry %s: %v", name, err)
}
}
add("META-INF/container.xml", `<?xml version="1.0"?>
<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
<rootfiles>
<rootfile full-path="OPS/content.opf" media-type="application/oebps-package+xml"/>
</rootfiles>
</container>`)
var identifierXML strings.Builder
for _, identifier := range identifiers {
identifierXML.WriteString(" <dc:identifier>")
identifierXML.WriteString(identifier)
identifierXML.WriteString("</dc:identifier>\n")
}
add("OPS/content.opf", `<?xml version="1.0" encoding="UTF-8"?>
<package xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">
<metadata>
<dc:title>The Test Ebook</dc:title>
<dc:creator>Ada Writer</dc:creator>
<dc:creator>Ben Author</dc:creator>
`+identifierXML.String()+`
<dc:publisher>Silo Press</dc:publisher>
<dc:date>2024-03-10</dc:date>
<dc:language>en</dc:language>
<dc:subject>Fiction</dc:subject>
<dc:subject>Adventure</dc:subject>
<dc:description>`+description+`</dc:description>
<meta name="calibre:series" content="Test Series"/>
<meta name="calibre:series_index" content="2"/>
`+extraMetaXML+`
</metadata>
</package>`)
if err := zw.Close(); err != nil {
t.Fatalf("close zip: %v", err)
}
if err := file.Close(); err != nil {
t.Fatalf("close epub: %v", err)
}
return path
}
+36 -6
View File
@@ -97,9 +97,12 @@ func shouldSkipMovieSupplementalFile(path string) bool {
}
// Scanner discovers and indexes media files in media folders.
// scannerImageCacher is the slice of imagecache.Cacher the audiobook
// branch uses. Wired via SetImageCacher; nil-safe when absent.
type scannerImageCacher = audiobookCoverCacher
// scannerImageCacher is the slice of imagecache.Cacher the book scanners use.
// Wired via SetImageCacher; nil-safe when absent.
type scannerImageCacher interface {
audiobookCoverCacher
ebookCoverCacher
}
type Scanner struct {
fileRepo *FileRepository
@@ -126,9 +129,9 @@ type Scanner struct {
seriesQueueSyncer SeriesQueueSyncer
}
// SetImageCacher installs the imagecache.Cacher used by the audiobook
// branch to push embedded M4B cover art into the public assets bucket.
// Optional; if unset, audiobook covers are not extracted.
// SetImageCacher installs the imagecache.Cacher used by book scanners to push
// embedded cover art into the public assets bucket. Optional; if unset, local
// covers are not extracted.
func (s *Scanner) SetImageCacher(cacher scannerImageCacher) {
if s == nil {
return
@@ -241,6 +244,13 @@ func (s *Scanner) ScanFolder(ctx context.Context, folder *models.MediaFolder) (*
return &ScanResult{}, nil
}
if isEbookLibraryType(folder.Type) {
if err := s.ScanEbookFolder(watchCtx, folder); err != nil {
return nil, err
}
return &ScanResult{}, nil
}
return s.scanPaths(watchCtx, folder, folder.Paths, folder.Paths, true)
}
@@ -250,6 +260,12 @@ func (s *Scanner) ScanSubtree(ctx context.Context, folder *models.MediaFolder, s
cleanSubtree := filepath.Clean(subtreePath)
watchCtx, stopWatch := s.watchFolderContext(ctx, folder.ID)
defer stopWatch()
if isEbookLibraryType(folder.Type) {
if err := s.scanEbookPaths(watchCtx, folder, []string{cleanSubtree}); err != nil {
return nil, err
}
return &ScanResult{}, nil
}
return s.scanPaths(watchCtx, folder, []string{cleanSubtree}, []string{cleanSubtree}, false)
}
@@ -278,6 +294,15 @@ func isPodcastLibraryType(libraryType string) bool {
}
}
func isEbookLibraryType(libraryType string) bool {
switch strings.ToLower(strings.TrimSpace(libraryType)) {
case "ebook", "ebooks":
return true
default:
return false
}
}
// walkMode tells walkLogicalTree which file extensions to surface and
// which library-specific filename heuristics (sample/extra skipping)
// to apply.
@@ -288,6 +313,7 @@ const (
walkModeMovie // movie library: video extensions + sample/extra skipping
walkModeAudiobook // audiobook library: audio extensions, no skipping
walkModePodcast // podcast library: audio extensions, no skipping
walkModeEbook // ebook library: ebook extensions, no skipping
)
// walkModeFor derives a walkMode from a media_folders.type string.
@@ -301,6 +327,8 @@ func walkModeFor(folderType string) walkMode {
return walkModeAudiobook
case isPodcastLibraryType(folderType):
return walkModePodcast
case isEbookLibraryType(folderType):
return walkModeEbook
default:
return walkModeVideo
}
@@ -312,6 +340,8 @@ func (m walkMode) acceptsExt(ext string) bool {
switch m {
case walkModeAudiobook, walkModePodcast:
return audioExtensions[ext]
case walkModeEbook:
return ebookExtensions[ext]
default:
return videoExtensions[ext]
}
+73
View File
@@ -200,3 +200,76 @@ func TestIsPodcastLibraryType(t *testing.T) {
}
}
}
func TestIsEbookLibraryType(t *testing.T) {
cases := []struct {
in string
want bool
}{
{"ebooks", true},
{"ebook", true},
{"Ebook", true},
{" EBOOKS ", true},
{"audiobooks", false},
{"movies", false},
{"", false},
}
for _, tc := range cases {
if got := isEbookLibraryType(tc.in); got != tc.want {
t.Errorf("isEbookLibraryType(%q) = %v, want %v", tc.in, got, tc.want)
}
}
}
func TestWalkModeForEbookLibraryTypes(t *testing.T) {
for _, libraryType := range []string{"ebook", "ebooks", " EBOOKS "} {
if got := walkModeFor(libraryType); got != walkModeEbook {
t.Fatalf("walkModeFor(%q) = %v, want %v", libraryType, got, walkModeEbook)
}
}
}
func TestWalkModeEbookAcceptsEbookExtensionsOnly(t *testing.T) {
for _, ext := range []string{".epub", ".pdf", ".mobi", ".azw", ".azw3", ".fb2", ".fbz", ".cbz", ".cbr", ".md"} {
if !walkModeEbook.acceptsExt(ext) {
t.Fatalf("walkModeEbook should accept %s", ext)
}
}
for _, ext := range []string{".txt", ".mp4", ".mp3", ".mkv"} {
if walkModeEbook.acceptsExt(ext) {
t.Fatalf("walkModeEbook should reject %s", ext)
}
}
}
func TestScanFolderEbookLibraryRoutesToEbookScanner(t *testing.T) {
scanner := &Scanner{}
result, err := scanner.ScanFolder(context.Background(), &models.MediaFolder{
ID: 0,
Type: " ebooks ",
Paths: []string{t.TempDir()},
})
if err != nil {
t.Fatalf("ScanFolder ebook error = %v, want nil", err)
}
if result == nil {
t.Fatal("ScanFolder ebook result = nil, want empty result")
}
}
func TestScanSubtreeEbookLibraryRoutesToEbookScanner(t *testing.T) {
subtree := t.TempDir()
scanner := &Scanner{}
result, err := scanner.ScanSubtree(context.Background(), &models.MediaFolder{
ID: 0,
Type: "ebook",
Paths: []string{subtree},
}, subtree)
if err != nil {
t.Fatalf("ScanSubtree ebook error = %v, want nil", err)
}
if result == nil {
t.Fatal("ScanSubtree ebook result = nil, want empty result")
}
}
+8
View File
@@ -175,6 +175,14 @@ func DefaultLibrarySectionsForType(libraryID *int, libraryType string) []*PageSe
{ID: "default-recommended-for-you", Scope: "library", LibraryID: libraryID, Position: 3, SectionType: SectionRecommendedForYou, Title: "Recommended for You", ItemLimit: 20, Config: emptyCfg, Enabled: true},
{ID: "default-random-audiobooks", Scope: "library", LibraryID: libraryID, Position: 4, SectionType: SectionRandom, Title: "Random Picks", ItemLimit: 20, Config: defaultMediaScopeConfig("audiobook"), Enabled: true},
}
case "ebooks", "ebook":
return []*PageSection{
{ID: "default-continue-reading", Scope: "library", LibraryID: libraryID, Position: 0, SectionType: SectionContinueWatching, Title: "Continue Reading", ItemLimit: 20, Config: emptyCfg, Enabled: true},
{ID: "default-recently-added-ebooks", Scope: "library", LibraryID: libraryID, Position: 1, SectionType: SectionRecentlyAdded, Title: "Recently Added Ebooks", ItemLimit: 20, Config: defaultMediaScopeConfig("ebook"), Enabled: true},
{ID: "default-recently-released-ebooks", Scope: "library", LibraryID: libraryID, Position: 2, SectionType: SectionRecentlyReleased, Title: "Recently Released Ebooks", ItemLimit: 20, Config: defaultMediaScopeConfig("ebook"), Enabled: true},
{ID: "default-recommended-for-you", Scope: "library", LibraryID: libraryID, Position: 3, SectionType: SectionRecommendedForYou, Title: "Recommended for You", ItemLimit: 20, Config: emptyCfg, Enabled: true},
{ID: "default-random-ebooks", Scope: "library", LibraryID: libraryID, Position: 4, SectionType: SectionRandom, Title: "Random Picks", ItemLimit: 20, Config: defaultMediaScopeConfig("ebook"), Enabled: true},
}
default:
return DefaultLibrarySections(libraryID)
}
+63
View File
@@ -275,6 +275,69 @@ func TestDefaultLibrarySectionsForTypeAudiobooks(t *testing.T) {
})
}
func TestDefaultLibrarySectionsForTypeEbooks(t *testing.T) {
libraryID := 11
got := DefaultLibrarySectionsForType(&libraryID, "ebooks")
if len(got) != 5 {
t.Fatalf("expected 5 ebook default sections, got %d", len(got))
}
tests := []struct {
index int
id string
sectionType SectionType
title string
position int
}{
{index: 0, id: "default-continue-reading", sectionType: SectionContinueWatching, title: "Continue Reading", position: 0},
{index: 1, id: "default-recently-added-ebooks", sectionType: SectionRecentlyAdded, title: "Recently Added Ebooks", position: 1},
{index: 2, id: "default-recently-released-ebooks", sectionType: SectionRecentlyReleased, title: "Recently Released Ebooks", position: 2},
{index: 3, id: "default-recommended-for-you", sectionType: SectionRecommendedForYou, title: "Recommended for You", position: 3},
{index: 4, id: "default-random-ebooks", sectionType: SectionRandom, title: "Random Picks", position: 4},
}
for _, tt := range tests {
section := got[tt.index]
if section.ID != tt.id {
t.Fatalf("section %d id = %q, want %q", tt.index, section.ID, tt.id)
}
if section.SectionType != tt.sectionType {
t.Fatalf("section %d type = %q, want %q", tt.index, section.SectionType, tt.sectionType)
}
if section.Title != tt.title {
t.Fatalf("section %d title = %q, want %q", tt.index, section.Title, tt.title)
}
if section.Position != tt.position {
t.Fatalf("section %d position = %d, want %d", tt.index, section.Position, tt.position)
}
if section.Featured {
t.Fatalf("section %d featured = true, want false", tt.index)
}
}
assertEmptyJSON(t, got[0].Config)
assertQueryDefinition(t, got[1].Config, catalog.QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []catalog.QueryGroup{},
Sort: catalog.QuerySort{Field: "added_at", Order: "desc"},
})
assertQueryDefinition(t, got[2].Config, catalog.QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []catalog.QueryGroup{},
Sort: catalog.QuerySort{Field: "added_at", Order: "desc"},
})
assertEmptyJSON(t, got[3].Config)
assertQueryDefinition(t, got[4].Config, catalog.QueryDefinition{
MediaScope: "ebook",
Match: "all",
Groups: []catalog.QueryGroup{},
Sort: catalog.QuerySort{Field: "added_at", Order: "desc"},
})
}
func TestDefaultLibrarySectionsForTypeMixed(t *testing.T) {
libraryID := 99
got := DefaultLibrarySectionsForType(&libraryID, "mixed")
+75
View File
@@ -371,6 +371,11 @@ func (f *Fetcher) fetchContinueWatchingSection(ctx context.Context, resolved Res
if err != nil {
return SectionWithItems{}, fmt.Errorf("listing progress: %w", err)
}
ebookProgressEntries, err := f.listEbookContinueWatchingProgress(ctx, userID, profileID, limit)
if err != nil {
return SectionWithItems{}, fmt.Errorf("listing ebook reader progress: %w", err)
}
progressEntries = mergeContinueWatchingProgress(progressEntries, ebookProgressEntries, limit)
progressEntries = f.filterContinueWatchingDismissals(ctx, store, profileID, progressEntries)
// Build in-progress items
@@ -484,6 +489,76 @@ func (f *Fetcher) fetchContinueWatchingSection(ctx context.Context, resolved Res
}, nil
}
func (f *Fetcher) listEbookContinueWatchingProgress(ctx context.Context, userID int, profileID string, limit int) ([]userstore.WatchProgress, error) {
if f.pool == nil || userID <= 0 || profileID == "" || limit <= 0 {
return nil, nil
}
rows, err := f.pool.Query(ctx, `
SELECT content_id, progress::double precision, updated_at
FROM ebook_reader_progress
WHERE user_id = $1
AND profile_id = $2
AND progress > 0
AND progress < 0.9
ORDER BY updated_at DESC, content_id ASC
LIMIT $3
`, userID, profileID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
entries := make([]userstore.WatchProgress, 0)
for rows.Next() {
var contentID string
var progress float64
var updatedAt time.Time
if err := rows.Scan(&contentID, &progress, &updatedAt); err != nil {
return nil, err
}
entries = append(entries, userstore.WatchProgress{
MediaItemID: contentID,
PositionSeconds: progress,
DurationSeconds: 1,
UpdatedAt: updatedAt.UTC().Format(time.RFC3339),
})
}
if err := rows.Err(); err != nil {
return nil, err
}
return entries, nil
}
func mergeContinueWatchingProgress(videoEntries, ebookEntries []userstore.WatchProgress, limit int) []userstore.WatchProgress {
if len(ebookEntries) == 0 {
return videoEntries
}
merged := make([]userstore.WatchProgress, 0, len(videoEntries)+len(ebookEntries))
merged = append(merged, videoEntries...)
merged = append(merged, ebookEntries...)
sort.SliceStable(merged, func(i, j int) bool {
left := parseWatchProgressUpdatedAt(merged[i].UpdatedAt)
right := parseWatchProgressUpdatedAt(merged[j].UpdatedAt)
if left.Equal(right) {
return merged[i].MediaItemID < merged[j].MediaItemID
}
return left.After(right)
})
if limit > 0 && len(merged) > limit {
merged = merged[:limit]
}
return merged
}
func parseWatchProgressUpdatedAt(value string) time.Time {
updatedAt, err := time.Parse(time.RFC3339, value)
if err != nil {
return time.Time{}
}
return updatedAt
}
func (f *Fetcher) fetchNextUpSection(ctx context.Context, resolved ResolvedSection, libraryID *int, libraryIDs []int, userID int, profileID string, filter catalog.AccessFilter) (SectionWithItems, error) {
emptyResult := SectionWithItems{
ResolvedSection: resolved,
@@ -107,6 +107,36 @@ func TestFilterSupersededEpisodeProgressEntriesDropsOlderPartialsAfterLaterCompl
}
}
func TestMergeContinueWatchingProgressIncludesEbooksByUpdatedAt(t *testing.T) {
t.Parallel()
videoEntries := []userstore.WatchProgress{
{
MediaItemID: "movie-1",
PositionSeconds: 240,
DurationSeconds: 1200,
UpdatedAt: time.Date(2026, 6, 1, 12, 0, 0, 0, time.UTC).Format(time.RFC3339),
},
}
ebookEntries := []userstore.WatchProgress{
{
MediaItemID: "ebook-1",
PositionSeconds: 0.42,
DurationSeconds: 1,
UpdatedAt: time.Date(2026, 6, 2, 12, 0, 0, 0, time.UTC).Format(time.RFC3339),
},
}
merged := mergeContinueWatchingProgress(videoEntries, ebookEntries, 10)
if got, want := progressIDs(merged), []string{"ebook-1", "movie-1"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("merged IDs = %v, want %v", got, want)
}
if merged[0].PositionSeconds != 0.42 || merged[0].DurationSeconds != 1 {
t.Fatalf("ebook progress = %+v, want normalized reader progress", merged[0])
}
}
func TestCompletedProgressSnapshotsPagesThroughConfiguredStore(t *testing.T) {
t.Parallel()
@@ -171,6 +201,14 @@ func contentIDs(items []*models.MediaItem) []string {
return ids
}
func progressIDs(entries []userstore.WatchProgress) []string {
ids := make([]string, 0, len(entries))
for _, entry := range entries {
ids = append(ids, entry.MediaItemID)
}
return ids
}
func intPtr(v int) *int {
return &v
}
@@ -0,0 +1,56 @@
package tasks
import (
"context"
"encoding/json"
"fmt"
"github.com/Silo-Server/silo-server/internal/taskmanager"
)
type ebookMetadataEnricher interface {
Run(ctx context.Context) (int, error)
}
// SyncEbookMetadataTask runs the periodic ebook enrichment sweep.
// It calls ebooks.Enricher.Run() which selects unenriched ebook media_items,
// resolves the per-folder metadata-provider chain at content_level='ebook',
// and writes results back to the database.
type SyncEbookMetadataTask struct {
enricher ebookMetadataEnricher
}
// NewSyncEbookMetadataTask constructs the task.
func NewSyncEbookMetadataTask(enricher ebookMetadataEnricher) *SyncEbookMetadataTask {
return &SyncEbookMetadataTask{enricher: enricher}
}
func (t *SyncEbookMetadataTask) Key() string { return "sync_ebook_metadata" }
func (t *SyncEbookMetadataTask) Name() string { return "Sync Ebook Metadata" }
func (t *SyncEbookMetadataTask) Description() string {
return "Fetches metadata (cover art, overview, authors) for ebooks that have not yet been enriched"
}
func (t *SyncEbookMetadataTask) Category() taskmanager.TaskCategory {
return taskmanager.TaskCategoryMetadata
}
func (t *SyncEbookMetadataTask) IsHidden() bool { return false }
func (t *SyncEbookMetadataTask) DefaultTriggers() []taskmanager.TriggerConfig {
return []taskmanager.TriggerConfig{
{Type: taskmanager.TriggerTypeInterval, IntervalMs: 5 * 60 * 1000},
}
}
func (t *SyncEbookMetadataTask) Execute(ctx context.Context, progress taskmanager.ProgressReporter) error {
progress.Report(0, "Scanning for unenriched ebooks")
enriched, err := t.enricher.Run(ctx)
if err != nil {
return fmt.Errorf("ebook metadata sync: %w", err)
}
result, _ := json.Marshal(map[string]int{"items_enriched": enriched})
progress.SetResultData(result)
progress.Report(100, fmt.Sprintf("Ebook metadata sync complete (%d items enriched)", enriched))
return nil
}
@@ -0,0 +1,92 @@
package tasks
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/Silo-Server/silo-server/internal/taskmanager"
)
type fakeEbookMetadataEnricher struct {
enriched int
err error
called bool
}
func (f *fakeEbookMetadataEnricher) Run(context.Context) (int, error) {
f.called = true
return f.enriched, f.err
}
type ebookMetadataProgressReporter struct {
messages []string
result json.RawMessage
}
func (p *ebookMetadataProgressReporter) Report(_ float64, message string) {
p.messages = append(p.messages, message)
}
func (p *ebookMetadataProgressReporter) SetResultData(data json.RawMessage) {
p.result = data
}
func TestSyncEbookMetadataTaskProperties(t *testing.T) {
task := NewSyncEbookMetadataTask(&fakeEbookMetadataEnricher{})
if task.Key() != "sync_ebook_metadata" {
t.Fatalf("Key() = %q, want sync_ebook_metadata", task.Key())
}
if task.Name() != "Sync Ebook Metadata" {
t.Fatalf("Name() = %q, want Sync Ebook Metadata", task.Name())
}
if task.Category() != taskmanager.TaskCategoryMetadata {
t.Fatalf("Category() = %q, want %q", task.Category(), taskmanager.TaskCategoryMetadata)
}
if task.IsHidden() {
t.Fatal("IsHidden() = true, want false")
}
triggers := task.DefaultTriggers()
if len(triggers) != 1 || triggers[0].Type != taskmanager.TriggerTypeInterval || triggers[0].IntervalMs != 5*60*1000 {
t.Fatalf("DefaultTriggers() = %#v, want one 5 minute interval", triggers)
}
if strings.Contains(strings.ToLower(task.Description()), "narrator") {
t.Fatalf("Description() mentions narrator: %q", task.Description())
}
}
func TestSyncEbookMetadataTaskExecuteReportsResult(t *testing.T) {
enricher := &fakeEbookMetadataEnricher{enriched: 3}
task := NewSyncEbookMetadataTask(enricher)
progress := &ebookMetadataProgressReporter{}
if err := task.Execute(context.Background(), progress); err != nil {
t.Fatalf("Execute() error = %v", err)
}
if !enricher.called {
t.Fatal("enricher was not called")
}
var result map[string]int
if err := json.Unmarshal(progress.result, &result); err != nil {
t.Fatalf("result data JSON error: %v", err)
}
if result["items_enriched"] != 3 {
t.Fatalf("items_enriched = %d, want 3", result["items_enriched"])
}
if len(progress.messages) == 0 || !strings.Contains(progress.messages[len(progress.messages)-1], "3 items enriched") {
t.Fatalf("progress messages = %#v, want completion count", progress.messages)
}
}
func TestSyncEbookMetadataTaskExecuteWrapsError(t *testing.T) {
enricher := &fakeEbookMetadataEnricher{err: errors.New("boom")}
task := NewSyncEbookMetadataTask(enricher)
err := task.Execute(context.Background(), &ebookMetadataProgressReporter{})
if err == nil || !strings.Contains(err.Error(), "ebook metadata sync") {
t.Fatalf("Execute() error = %v, want wrapped ebook metadata sync error", err)
}
}
@@ -0,0 +1,17 @@
-- +goose Up
-- +goose StatementBegin
CREATE TABLE ebook_series (
content_id TEXT PRIMARY KEY REFERENCES media_items(content_id) ON DELETE CASCADE,
series_name TEXT NOT NULL,
series_index NUMERIC,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX ebook_series_name_lower
ON ebook_series (LOWER(series_name), series_index NULLS LAST);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS ebook_series;
-- +goose StatementEnd
@@ -0,0 +1,22 @@
-- +goose Up
-- +goose StatementBegin
CREATE TABLE ebook_reader_progress (
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
profile_id TEXT NOT NULL,
content_id TEXT NOT NULL REFERENCES media_items(content_id) ON DELETE CASCADE,
file_id INTEGER NOT NULL REFERENCES media_files(id) ON DELETE CASCADE,
location TEXT NOT NULL,
progress DOUBLE PRECISION NOT NULL DEFAULT 0,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, profile_id, content_id),
CHECK (progress >= 0 AND progress <= 1)
);
CREATE INDEX ebook_reader_progress_profile_updated
ON ebook_reader_progress (user_id, profile_id, updated_at DESC);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS ebook_reader_progress;
-- +goose StatementEnd
+4
View File
@@ -23,15 +23,19 @@
"@fontsource/noto-sans-thai": "^5.2.8",
"@tanstack/react-query": "^5.90.21",
"@tanstack/react-virtual": "^3.13.19",
"@zip.js/zip.js": "^2.8.26",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"embla-carousel": "^8.6.0",
"embla-carousel-react": "^8.6.0",
"embla-carousel-wheel-gestures": "^8.1.0",
"foliate-js": "file:vendor/foliate-js",
"framer-motion": "^12.38.0",
"hls.js": "^1.6.15",
"jassub": "^2.4.2",
"lucide-react": "^0.576.0",
"node-unrar-js": "^2.0.2",
"pdfjs-dist": "^5.7.284",
"radix-ui": "^1.4.3",
"react": "^19.0.0",
"react-dom": "^19.0.0",
+167
View File
@@ -37,6 +37,9 @@ importers:
'@tanstack/react-virtual':
specifier: ^3.13.19
version: 3.13.22(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
'@zip.js/zip.js':
specifier: ^2.8.26
version: 2.8.26
class-variance-authority:
specifier: ^0.7.1
version: 0.7.1
@@ -52,6 +55,9 @@ importers:
embla-carousel-wheel-gestures:
specifier: ^8.1.0
version: 8.1.0(embla-carousel@8.6.0)
foliate-js:
specifier: file:vendor/foliate-js
version: file:vendor/foliate-js
framer-motion:
specifier: ^12.38.0
version: 12.38.0(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -64,6 +70,12 @@ importers:
lucide-react:
specifier: ^0.576.0
version: 0.576.0(react@19.2.4)
node-unrar-js:
specifier: ^2.0.2
version: 2.0.2
pdfjs-dist:
specifier: ^5.7.284
version: 5.7.284
radix-ui:
specifier: ^1.4.3
version: 1.4.3(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -586,6 +598,81 @@ packages:
'@jridgewell/trace-mapping@0.3.31':
resolution: {integrity: sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw==}
'@napi-rs/canvas-android-arm64@0.1.100':
resolution: {integrity: sha512-hjhCKhntPv9+t4ckHymdx0phYNcVW+GKQR6Lzw2zE+pOVjOplSmtx9nNNknTjbEDLcuLZqA1y8ufKg1XfgftzQ==}
engines: {node: '>= 10'}
cpu: [arm64]
os: [android]
'@napi-rs/canvas-darwin-arm64@0.1.100':
resolution: {integrity: sha512-2PcswRaC7Ly645DGt88///zuFDhJxJYdKAs1uU3mfk1atYkXufgcgLfBpk6Tm12nCQBaNt1wpybuPZ4qOhTo8A==}
engines: {node: '>= 10'}
cpu: [arm64]
os: [darwin]
'@napi-rs/canvas-darwin-x64@0.1.100':
resolution: {integrity: sha512-ePNZtj7pNIva/siZMg+HmbeozkIjqUIYdoymH8HaA3qK7LfzFN4WMBM8G6HQ9ZC+H3+Dnn5pqtiXpgLykaPOhw==}
engines: {node: '>= 10'}
cpu: [x64]
os: [darwin]
'@napi-rs/canvas-linux-arm-gnueabihf@0.1.100':
resolution: {integrity: sha512-d5cDB48oWFGU8/XPhUOFAlySgb/VAu7D+s8fi55K1Pcfg8aPplHWqMgibhVLU8ky7Pyg/fuiVLz4Nf3JrSTuUA==}
engines: {node: '>= 10'}
cpu: [arm]
os: [linux]
'@napi-rs/canvas-linux-arm64-gnu@0.1.100':
resolution: {integrity: sha512-rDxgxRu69RvDlX/bh9o22DxLsGr8EqsNgotL9+RwQE1S0b0cqeatqsw6aW45mukm0B42DIAaAacKaYQ8cqS1nw==}
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@napi-rs/canvas-linux-arm64-musl@0.1.100':
resolution: {integrity: sha512-K3mDW66N+xT2/V439u1alFANiBUjdEx2gLiNYnCmUsva5jZMxWTjafBYwTzYK+EMFMHrUoabuU+T1BIP5CgbYQ==}
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [musl]
'@napi-rs/canvas-linux-riscv64-gnu@0.1.100':
resolution: {integrity: sha512-mooqUBTIsccZpnoQC4NgrC1v6C1vof39etLNMnBwCY+p0gajWJvAHLGQ6g/gGyS5YrpDW+GefSN4+Cvcr08UWw==}
engines: {node: '>= 10'}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@napi-rs/canvas-linux-x64-gnu@0.1.100':
resolution: {integrity: sha512-1eCvkDCazm7FFhsT7DfGOdSaHgZVK3bt/dSBl5EWHOWmnz+I7j8tPseJqqD81NF+MH21jKUK4wQSDjN0mdhnTg==}
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [glibc]
'@napi-rs/canvas-linux-x64-musl@0.1.100':
resolution: {integrity: sha512-20arT6lnI19S68qNlii73TSEDbECNgzMz2EpldC1V3mZFuRkeujXkcebRk0LRJe9SEUAooYiLokfMViY8IX7yA==}
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [musl]
'@napi-rs/canvas-win32-arm64-msvc@0.1.100':
resolution: {integrity: sha512-DZFFT1wIAg37LJw37yhMRFfjATd3vTQzjZ1Yki8u2vhO6Hi5VE6BVaGQ1aaDu7xb4iMErz+9EOwjpS7xcxFeBw==}
engines: {node: '>= 10'}
cpu: [arm64]
os: [win32]
'@napi-rs/canvas-win32-x64-msvc@0.1.100':
resolution: {integrity: sha512-MyT1j3mHC2+Lu4pBi9mKyMJhtP6U7k7EldY7sj/uS5gJA65gTXt8MefJQXLJo5d/vZbuWmfxzkEUNc/urV3pHA==}
engines: {node: '>= 10'}
cpu: [x64]
os: [win32]
'@napi-rs/canvas@0.1.100':
resolution: {integrity: sha512-xglYA6q3XO5P3BNJYxVZ1IV7DLVjp1Py6nwag88YntrS+3vKHyYcMqXVS4ZztJmwz2uGvz1FWhI/4LgbR5uQDA==}
engines: {node: '>= 10'}
'@radix-ui/number@1.1.1':
resolution: {integrity: sha512-MkKCwxlXTgz6CFoJx3pCwn07GKp36+aZyu/u2Ln2VrA5DcdyCZkASEDBTd8x5whTQQL5CiYf4prXKLcgQdv29g==}
@@ -1695,6 +1782,10 @@ packages:
'@vitest/utils@4.1.0':
resolution: {integrity: sha512-XfPXT6a8TZY3dcGY8EdwsBulFCIw+BeeX0RZn2x/BtiY/75YGh8FeWGG8QISN/WhaqSrE2OrlDgtF8q5uhOTmw==}
'@zip.js/zip.js@2.8.26':
resolution: {integrity: sha512-RQ4h9F6DOiHxpdocUDrOl6xBM+yOtz+LkUol47AVWcfebGBDpZ7w7Xvz9PS24JgXvLGiXXzSAfdCdVy1tPlaFA==}
engines: {bun: '>=0.7.0', deno: '>=1.0.0', node: '>=18.0.0'}
abslink@1.1.6:
resolution: {integrity: sha512-8fQgnUoVSgc1IhOrYzdDY+wTDPktbuYjds2LKf9kWYWKwDnHgXU168gdV3sPei7vevFf5m2fPZ6IQSKZkSjHVg==}
@@ -1800,6 +1891,9 @@ packages:
concat-map@0.0.1:
resolution: {integrity: sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==}
construct-style-sheets-polyfill@3.1.0:
resolution: {integrity: sha512-HBLKP0chz8BAY6rBdzda11c3wAZeCZ+kIG4weVC2NM3AXzxx09nhe8t0SQNdloAvg5GLuHwq/0SPOOSPvtCcKw==}
convert-source-map@2.0.0:
resolution: {integrity: sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==}
@@ -2006,6 +2100,9 @@ packages:
flatted@3.4.2:
resolution: {integrity: sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA==}
foliate-js@file:vendor/foliate-js:
resolution: {directory: vendor/foliate-js, type: directory}
framer-motion@12.38.0:
resolution: {integrity: sha512-rFYkY/pigbcswl1XQSb7q424kSTQ8q6eAC+YUsSKooHQYuLdzdHjrt6uxUC+PRAO++q5IS7+TamgIw1AphxR+g==}
peerDependencies:
@@ -2359,6 +2456,10 @@ packages:
node-releases@2.0.36:
resolution: {integrity: sha512-TdC8FSgHz8Mwtw9g5L4gR/Sh9XhSP/0DEkQxfEFXOpiul5IiHgHan2VhYYb6agDSfp4KuvltmGApc8HMgUrIkA==}
node-unrar-js@2.0.2:
resolution: {integrity: sha512-hLNmoJzqaKJnod8yiTVGe9hnlNRHotUi0CreSv/8HtfRi/3JnRC8DvsmKfeGGguRjTEulhZK6zXX5PXoVuDZ2w==}
engines: {node: '>=10.0.0'}
object-assign@4.1.1:
resolution: {integrity: sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==}
engines: {node: '>=0.10.0'}
@@ -2396,6 +2497,10 @@ packages:
pathe@2.0.3:
resolution: {integrity: sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==}
pdfjs-dist@5.7.284:
resolution: {integrity: sha512-h4EdYQczmGhbOlqc3PPZwxevn7ApdWPbovAuWXOB/DjIyigSnwfy2oze7c6mRcSr9XgLp3eN3EeL4DyySTPMFw==}
engines: {node: '>=22.13.0 || >=24'}
picocolors@1.1.1:
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
@@ -3271,6 +3376,54 @@ snapshots:
'@jridgewell/resolve-uri': 3.1.2
'@jridgewell/sourcemap-codec': 1.5.5
'@napi-rs/canvas-android-arm64@0.1.100':
optional: true
'@napi-rs/canvas-darwin-arm64@0.1.100':
optional: true
'@napi-rs/canvas-darwin-x64@0.1.100':
optional: true
'@napi-rs/canvas-linux-arm-gnueabihf@0.1.100':
optional: true
'@napi-rs/canvas-linux-arm64-gnu@0.1.100':
optional: true
'@napi-rs/canvas-linux-arm64-musl@0.1.100':
optional: true
'@napi-rs/canvas-linux-riscv64-gnu@0.1.100':
optional: true
'@napi-rs/canvas-linux-x64-gnu@0.1.100':
optional: true
'@napi-rs/canvas-linux-x64-musl@0.1.100':
optional: true
'@napi-rs/canvas-win32-arm64-msvc@0.1.100':
optional: true
'@napi-rs/canvas-win32-x64-msvc@0.1.100':
optional: true
'@napi-rs/canvas@0.1.100':
optionalDependencies:
'@napi-rs/canvas-android-arm64': 0.1.100
'@napi-rs/canvas-darwin-arm64': 0.1.100
'@napi-rs/canvas-darwin-x64': 0.1.100
'@napi-rs/canvas-linux-arm-gnueabihf': 0.1.100
'@napi-rs/canvas-linux-arm64-gnu': 0.1.100
'@napi-rs/canvas-linux-arm64-musl': 0.1.100
'@napi-rs/canvas-linux-riscv64-gnu': 0.1.100
'@napi-rs/canvas-linux-x64-gnu': 0.1.100
'@napi-rs/canvas-linux-x64-musl': 0.1.100
'@napi-rs/canvas-win32-arm64-msvc': 0.1.100
'@napi-rs/canvas-win32-x64-msvc': 0.1.100
optional: true
'@radix-ui/number@1.1.1': {}
'@radix-ui/primitive@1.1.3': {}
@@ -4406,6 +4559,8 @@ snapshots:
convert-source-map: 2.0.0
tinyrainbow: 3.1.0
'@zip.js/zip.js@2.8.26': {}
abslink@1.1.6: {}
acorn-jsx@5.3.2(acorn@8.16.0):
@@ -4495,6 +4650,8 @@ snapshots:
concat-map@0.0.1: {}
construct-style-sheets-polyfill@3.1.0: {}
convert-source-map@2.0.0: {}
cookie@1.1.1: {}
@@ -4721,6 +4878,10 @@ snapshots:
flatted@3.4.2: {}
foliate-js@file:vendor/foliate-js:
dependencies:
construct-style-sheets-polyfill: 3.1.0
framer-motion@12.38.0(react-dom@19.2.4(react@19.2.4))(react@19.2.4):
dependencies:
motion-dom: 12.38.0
@@ -4997,6 +5158,8 @@ snapshots:
node-releases@2.0.36: {}
node-unrar-js@2.0.2: {}
object-assign@4.1.1: {}
obug@2.1.1: {}
@@ -5032,6 +5195,10 @@ snapshots:
pathe@2.0.3: {}
pdfjs-dist@5.7.284:
optionalDependencies:
'@napi-rs/canvas': 0.1.100
picocolors@1.1.1: {}
picomatch@4.0.4: {}
+407
View File
@@ -0,0 +1,407 @@
/* Copyright 2014 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
.annotationLayer {
color-scheme: only light;
--annotation-unfocused-field-background: url("data:image/svg+xml;charset=UTF-8,<svg width='1px' height='1px' xmlns='http://www.w3.org/2000/svg'><rect width='100%' height='100%' style='fill:rgba(0, 54, 255, 0.13);'/></svg>");
--input-focus-border-color: Highlight;
--input-focus-outline: 1px solid Canvas;
--input-unfocused-border-color: transparent;
--input-disabled-border-color: transparent;
--input-hover-border-color: black;
--link-outline: none;
@media screen and (forced-colors: active) {
--input-focus-border-color: CanvasText;
--input-unfocused-border-color: ActiveText;
--input-disabled-border-color: GrayText;
--input-hover-border-color: Highlight;
--link-outline: 1.5px solid LinkText;
.textWidgetAnnotation :is(input, textarea):required,
.choiceWidgetAnnotation select:required,
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input:required {
outline: 1.5px solid selectedItem;
}
.linkAnnotation {
outline: var(--link-outline);
&:hover {
backdrop-filter: var(--hcm-highlight-filter);
}
& > a:hover {
opacity: 0 !important;
background: none !important;
box-shadow: none;
}
}
.popupAnnotation .popup {
outline: calc(1.5px * var(--total-scale-factor)) solid CanvasText !important;
background-color: ButtonFace !important;
color: ButtonText !important;
}
.highlightArea:hover::after {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
backdrop-filter: var(--hcm-highlight-filter);
content: "";
pointer-events: none;
}
.popupAnnotation.focused .popup {
outline: calc(3px * var(--total-scale-factor)) solid Highlight !important;
}
}
position: absolute;
top: 0;
left: 0;
pointer-events: none;
transform-origin: 0 0;
&[data-main-rotation="90"] .norotate {
transform: rotate(270deg) translateX(-100%);
}
&[data-main-rotation="180"] .norotate {
transform: rotate(180deg) translate(-100%, -100%);
}
&[data-main-rotation="270"] .norotate {
transform: rotate(90deg) translateY(-100%);
}
&.disabled {
section,
.popup {
pointer-events: none;
}
}
.annotationContent {
position: absolute;
width: 100%;
height: 100%;
pointer-events: none;
&.freetext {
background: transparent;
border: none;
inset: 0;
overflow: visible;
white-space: nowrap;
font: 10px sans-serif;
line-height: 1.35;
}
}
section {
position: absolute;
text-align: initial;
pointer-events: auto;
box-sizing: border-box;
transform-origin: 0 0;
user-select: none;
&:has(div.annotationContent) {
canvas.annotationContent {
display: none;
}
}
.overlaidText {
position: absolute;
top: 0;
left: 0;
width: 0;
height: 0;
display: inline-block;
overflow: hidden;
}
}
.textLayer.selecting ~ & section {
pointer-events: none;
}
:is(.linkAnnotation, .buttonWidgetAnnotation.pushButton) > a {
position: absolute;
font-size: 1em;
top: 0;
left: 0;
width: 100%;
height: 100%;
}
:is(.linkAnnotation, .buttonWidgetAnnotation.pushButton):not(.hasBorder)
> a:hover {
opacity: 0.2;
background-color: rgb(255 255 0);
}
.linkAnnotation.hasBorder:hover {
background-color: rgb(255 255 0 / 0.2);
}
.hasBorder {
background-size: 100% 100%;
}
.textAnnotation img {
position: absolute;
cursor: pointer;
width: 100%;
height: 100%;
top: 0;
left: 0;
}
.textWidgetAnnotation :is(input, textarea),
.choiceWidgetAnnotation select,
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input {
background-image: var(--annotation-unfocused-field-background);
border: 2px solid var(--input-unfocused-border-color);
box-sizing: border-box;
font: calc(9px * var(--total-scale-factor)) sans-serif;
height: 100%;
margin: 0;
vertical-align: top;
width: 100%;
}
.textWidgetAnnotation :is(input, textarea):required,
.choiceWidgetAnnotation select:required,
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input:required {
outline: 1.5px solid red;
}
.choiceWidgetAnnotation select option {
padding: 0;
}
.buttonWidgetAnnotation.radioButton input {
border-radius: 50%;
}
.textWidgetAnnotation textarea {
resize: none;
}
.textWidgetAnnotation :is(input, textarea)[disabled],
.choiceWidgetAnnotation select[disabled],
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input[disabled] {
background: none;
border: 2px solid var(--input-disabled-border-color);
cursor: not-allowed;
}
.textWidgetAnnotation :is(input, textarea):hover,
.choiceWidgetAnnotation select:hover,
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input:hover {
border: 2px solid var(--input-hover-border-color);
}
.textWidgetAnnotation :is(input, textarea):hover,
.choiceWidgetAnnotation select:hover,
.buttonWidgetAnnotation.checkBox input:hover {
border-radius: 2px;
}
.textWidgetAnnotation :is(input, textarea):focus,
.choiceWidgetAnnotation select:focus {
background: none;
border: 2px solid var(--input-focus-border-color);
border-radius: 2px;
outline: var(--input-focus-outline);
}
.buttonWidgetAnnotation:is(.checkBox, .radioButton) :focus {
background-image: none;
background-color: transparent;
}
.buttonWidgetAnnotation.checkBox :focus {
border: 2px solid var(--input-focus-border-color);
border-radius: 2px;
outline: var(--input-focus-outline);
}
.buttonWidgetAnnotation.radioButton :focus {
border: 2px solid var(--input-focus-border-color);
outline: var(--input-focus-outline);
}
.buttonWidgetAnnotation.checkBox input:checked::before,
.buttonWidgetAnnotation.checkBox input:checked::after,
.buttonWidgetAnnotation.radioButton input:checked::before {
background-color: CanvasText;
content: "";
display: block;
position: absolute;
}
.buttonWidgetAnnotation.checkBox input:checked::before,
.buttonWidgetAnnotation.checkBox input:checked::after {
height: 80%;
left: 45%;
width: 1px;
}
.buttonWidgetAnnotation.checkBox input:checked::before {
transform: rotate(45deg);
}
.buttonWidgetAnnotation.checkBox input:checked::after {
transform: rotate(-45deg);
}
.buttonWidgetAnnotation.radioButton input:checked::before {
border-radius: 50%;
height: 50%;
left: 25%;
top: 25%;
width: 50%;
}
.textWidgetAnnotation input.comb {
font-family: monospace;
padding-left: 2px;
padding-right: 0;
}
.textWidgetAnnotation input.comb:focus {
/*
* Letter spacing is placed on the right side of each character. Hence, the
* letter spacing of the last character may be placed outside the visible
* area, causing horizontal scrolling. We avoid this by extending the width
* when the element has focus and revert this when it loses focus.
*/
width: 103%;
}
.buttonWidgetAnnotation:is(.checkBox, .radioButton) input {
appearance: none;
}
.fileAttachmentAnnotation .popupTriggerArea {
height: 100%;
width: 100%;
}
.popupAnnotation {
position: absolute;
font-size: calc(9px * var(--total-scale-factor));
pointer-events: none;
width: max-content;
max-width: 45%;
height: auto;
}
.popup {
background-color: rgb(255 255 153);
color: black;
box-shadow: 0 calc(2px * var(--total-scale-factor))
calc(5px * var(--total-scale-factor)) rgb(136 136 136);
border-radius: calc(2px * var(--total-scale-factor));
outline: 1.5px solid rgb(255 255 74);
padding: calc(6px * var(--total-scale-factor));
cursor: pointer;
font: message-box;
white-space: normal;
word-wrap: break-word;
pointer-events: auto;
user-select: text;
}
.popupAnnotation.focused .popup {
outline-width: 3px;
}
.popup * {
font-size: calc(9px * var(--total-scale-factor));
}
.popup > .header {
display: inline-block;
}
.popup > .header > .title {
display: inline;
font-weight: bold;
}
.popup > .header .popupDate {
display: inline-block;
margin-left: calc(5px * var(--total-scale-factor));
width: fit-content;
}
.popupContent {
border-top: 1px solid rgb(51 51 51);
margin-top: calc(2px * var(--total-scale-factor));
padding-top: calc(2px * var(--total-scale-factor));
}
.richText > * {
white-space: pre-wrap;
font-size: calc(9px * var(--total-scale-factor));
}
.popupTriggerArea {
cursor: pointer;
&:hover {
backdrop-filter: var(--hcm-highlight-filter);
}
}
section svg {
position: absolute;
width: 100%;
height: 100%;
top: 0;
left: 0;
}
.annotationTextContent {
position: absolute;
width: 100%;
height: 100%;
opacity: 0;
color: transparent;
user-select: none;
pointer-events: none;
span {
width: 100%;
display: inline-block;
}
}
svg.quadrilateralsContainer {
contain: strict;
width: 0;
height: 0;
position: absolute;
top: 0;
left: 0;
z-index: -1;
}
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More