* docs: design + plan for shared AI core, metadata translation, Whisper ASR Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): shared LLM client, segment translator, and job runner packages internal/ai/llm: OpenAI-compatible chat client moved out of subtitles/ai, plus /v1/audio/transcriptions (verbose_json) for the ASR work; one shared retry/backoff loop for both. internal/ai/translate: the batched indexed-JSON translation protocol generalized to text segments. internal/ai/jobrunner: dispatch/heartbeat/reaper/cancel lifecycle extracted behind a minimal store interface, with a semaphore shareable across job services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(subtitles): consume shared AI core LLMTranslator becomes a thin cue<->segment adapter over aitranslate; the service delegates dispatch/heartbeat/reaper/cancel to jobrunner; the local OpenAI client is gone in favor of internal/ai/llm. Behavior (prompts, wire protocol, job rows, recovery semantics) is unchanged. NewService now takes the dispatch semaphore so all AI job services can share one bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): shared ai.* settings, metadata translation job table, localization provenance columns ai.* connection keys (chat + optional separate ASR endpoint) load with a fallback to the legacy subtitle_ai.* rows — those are never renamed in SQL because encrypted values are GCM-bound to their setting key. New toggles: subtitle_ai.transcribe_enabled, metadata_ai.enabled. Migration adds metadata_translation_jobs, per-field provenance (provider|ai|manual) on the localization tables, and media_folders.auto_translate_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): localization field provenance with provider/ai/manual precedence Provider upserts keep manual values and never blank a field with an empty incoming value; new UpsertAITranslation/UpsertAIOverview methods write AI fields only over empty or ai-sourced values (force adds provider, never manual) — all enforced in single-statement SQL. Serving now merges only non-empty localized fields onto the base item, since localization rows are legitimately partial (AI rows carry no titles/artwork). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(metadata): AI translation service, refresh auto-fallback, and admin API internal/metadata/translation: job service over the shared AI core that expands an item to its season/episode overviews, skips already-localized fields (zero model calls on repeat runs), batches paragraphs through the generic translator, and persists per batch with provenance-aware upserts. MetadataService gains an AutoTranslator seam invoked after each refresh for libraries with auto_translate_metadata. Admin endpoints under the metadata curation guard: enqueue, list (poll), cancel; plus a status probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(subtitles): Whisper ASR transcribe and transcribe_translate jobs New WhisperTranscriber: one ffmpeg pass extracts the audio track to 10-min 16kHz mono WAV chunks (temp dir cleaned on every exit path), each chunk goes to the OpenAI-compatible /v1/audio/transcriptions endpoint (verbose_json, per-request timeout sized to 3x chunk duration), segment timestamps are offset and built into wrapped cues. Chunks process playhead-first and stream live to the requesting session. The transcript is stored as an ordinary downloaded subtitle (provider 'transcribed'); transcribe_translate chains the existing translator and stores the translated track as the job result. Enqueue accepts an optional kind; status reports transcribe_enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): AI services settings, metadata translate action, library auto-translate, generate-from-audio New AI Services admin page hosts the shared endpoint config (reads fall back to legacy subtitle_ai.* values, writes target ai.*) and the three feature toggles; the AI card moves out of Subtitles settings. The metadata editor gains a Translate-with-AI panel with job polling and force/re-translate. The library form gains the auto-translate toggle (threaded through the libraries API). The player translate modal gains a From-audio mode that lists audio tracks and submits transcribe / transcribe_translate jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt import grouping in router and translation tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): per-profile metadata language and viewer-triggered description translation user_profiles.preferred_metadata_language threads through the access scope into catalog serving: presentation language now resolves explicit param -> profile preference -> library metadata language (native API and jellycompat). ItemDetail gains pending_translation_language when the viewer's language is missing a localized overview. New metadata_ai.on_view setting (off|button| auto) gates POST /items/{id}/translate-description: any profile with item access may request its language, with in-flight dedup and a 15-minute failure cooldown so page views never hammer a broken endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): on-view description translation with per-profile metadata language Profile playback settings gain a Metadata language picker (library default inherit). Detail pages: when the server reports pending_translation_language and metadata_ai.on_view is 'auto', the description translates on view with a pulse animation until the refetched detail comes back localized (45s timeout); in 'button' mode a small Translate chip triggers the same flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): expose metadata_ai.on_view in AI Services settings The on-view translation mode had no UI control, so it could only ever be 'off' — viewers got neither the auto translation nor the fallback button. Adds the off/button/auto selector to the Features card, and the config loader now warns and falls back to 'off' on a bad row instead of refusing to start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): clear configuration hint when the transcription endpoint is chat-only A blank Transcription base URL falls back to the chat endpoint; chat-only gateways reject the multipart upload with an opaque 400 that reads like a pipeline bug. 400/404/405 transcription failures now carry a hint to set a Whisper-compatible endpoint in AI Services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): wrap ASR cue text by rune count, not bytes Arabic/Cyrillic/Greek text is 2+ bytes per character in UTF-8, so byte-based wrapping broke lines at roughly half the intended visual width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(web): steer transcription base URL hint away from chat-only gateways Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): block chat-only gateways for transcription, add endpoint presets llm.IsChatOnlyGateway (OpenRouter et al — no timestamped transcription API) is enforced in three layers: the settings API rejects ai.asr_base_url values pointing at one, the router disables ASR with a warning when the blank-URL fallback would land on one, and llm.Transcribe refuses outright. The AI Services page gains one-click transcription presets (Groq turbo/accurate, OpenAI, self-hosted speaches) plus the mirrored client-side check, and the settings API now also validates metadata_ai.on_view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): tighten ASR subtitle sync Three systematic timing-error sources addressed: cue offsets now use the segment muxer's exact per-chunk start times (segment_list CSV) instead of assuming index*chunk_seconds; the audio stream's start delay relative to the container timeline (common in TS remuxes) is probed via ffprobe and added to every cue; and the chunk length is now operator-tunable via subtitle_ai.asr_chunk_seconds (60-600s, default 600) since shorter chunks bound Whisper's within-chunk timestamp drift. Playhead-first ordering now pivots on real chunk starts, and a beyond-end playhead starts at the final chunk instead of restarting from zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): tolerate base URLs that already include the /v1 segment Providers like DeepInfra expose their OpenAI-compatible API under a base that contains the version segment (api.deepinfra.com/v1/openai); always appending /v1/... mangled those. endpointURL now appends bare paths when the base already carries /v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): prefer self-hosted transcription in presets and hints Preset order becomes self-hosted (recommended) -> Groq turbo -> Groq large-v3 -> OpenAI, and the settings hint plus the job-error hint lead with the self-hosted option. The self-hosted preset now fills the turbo CT2 model to match the recommended speaches setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): request VAD and word timestamps for ASR cue accuracy Without vad_filter, faster-whisper servers report wall-to-wall segment times: cues linger on screen through silence (verified up to 91s) and paragraph-length segments become single 400+ char cues. Request vad_filter=true (skipped for hosted providers that reject non-OpenAI fields and run VAD server-side) plus timestamp_granularities word+segment, and rebuild cues from word timings: split at speech pauses, sentence ends, text capacity, and a 7s max duration; cap word-less segments instead of trusting their reported end; stretch sub-second cues to a readable minimum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
326 lines
10 KiB
Go
326 lines
10 KiB
Go
package pgstore
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
"golang.org/x/crypto/bcrypt"
|
|
|
|
"github.com/Silo-Server/silo-server/internal/userstore"
|
|
)
|
|
|
|
// scanProfile scans a profile row, converting TIMESTAMPTZ to string.
|
|
func scanProfile(scanner interface {
|
|
Scan(dest ...any) error
|
|
}) (*userstore.Profile, error) {
|
|
var p userstore.Profile
|
|
var createdAt, updatedAt time.Time
|
|
err := scanner.Scan(
|
|
&p.ID, &p.Name, &p.Avatar, &p.PINHash, &p.IsChild, &p.IsPrimary, &p.MaxContentRating,
|
|
&p.QualityPreference, &p.Language, &p.PreferredMetadataLanguage, &p.SubtitleLanguage, &p.SubtitleMode,
|
|
&p.AutoSkipIntro, &p.AutoSkipCredits, &p.AutoSkipRecap, &p.AutoPlayNextPreview,
|
|
&p.LibraryRestrictionsEnabled,
|
|
&p.ShowForcedSubtitles, &p.MaxPlaybackQuality, &createdAt, &updatedAt,
|
|
)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
p.CreatedAt = timeToString(createdAt)
|
|
p.UpdatedAt = timeToString(updatedAt)
|
|
return &p, nil
|
|
}
|
|
|
|
func (s *PostgresUserStore) CreateProfile(ctx context.Context, p userstore.Profile) error {
|
|
if p.ID == "" {
|
|
p.ID = generateUUID()
|
|
}
|
|
now := nowUTC()
|
|
if p.CreatedAt == "" {
|
|
p.CreatedAt = now
|
|
}
|
|
if p.UpdatedAt == "" {
|
|
p.UpdatedAt = now
|
|
}
|
|
if !p.ShowForcedSubtitles {
|
|
p.ShowForcedSubtitles = true
|
|
}
|
|
|
|
// The first profile created for a user becomes the primary, giving it
|
|
// rights to manage the household's other profiles without requiring a
|
|
// server-wide admin role.
|
|
var hasExisting bool
|
|
if err := s.pool.QueryRow(ctx,
|
|
"SELECT EXISTS(SELECT 1 FROM user_profiles WHERE user_id = $1)", s.userID,
|
|
).Scan(&hasExisting); err != nil {
|
|
return fmt.Errorf("checking existing profiles for user %d: %w", s.userID, err)
|
|
}
|
|
if !hasExisting {
|
|
p.IsPrimary = true
|
|
} else {
|
|
p.IsPrimary = false
|
|
}
|
|
|
|
_, err := s.pool.Exec(ctx, `
|
|
INSERT INTO user_profiles (
|
|
id, user_id, name, avatar, pin_hash, is_child, is_primary, max_content_rating,
|
|
quality_preference, language, preferred_metadata_language, subtitle_language, subtitle_mode,
|
|
auto_skip_intro, auto_skip_credits, auto_skip_recap, auto_play_next_preview,
|
|
library_restrictions_enabled,
|
|
show_forced_subtitles, max_playback_quality, created_at, updated_at
|
|
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22)`,
|
|
p.ID, s.userID, p.Name, p.Avatar, p.PINHash, p.IsChild, p.IsPrimary, p.MaxContentRating,
|
|
p.QualityPreference, p.Language, p.PreferredMetadataLanguage, p.SubtitleLanguage, p.SubtitleMode,
|
|
p.AutoSkipIntro, p.AutoSkipCredits, p.AutoSkipRecap, p.AutoPlayNextPreview,
|
|
p.LibraryRestrictionsEnabled,
|
|
p.ShowForcedSubtitles, p.MaxPlaybackQuality, p.CreatedAt, p.UpdatedAt,
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("inserting profile %s: %w", p.ID, err)
|
|
}
|
|
if err := replaceProfileAllowedLibraries(ctx, s.pool, s.userID, p.ID, p.AllowedLibraryIDs); err != nil {
|
|
return err
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (s *PostgresUserStore) GetProfile(ctx context.Context, id string) (*userstore.Profile, error) {
|
|
row := s.pool.QueryRow(ctx, `
|
|
SELECT id, name, avatar, pin_hash, is_child, is_primary, max_content_rating,
|
|
quality_preference, language, preferred_metadata_language, subtitle_language, subtitle_mode,
|
|
auto_skip_intro, auto_skip_credits, auto_skip_recap, auto_play_next_preview, library_restrictions_enabled,
|
|
show_forced_subtitles, max_playback_quality, created_at, updated_at
|
|
FROM user_profiles WHERE user_id = $1 AND id = $2`, s.userID, id)
|
|
|
|
p, err := scanProfile(row)
|
|
if err == pgx.ErrNoRows {
|
|
return nil, nil
|
|
}
|
|
if err != nil {
|
|
return nil, fmt.Errorf("querying profile %s: %w", id, err)
|
|
}
|
|
p.AllowedLibraryIDs, err = listProfileAllowedLibraries(ctx, s.pool, s.userID, p.ID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return p, nil
|
|
}
|
|
|
|
func (s *PostgresUserStore) ListProfiles(ctx context.Context) ([]userstore.Profile, error) {
|
|
rows, err := s.pool.Query(ctx, `
|
|
SELECT id, name, avatar, pin_hash, is_child, is_primary, max_content_rating,
|
|
quality_preference, language, preferred_metadata_language, subtitle_language, subtitle_mode,
|
|
auto_skip_intro, auto_skip_credits, auto_skip_recap, auto_play_next_preview, library_restrictions_enabled,
|
|
show_forced_subtitles, max_playback_quality, created_at, updated_at
|
|
FROM user_profiles WHERE user_id = $1 ORDER BY created_at ASC`, s.userID)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("listing profiles: %w", err)
|
|
}
|
|
defer rows.Close()
|
|
|
|
var profiles []userstore.Profile
|
|
for rows.Next() {
|
|
p, err := scanProfile(rows)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("scanning profile row: %w", err)
|
|
}
|
|
p.AllowedLibraryIDs, err = listProfileAllowedLibraries(ctx, s.pool, s.userID, p.ID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
profiles = append(profiles, *p)
|
|
}
|
|
if err := rows.Err(); err != nil {
|
|
return nil, fmt.Errorf("iterating profile rows: %w", err)
|
|
}
|
|
return profiles, nil
|
|
}
|
|
|
|
func (s *PostgresUserStore) UpdateProfile(ctx context.Context, id string, u userstore.UpdateProfileInput) error {
|
|
var setClauses []string
|
|
var args []any
|
|
argIdx := 1
|
|
accessPolicyChanged := u.PIN != nil ||
|
|
u.IsChild != nil ||
|
|
u.MaxContentRating != nil ||
|
|
u.LibraryRestrictionsEnabled != nil ||
|
|
u.AllowedLibraryIDs != nil ||
|
|
u.MaxPlaybackQuality != nil
|
|
|
|
addArg := func(clause string, val any) {
|
|
setClauses = append(setClauses, fmt.Sprintf("%s = $%d", clause, argIdx))
|
|
args = append(args, val)
|
|
argIdx++
|
|
}
|
|
|
|
if u.Name != nil {
|
|
addArg("name", *u.Name)
|
|
}
|
|
if u.Avatar != nil {
|
|
addArg("avatar", *u.Avatar)
|
|
}
|
|
if u.PIN != nil {
|
|
if *u.PIN == "" {
|
|
// Empty string clears the PIN rather than hashing an empty value.
|
|
addArg("pin_hash", "")
|
|
} else {
|
|
hash, err := bcrypt.GenerateFromPassword([]byte(*u.PIN), bcrypt.DefaultCost)
|
|
if err != nil {
|
|
return fmt.Errorf("hashing PIN: %w", err)
|
|
}
|
|
addArg("pin_hash", string(hash))
|
|
}
|
|
}
|
|
if u.IsChild != nil {
|
|
addArg("is_child", *u.IsChild)
|
|
}
|
|
if u.MaxContentRating != nil {
|
|
addArg("max_content_rating", *u.MaxContentRating)
|
|
}
|
|
if u.QualityPreference != nil {
|
|
addArg("quality_preference", *u.QualityPreference)
|
|
}
|
|
if u.Language != nil {
|
|
addArg("language", *u.Language)
|
|
}
|
|
if u.PreferredMetadataLanguage != nil {
|
|
addArg("preferred_metadata_language", *u.PreferredMetadataLanguage)
|
|
}
|
|
if u.SubtitleLanguage != nil {
|
|
addArg("subtitle_language", *u.SubtitleLanguage)
|
|
}
|
|
if u.SubtitleMode != nil {
|
|
addArg("subtitle_mode", *u.SubtitleMode)
|
|
}
|
|
if u.AutoSkipIntro != nil {
|
|
addArg("auto_skip_intro", *u.AutoSkipIntro)
|
|
}
|
|
if u.AutoSkipCredits != nil {
|
|
addArg("auto_skip_credits", *u.AutoSkipCredits)
|
|
}
|
|
if u.AutoSkipRecap != nil {
|
|
addArg("auto_skip_recap", *u.AutoSkipRecap)
|
|
}
|
|
if u.AutoPlayNextPreview != nil {
|
|
addArg("auto_play_next_preview", *u.AutoPlayNextPreview)
|
|
}
|
|
if u.LibraryRestrictionsEnabled != nil {
|
|
addArg("library_restrictions_enabled", *u.LibraryRestrictionsEnabled)
|
|
}
|
|
if u.ShowForcedSubtitles != nil {
|
|
addArg("show_forced_subtitles", *u.ShowForcedSubtitles)
|
|
}
|
|
if u.MaxPlaybackQuality != nil {
|
|
addArg("max_playback_quality", *u.MaxPlaybackQuality)
|
|
}
|
|
|
|
if len(setClauses) == 0 && u.AllowedLibraryIDs == nil {
|
|
return nil
|
|
}
|
|
|
|
var exists bool
|
|
if err := s.pool.QueryRow(ctx, "SELECT EXISTS(SELECT 1 FROM user_profiles WHERE user_id = $1 AND id = $2)", s.userID, id).Scan(&exists); err != nil {
|
|
return fmt.Errorf("checking profile %s existence: %w", id, err)
|
|
}
|
|
if !exists {
|
|
return fmt.Errorf("profile %s not found", id)
|
|
}
|
|
|
|
if len(setClauses) > 0 {
|
|
addArg("updated_at", nowUTC())
|
|
|
|
whereClause := fmt.Sprintf("WHERE user_id = $%d AND id = $%d", argIdx, argIdx+1)
|
|
args = append(args, s.userID, id)
|
|
|
|
query := fmt.Sprintf("UPDATE user_profiles SET %s %s", strings.Join(setClauses, ", "), whereClause)
|
|
result, err := s.pool.Exec(ctx, query, args...)
|
|
if err != nil {
|
|
return fmt.Errorf("updating profile %s: %w", id, err)
|
|
}
|
|
if result.RowsAffected() == 0 {
|
|
return fmt.Errorf("profile %s not found", id)
|
|
}
|
|
}
|
|
|
|
if u.AllowedLibraryIDs != nil {
|
|
if err := replaceProfileAllowedLibraries(ctx, s.pool, s.userID, id, *u.AllowedLibraryIDs); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
if accessPolicyChanged {
|
|
if _, err := s.pool.Exec(ctx, "UPDATE users SET access_policy_revision = access_policy_revision + 1 WHERE id = $1", s.userID); err != nil {
|
|
return fmt.Errorf("bumping access policy revision for user %d: %w", s.userID, err)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (s *PostgresUserStore) DeleteProfile(ctx context.Context, id string) error {
|
|
tx, err := s.pool.Begin(ctx)
|
|
if err != nil {
|
|
return fmt.Errorf("beginning transaction for profile delete: %w", err)
|
|
}
|
|
defer tx.Rollback(ctx) //nolint:errcheck
|
|
|
|
_, err = tx.Exec(ctx, `
|
|
DELETE FROM user_personal_collection_items
|
|
WHERE user_id = $1 AND collection_id IN (
|
|
SELECT id FROM user_personal_collections WHERE user_id = $1 AND profile_id = $2
|
|
)`, s.userID, id)
|
|
if err != nil {
|
|
return fmt.Errorf("deleting collection items for profile %s: %w", id, err)
|
|
}
|
|
|
|
cascadeTables := []string{
|
|
"user_favorites",
|
|
"user_watchlist",
|
|
"user_watch_progress",
|
|
"user_personal_collections",
|
|
"user_series_playback_preferences",
|
|
"user_library_playback_preferences",
|
|
}
|
|
for _, table := range cascadeTables {
|
|
if _, err := tx.Exec(ctx, fmt.Sprintf("DELETE FROM %s WHERE user_id = $1 AND profile_id = $2", table), s.userID, id); err != nil {
|
|
return fmt.Errorf("deleting from %s for profile %s: %w", table, id, err)
|
|
}
|
|
}
|
|
|
|
result, err := tx.Exec(ctx, "DELETE FROM user_profiles WHERE user_id = $1 AND id = $2", s.userID, id)
|
|
if err != nil {
|
|
return fmt.Errorf("deleting profile %s: %w", id, err)
|
|
}
|
|
if result.RowsAffected() == 0 {
|
|
return fmt.Errorf("profile %s not found", id)
|
|
}
|
|
|
|
return tx.Commit(ctx)
|
|
}
|
|
|
|
func (s *PostgresUserStore) VerifyPIN(ctx context.Context, profileID, pin string) (bool, error) {
|
|
var pinHash string
|
|
err := s.pool.QueryRow(ctx,
|
|
"SELECT pin_hash FROM user_profiles WHERE user_id = $1 AND id = $2",
|
|
s.userID, profileID,
|
|
).Scan(&pinHash)
|
|
if err == pgx.ErrNoRows {
|
|
return false, fmt.Errorf("profile %s not found", profileID)
|
|
}
|
|
if err != nil {
|
|
return false, fmt.Errorf("querying PIN hash for profile %s: %w", profileID, err)
|
|
}
|
|
if pinHash == "" {
|
|
return false, fmt.Errorf("profile %s has no PIN set", profileID)
|
|
}
|
|
|
|
err = bcrypt.CompareHashAndPassword([]byte(pinHash), []byte(pin))
|
|
if err == bcrypt.ErrMismatchedHashAndPassword {
|
|
return false, nil
|
|
}
|
|
if err != nil {
|
|
return false, fmt.Errorf("comparing PIN for profile %s: %w", profileID, err)
|
|
}
|
|
return true, nil
|
|
}
|