72 Commits
Author SHA1 Message Date
imSp4rky ee38477b11 feat(dl): add --all-drm to license Widevine and PlayReady per track
Each DRM system sends one challenge per KID set in a run, also when a vault has the content keys. Needs a CDM for each system in the cdm config.
2026-10-05 15:22:01 -06:00
imSp4rky 9a1abd23b6 feat(remote): license with the client's device when the user picked one
A remote client that passes --cdm, has a cdm entry for the service that named the loaded device, or sets server_cdm: false now declines the server CDM at session create, and warns when the server keeps it. A service option listed in SERVER_DEVICE_OPTIONS keeps the server CDM.
2026-10-01 13:21:53 -06:00
imSp4rky 4f188b866f fix(cdm): skip the server header probe for pywidevine and pyplayready remote CDMs
A reverse proxy or CDN rewrites the Server header, so upstream refused a working serve API.
2026-09-25 22:54:08 -06:00
imSp4rky 746bf184d2 refactor(proxies): share proxy dispatch between dl, search and resolve_proxy
A bare query skips a failing provider and errors when none answers; an unknown prefix such as us:seattle fails with a hint. The REST path now loads WindscribeVPN.
2026-09-25 18:14:26 -06:00
imSp4rky 69360d1b4c feat(remote): let a remote client license with its own device identity
Relay the service's CDM calls to the client, accept the service's allowed identity config (for example ESN/Kpe/Kph), add the server_vault grant and a lent server-device mode that reads keys only from vaults and returns no login cache. Also fixes the proxied certificate fetch, Base64 Widevine licences, an answered call that stayed pending and a missing default quality.
2026-09-25 12:32:49 -06:00
imSp4rky 7d31153c7f feat(remote): cap server CDM licensing by height per API key and service
A session above the cap licenses with the client's own device, and a track above it in an uncapped -q run is handed back to the client. Also names the fallback in the ccextractor crash warning.
2026-09-24 16:50:26 -06:00
imSp4rky 91db1fa0b1 feat(remote): keep slow remote session requests alive with a heartbeat
Titles, tracks and licence stream a newline every 30s until the body is ready, so a slow service outlasts the client timeout and proxy idle limits; add a per-server timeout.
2026-09-24 10:18:03 -06:00
imSp4rkyandAntCardFr 783c8ae1c2 feat(proxies): add Control D proxy provider
Runs a loopback forwarder per region-named profile (unshackle-<region>), so
--proxy and --proxy-download can exit in different countries

Thanks to Orni_ for the original code and the test account.

Co-authored-by: AntCardFr <24798811+AntCardFR@users.noreply.github.com>
2026-09-23 13:23:50 -06:00
imSp4rky d239964f0b fix(serve): apply staged service reloads when sessions end, with a max staging age
A staged tag was only retried on the refresh tick or a job completion, so a service with steady remote sessions never swapped in. Session end, expiry and eviction now trigger a single-flight apply, the tick applies after the pull, and serve.services_staged_max_age swaps an overdue tag in even while busy.
2026-09-22 10:40:49 -06:00
imSp4rky 6e681af513 feat(manifests): extend merge_segments rolling merge to HLS and ISM
RollingMerge takes the segment path and an append hook. ISM synthesises the init from segment 0 and converts PIFF per fragment; HLS qualifies only with one init, one key and no discontinuity. Both parsers check the merge reached every segment, and resume is decided per track.
2026-09-21 20:53:37 -06:00
imSp4rky 79d109d548 feat(dash): add merge_segments to append segments during the download
Opt-in rolling merge for DASH video and audio: each contiguous segment is appended to the output as it lands, so the post-download "Merging" stage goes away. Decrypts the init segment eagerly when decrypt_segments is on, disables resume for the track, and skips subtitle tracks.
2026-09-21 19:13:39 -06:00
imSp4rky b182ed97f2 fix(post-scripts): expose {part} and all title-owned variables on failure
Build the failure context through the title's template builder with an empty MediaInfo, and blank season_episode, absolute and date in season mode. Document {part} and add a docs-to-code parity test.
2026-09-20 13:32:20 -06:00
imSp4rky b30e6d7cbe fix(vaults): verify vault content keys per KID from the fragment map
Map each KID to its fragments from the ciphertext moov/moof (tenc, seig, ISM duration clock) and decode up to three windows per KID that never cross into another KID, so a clear lead cannot hide a wrong key and only the failing KID is flagged. Decode every frame (wrong-key HEVC keyframes decode clean), re-judge mid-GOP starts on keyframes, time out every FFmpeg run, and refuse to store a flagged pair again in SQLite.
2026-09-18 01:10:14 -06:00
imSp4rky ffda836442 fix(remote): return session cache for device-code and cache-only logins
Answered login prompts and uploaded cache now count as client auth; the client closes on exit and keeps the remote session alive during long downloads.
2026-09-16 19:36:31 -06:00
imSp4rky e40421a855 fix(serve): defer service hot reload while a remote session uses it
Remote-only serve no longer builds the download queue manager just to read busy services.
2026-09-13 12:21:31 -06:00
imSp4rky a7d6791136 feat(remote): verify server-CDM vault keys on the client and report bad pairs
The licence response lists KIDs a server vault supplied, the client proves them with the decode check, and a bad pair goes to the new session keys/bad route so the server flags its own row. Client vault keys go first over --remote, and the dashboard health warns when no SQLite vault can store flags.
2026-09-12 15:13:08 -06:00
imSp4rky c8a674d85d fix(naming): correct language tag rule matching and add state conditions
Rules took only a scalar, so the documented list syntax raised LanguageTagError after the mux. Normalise every condition, treat an invalid tag as no match, guard the surrounding post-mux config reads, and let a rule match dual, multi and dubbed so a release can carry its own tag.
2026-09-11 20:00:08 -06:00
imSp4rky 0b055e43ba fix(api): stop serve leaking server credentials, paths, and cross-tenant data 2026-09-08 16:43:39 -06:00
imSp4rky d36797641d feat(dl): add --no-postscript to skip post-scripts for one run 2026-09-06 21:02:58 -06:00
imSp4rky b391ffc80a perf(kv): push keys to vaults concurrently, log one summary per title
Vaults.add_key and add_keys now push to every vault at the same time, so one unreachable vault costs its own timeout instead of the sum. MySQL defaults connect_timeout to vault_timeout.
2026-09-04 09:13:31 -06:00
imSp4rky 8354235d56 feat(serve): add operator dashboard endpoints and per-key rate limits 2026-09-02 23:52:55 -06:00
imSp4rky d9b21229c3 feat(serve)!: add server-held accounts for remote sessions
serve.server_accounts lends region-tagged server credentials to remote sessions (round-robin per region, per-key opt-in, per-account cache). Remote sessions no longer fall back to the server's local credentials when the client sends none.
2026-08-28 16:36:39 -06:00
imSp4rky e3f4e52444 feat(serve): add --quiet mode and developer dashboard API
Headless serve with plain stderr logging; /api/dashboard/* (status, sessions, jobs, logs, SSE/poll events) behind a separate serve.dashboard.key.
2026-08-28 13:18:41 -06:00
imSp4rky c49fccf3b5 fix(remote): resolve service names against the remote server
Remote mode resolved names with the local alias table and built an empty stub for any unknown tag, so an unentitled service showed a blank help page. Resolve against the server's tags and aliases, and reject unknown names with the list the server does offer.
2026-08-28 10:54:21 -06:00
imSp4rky dcc1c8af18 feat(serve)!: gate the server's own proxies behind server_proxy
The server no longer spends its configured proxy providers on a remote client unless the API key sets server_proxy. It places the client by geolocating the connection address and asks for a proxy when that region does not match its own.
2026-08-27 12:43:53 -06:00
imSp4rky 3aa275035b feat(serve): gate server_cdm per service and let clients follow it
serve.users.<key>.server_cdm accepts a list of service tags; retry re-attaches the job service and a bare string no longer fails open. An unset client server_cdm now follows the server's per-service /tracks flag.
2026-08-27 10:10:46 -06:00
imSp4rky 5128fef473 feat(serve): hot-reload service repos on a timer without a restart
Add serve.services_refresh_interval; refresh pulls the repos and re-imports only the changed services, staging any with active jobs until they finish. Report skipped/broken services in the log and on /api/services.
2026-08-26 23:14:23 -06:00
imSp4rky 4964869258 feat(services): add services_repo_force and compact refresh output
Force-reset repo clones on the automatic TTL refresh when enabled. Report refresh changes as one +/~/-/!TAG line per repo.
2026-08-26 22:15:39 -06:00
imSp4rky 89f09f8e84 perf(drm): decrypt fMP4 segments during download with mp4decrypt
Opt-in decrypt_segments runs mp4decrypt --fragments-info per segment on a thread pool as segments land (DASH, single-init HLS), then decrypts the init alone; hides the post-download decrypt step with byte-identical output.
2026-08-26 16:27:55 -06:00
imSp4rky b94f338619 docs(post-scripts): use post-script as the one term for hooks
Add the post-script row to the ste-writing terminology contract and
replace hook/postscript/post-download script across docs and docstrings.
2026-08-25 19:57:41 -06:00
imSp4rky 40cad41cc4 feat(post-scripts): add wait option to block until the script exits
Per-entry `wait: true` (default false) makes dispatch() wait for the script and log its exit code; a non-zero exit never fails the download.
2026-08-25 19:19:41 -06:00
imSp4rky e804d09f0a feat(dl): add --cdm to pin one CDM device for a run
Overrides the cdm config mapping (and its quality/DRM sub-entries) for the whole run; the REST API cdm field now flows through the same param.
2026-08-24 16:37:48 -06:00
imSp4rky 8c43250c9b feat(dl): add --no-attachments to skip cover art and subtitle fonts
Drops service attachments at track selection and skips the font attach pass; exposed as no_attachments in the REST dl params and documented.
2026-08-24 16:21:52 -06:00
imSp4rky 1f96a4bd78 fix(subtitle): keep WebVTT-only cue markup out of converted subtitles
Strip cue classes, <v>/<lang>/ruby and karaoke timestamps before conversion, decode entities for SRT/SSA/ASS, load pysubs2 as utf-8-sig, and route sdh_method: filter-subs through convert_before_strip instead of raising.
2026-08-24 15:55:06 -06:00
imSp4rky d8739b3561 feat(dl): add --require-audio/--require-video and --warn-only alias
Gate a title on audio or video languages that must exist, independent of what -l/-al/-vl select. --require-subs joins the same gate: its --s-lang conflict and download-all side effect are gone, so -sl now decides what to keep. Requiring a language that is also excluded fails at parse time.
2026-08-23 12:24:45 -06:00
imSp4rky 537a1dc973 perf(dl): parallelise mux, licensing, and vault lookups; size the shared pool
Grow the service session pool to downloads * workers before track downloads, run mkvmerge jobs concurrently (muxing.concurrency, default 4), lock DRM preparation per key set instead of globally, and query local vaults first then remote vaults together.
2026-08-20 11:24:12 -06:00
imSp4rky 8b10b793e2 docs: phase 2 of the prose style across docs and docstrings 2026-08-19 20:59:09 -06:00
imSp4rky 66971230c3 docs: apply simplified technical english across docs and docstrings. 2026-08-19 16:31:06 -06:00
imSp4rky 29d0828c33 style: clean up prose 2026-08-19 13:45:58 -06:00
imSp4rky 68c7d95173 refactor(music)!: make Song an ordinary Title on the generic path
Delete the dead grouped-download block, the planner, and the music service hooks; Song now flows through the one download loop, gets tagged via tag_file, and answers -w with {disc}x{track} keys.

BREAKING CHANGE: GROUP_AUDIO_DOWNLOADS, get_music_track_options and get_music_collection_label are gone. A music service returns one Audio track per quality and lets the framework choose.
2026-08-18 15:05:30 -06:00
imSp4rky 89acfab279 feat(dl): run user post-download scripts with metadata variables
New post_scripts config and repeatable --postscript flag run a command per output, season/album folder, or run, substituting filename-template variables into pre-tokenized argv. The REST API rejects hook commands.

Closes #150
2026-08-18 08:26:14 -06:00
imSp4rky 619c86cd10 feat(downloader): resume failed downloads from completed segments across runs
Fingerprint-gated reuse of atomically-published segment files (DASH/ISM/HLS) behind a continue_downloads config option and one-off dl --continue-downloads flag.
2026-08-16 19:38:06 -06:00
imSp4rky ee9d3edfe3 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/api/download_manager.py
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
#	unshackle/core/session.py
#	unshackle/core/utils/webvtt.py
2026-08-16 12:43:01 -06:00
imSp4rky db9ec3e334 feat(cli): add named color themes and rich-formatted help
Render all click help through rich-click, themed by a new `theme` config key (9 palettes + aliases, previewable via `unshackle env theme`), and show service docstrings in local and remote `dl <TAG> -h`.
2026-08-13 16:25:14 -06:00
imSp4rky a248ae79bd feat(serve): gate server CDM licensing behind a per-key server_cdm opt-in
Keys configured under serve.users default to off, so they license with their own local CDM; session licensing and download jobs both refuse otherwise.
2026-08-13 11:40:30 -06:00
imSp4rky 5796729bf3 feat(remote): accept serve.users keys in api-only mode and add client auth-header fallback
api-only/remote-only serve now authenticates every serve.users key (str-coerced), not just api_secret, so per-user service allowlists apply. RemoteClient tries X-Secret-Key then X-Api-Key on 401 (sticky winner); remote_services.auth_headers prepends extra header names to the defaults.
2026-08-12 17:58:55 -06:00
imSp4rky e4f6ddec85 feat(dl): add absolute episode numbers filled from tvdb absolute order
Episode gains an additive absolute field services can set, an {absolute}
naming token (3-wide), and an --enrich fill from TVDB's absolute order
that never rewrites season or episode numbers.
2026-08-11 16:42:07 -06:00
imSp4rky 72581ba5d2 feat(metadata)!: replace animeapi with a first-class anilist provider
--anilist supersedes --animeapi (REST: anilist_id); the keyless AniList
GraphQL provider joins the registry with title search, mal: ID lookup,
an ANILIST mkv tag, and the anilist_title_language config option.
Services mark anime via Service.ANIME / Title.anime for anilist-first
metadata resolution with fall-through.
2026-08-11 16:42:05 -06:00
imSp4rky ebb4e74318 feat(metadata)!: authoritative id resolver and keyless imdb provider
Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working.
BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
2026-08-11 15:07:24 -06:00
imSp4rky 79ad70a9cf feat(dl): exclude languages with a - prefix on -l, -vl, -al, -sl and -fsl 2026-08-10 12:18:30 -06:00