diff --git a/CHANGELOG.md b/CHANGELOG.md index 03faef5d5..5400b489e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,37 +4,12 @@ ## โš ๏ธ Breaking Changes -- **Proxied resource URLs are now restricted to public destinations by default.** Tuliprox proxies external - resource URLs that come from provider, playlist, and EPG content: channel and small logos, EPG channel and - programme icons, cover images, posters, and backdrops. These requests now enforce a destination policy on every - route that serves them (`/resource/m3u/...`, the Xtream resource routes, `/resource/epg/...`, and - `/api/v1/playlist/resource/...`), where previously three of them fetched any destination reachable by the - configured HTTP client. - - A resource URL whose DNS host name resolves to a private address (RFC 1918 or IPv6 ULA) is rejected unless the - input that supplied it lists the exact host name in `resource_policy.allowed_hosts` **and** the address in - `resource_policy.allowed_networks`. A private IP literal requires only a matching - `resource_policy.allowed_networks` entry because IP literals are not valid `allowed_hosts` values. Add the policy - to the input that provides the logo or icon; for icons that `logo_override` copies out of EPG, that is the EPG - input. - - Loopback, link-local, cloud-metadata, CGNAT, multicast, and reserved addresses stay blocked with or without a - policy. Redirects are re-checked on every hop and are bounded. - - Resource ownership is stored generically with each URL, including nested cover, poster, backdrop, and episode - image fields. Legacy raw playlist/Xtream item resources use their containing item's input; legacy EPG resources - without an authoritative input remain public-only until regenerated. - - `resource://` is an internal reserved scheme. Provider data and mapping configuration must never supply it; - such values are rejected rather than interpreted as authorization claims. - - The canonical input name is the authorization identity of a resource origin. Configured input and alias names - must be non-empty, globally unique strings; a configuration with duplicate input names, duplicate alias names, - or an alias name that shadows an input name is now rejected while loading. Internal IDs are managed separately. - - The resource cache is keyed by the policy that authorized the entry, so an entry fetched under one policy is - never served to another. The cache starts cold once on upgrade because the key layout changes. - - Resource proxying now always connects directly: a configured proxy and the `HTTP_PROXY` / `HTTPS_PROXY` / - `ALL_PROXY` environment variables are ignored for these requests, and Tuliprox logs a warning at startup and on - reload when one is set. Provider fetches, playlist and EPG downloads, and streams keep using the proxy. - - The Source Editor exposes the policy on every input under the shield-shaped **Resource Policy** page. Empty host - and network lists restore the public-only default and omit the policy from the saved input. - - See [Resource Policy](docs/src/configuration/source.md#27-resource-policy-resource_policy) for the parameters - and the exact host-plus-network rule. +- **The per-input resource policy is gone again.** `resource_policy` (with `allowed_hosts` / `allowed_networks`) is no + longer a valid input field, so configurations written for that feature are rejected while loading, and resource links + minted by it are no longer accepted. Resource destinations are classified instead of configured: a destination on a + private network is proxied, a public one may be handed to the client directly. Resource links that were issued + before this change (the `obscure_text` format) are rejected as well, so an EPG a client has already cached shows + broken icons until the EPG is fetched again. - **The Web UI WebSocket protocol is now version 4.** Playlist update completion messages carry the correlated run ID and execution order instead of a bare status. Reload existing browser tabs after upgrading the server; @@ -1049,6 +1024,34 @@ ## ๐Ÿ› Fixes +- **Resource URLs no longer expose internal destinations.** Resource URLs are classified before they are written into + playlist or EPG output: only a provably public destination is handed to the client, everything else (`tvg-logo`, + `tvg-logo-small`, Xtream covers and backdrops, EPG channel and programme icons, Web UI item icons) is replaced by an + authenticated resource link. This also covers redirect mode (`resource_rewrite_disabled: true`), where a + network-internal image is proxied instead of being published, and an image without a client-visible base URL is dropped + rather than written through. Destinations local to the Tuliprox host โ€” loopback, link-local, cloud metadata โ€” are + dropped from the output and refused by the resource route, and an upstream failure is reported as a generic + `502 Bad Gateway` instead of relaying the destination's error response. The refusal also happens while the connection + is built, so a destination cannot resolve to a public address during the check and to a local one afterwards. A name + whose lookup produced no answer stays non-public for ten seconds rather than for minutes, so a resolver hiccup does + not pin a destination to the proxy for the rest of its verdict lifetime. +- **Resource proxying keeps respecting the configured proxy.** Every redirect hop of a resource URL is classified on + its own: a hop that is not provably public connects directly, so a self-hosted media server on the local network stays + reachable, while every other hop - including the public destinations of the Web UI resource route and a public hop + reached through a redirect - goes through the configured `proxy` block. Resource requests therefore cannot disclose + the operator's address to a resource host. A redirect to a destination local to the Tuliprox host is refused without + being requested, and both resource clients refuse such an address again while the connection is built, so a name that + was classified as public cannot resolve to a local address afterwards. The configured proxy hosts are exempt from + that guard, so a proxy on the loopback interface (`proxy: http://localhost:8118`) keeps working instead of making + every public resource fetch fail. A resource name that does not resolve locally is fetched through the configured + proxy, which can still resolve it (for example with remote DNS); a destination that resolves to a private address + keeps connecting directly. Redirects are followed one hop at a time, bounded to five hops. + +- **Duplicate input and alias names fail the configuration load again.** Input names and alias names share one + namespace, and a duplicate silently resolved to whichever member was inserted last, so a request could run against + another input's playlist, credentials, and limits. The check that rejects such a configuration is part of the sources + load again, which also means an invalid reload is reported instead of replacing the running configuration. + - **Streaming and connection management: resolved silent async hang / deadlock during client kicks and concurrent stream load.** Under concurrent stream traffic, `tuliprox` would occasionally stop logging and serving requests (the Web UI became unreachable and active streams dropped) while the container remained in a running state with near-zero CPU and memory @@ -1417,14 +1420,6 @@ ## โš™๏ธ New Settings -- **source.yml (input `resource_policy`)**: Added an optional per-input policy for private resource destinations. - - `allowed_hosts` (list of exact DNS names, default empty) and `allowed_networks` (list of private CIDR ranges, - default empty) authorize a private address only together: the host name must match and the resolved address must - fall inside one of the networks. An IP literal is authorized by `allowed_networks` alone. An absent or empty - policy means public-only. - - Invalid entries (scheme, path, port, wildcard, IP literal in `allowed_hosts`; a range outside - `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, or `fc00::/7`) are rejected while the configuration is loaded. - - **Runtime diagnostics (environment variables)**: - `TULIPROX_WATCHDOG` (default unset = off) is a mode selector: `1` (`true`/`on`/`yes`/`enabled`) observes and logs stalls, `2` (`restart`) additionally exits the process after the stall persists so a supervisor restarts it. diff --git a/Cargo.lock b/Cargo.lock index cd6088e8f..83b97f758 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3784,6 +3784,7 @@ dependencies = [ name = "shared" version = "3.3.122" dependencies = [ + "aes", "base64 0.23.1", "bitflags 2.13.1", "blake3", @@ -3792,6 +3793,7 @@ dependencies = [ "chrono", "chrono-tz", "ciborium", + "ctr", "dashmap", "deunicode", "fastrand", @@ -4546,6 +4548,7 @@ dependencies = [ "ipnet", "libc", "log", + "lru", "parking_lot", "paste", "path-clean", diff --git a/Cargo.toml b/Cargo.toml index 30c46add9..00da791ca 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -87,7 +87,6 @@ incremental = false [profile.dev] lto = false incremental = true -debug = "line-tables-only" codegen-units = 256 [profile.dev-slim] diff --git a/backend/app/src/api/api_utils/mod.rs b/backend/app/src/api/api_utils/mod.rs index 308baacdf..2335cb332 100644 --- a/backend/app/src/api/api_utils/mod.rs +++ b/backend/app/src/api/api_utils/mod.rs @@ -1,4 +1,3 @@ -mod resource; pub use crate::repository::{ evaluate_network_access, log_network_access_allowed_geoip_unavailable, log_network_access_denied, NetworkAccessDecision, NetworkAccessDenyReason, @@ -28,7 +27,6 @@ use crate::{ }, model::{ AppConfig, ConfigInput, ConfigInputFlags, ConfigTarget, InputUserInfo, PlaybackKind, ProxyUserCredentials, - ReverseProxyDisabledHeaderConfig, }, processing::{ parser::hls::{rewrite_hls, RewriteHlsProps}, @@ -36,9 +34,13 @@ use crate::{ }, repository::load_input_m3u_stream_url, utils::{ - async_file_reader, async_file_writer, create_new_file_for_write, debug_if_enabled, get_file_extension, request, - request::{content_type_from_ext, parse_range, send_with_retry_and_provider}, - resource_cache_key, trace_if_enabled, + async_file_reader, async_file_writer, classify_output_resource_url, classify_resource_hop, + create_new_file_for_write, debug_if_enabled, get_file_extension, request, + request::{ + classify_resource_destination, content_type_from_ext, parse_range, send_with_retry_and_provider, + ResourceDestination, + }, + trace_if_enabled, ResourceHop, }, BUILD_TIMESTAMP, }; @@ -52,18 +54,14 @@ use bytes::{Bytes, BytesMut}; use chrono::{DateTime, Utc}; use futures::{stream, Stream, StreamExt, TryStreamExt}; use log::{debug, error, info, log_enabled, trace, warn}; -pub use resource::{ - decode_resource_link, log_missing_resource_client, log_resource_rejection, rejection_status, resolve_resource, - resolve_resource_authorization, ResolvedResource, ResolvedResourceAuthorization, ResourceCacheMode, - ResourceFetchOptions, -}; use serde::Serialize; use shared::{ concat_string, defaults::{DASH_EXT, HLS_EXT}, model::{ ConfigTargetOptions, InputFetchMethod, InputType, PlaylistEntry, PlaylistItemType, ProxyType, - StalkerStreamKind, StreamChannel, StreamInfo, TargetType, UserConnectionPermission, VirtualId, XtreamCluster, + ResourceOutputPolicy, StalkerStreamKind, StreamChannel, StreamInfo, TargetType, UserConnectionPermission, + VirtualId, XtreamCluster, }, utils::{ bin_serialize, current_time_secs, extract_extension_from_url, get_credentials_from_url, human_readable_kbps, @@ -3612,60 +3610,6 @@ pub fn is_hls_stream_share_enabled(target: &ConfigTarget) -> bool { target.options.as_ref().is_some_and(ConfigTargetOptions::share_live_hls_enabled) } -fn resource_credential_context(input: Option<&ConfigInput>, request_headers: &HashMap>) -> String { - fn update_field(hasher: &mut blake3::Hasher, value: &[u8]) { - hasher.update(&(value.len() as u64).to_be_bytes()); - hasher.update(value); - } - - fn update_headers(hasher: &mut blake3::Hasher, headers: impl Iterator)>) { - let mut headers = headers.collect::>(); - headers.sort_unstable_by(|(left, _), (right, _)| left.cmp(right)); - for (name, value) in headers { - update_field(hasher, name.as_bytes()); - update_field(hasher, &value); - } - } - - let mut hasher = blake3::Hasher::new(); - hasher.update(b"tuliprox.resource.credentials.v1"); - if let Some(input) = input { - update_field(&mut hasher, input.name.as_bytes()); - update_field(&mut hasher, input.username.as_deref().unwrap_or_default().as_bytes()); - update_field(&mut hasher, input.password.as_deref().unwrap_or_default().as_bytes()); - update_headers( - &mut hasher, - input - .headers - .iter() - .filter(|(name, _)| !request::is_safe_cross_origin_redirect_header(name)) - .map(|(name, value)| (name.to_ascii_lowercase(), value.as_bytes().to_vec())), - ); - } else { - update_field(&mut hasher, &[]); - update_field(&mut hasher, &[]); - update_field(&mut hasher, &[]); - } - update_headers( - &mut hasher, - request_headers - .iter() - .filter(|(name, _)| !request::is_safe_cross_origin_redirect_header(name)) - .map(|(name, value)| (name.to_ascii_lowercase(), value.clone())), - ); - hasher.finalize().to_hex().to_string() -} - -fn resource_request_has_authorization(input: Option<&ConfigInput>, request_headers: &HashMap>) -> bool { - input.is_some_and(|input| { - input.headers.iter().any(|(name, value)| { - name.eq_ignore_ascii_case(header::AUTHORIZATION.as_str()) && HeaderValue::from_str(value).is_ok() - }) - }) || request_headers.iter().any(|(name, value)| { - name.eq_ignore_ascii_case(header::AUTHORIZATION.as_str()) && HeaderValue::from_bytes(value).is_ok() - }) -} - fn get_add_cache_content( res_url: &str, mime_type: Option, @@ -3722,9 +3666,9 @@ fn windows_file_identity(file: &tokio::fs::File) -> std::io::Result<(u32, u32, u async fn build_resource_stream_response( app_state: &Arc, cache_key: Option<&str>, + fetch_policy: ResourceFetchPolicy, resource_url: &str, response: reqwest::Response, - credential_varying: bool, ) -> axum::response::Response { let sanitized_resource_url = sanitize_sensitive_info(resource_url); let status = response.status(); @@ -3732,16 +3676,25 @@ async fn build_resource_stream_response( let mime_type = get_mime_type(response.headers(), resource_url); let has_content_range = response.headers().contains_key(header::CONTENT_RANGE); for (key, value) in response.headers() { - if !is_hop_by_hop_response_header(key) { + if !is_hop_by_hop_response_header(key) + && (fetch_policy == ResourceFetchPolicy::Public + || matches!( + *key, + header::CONTENT_TYPE + | header::CONTENT_LENGTH + | header::CONTENT_ENCODING + | header::CONTENT_RANGE + | header::ACCEPT_RANGES + | header::ETAG + | header::LAST_MODIFIED + | header::CACHE_CONTROL + )) + { response_builder = response_builder.header(key, value); } } - if credential_varying { - if let Some(headers) = response_builder.headers_mut() { - headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("private, no-store")); - } - } else if !response_builder.headers_ref().is_some_and(|h| h.contains_key(header::CACHE_CONTROL)) { + if !response_builder.headers_ref().is_some_and(|h| h.contains_key(header::CACHE_CONTROL)) { response_builder = response_builder.header(header::CACHE_CONTROL, "public, max-age=14400"); } @@ -3783,76 +3736,243 @@ async fn build_resource_stream_response( try_unwrap_body!(response_builder.body(axum::body::Body::from_stream(byte_stream))) } -struct ResourceRequestContext<'a> { - headers: &'a HashMap>, - input: Option<&'a ConfigInput>, - disabled_headers: Option<&'a ReverseProxyDisabledHeaderConfig>, - credential_varying: bool, +/// Upper bound of redirect hops a resource route follows. Every hop is classified, so the number of +/// requests a provider or EPG entry can trigger through one resource link stays bounded. +const RESOURCE_REDIRECT_LIMIT: u8 = 5; + +/// Whether outbound requests of this configuration may leave through a proxy, including one provided by +/// the environment, because the public resource client honours both. +fn proxy_in_use(app_state: &Arc) -> bool { + app_state.app_config.config.load().proxy.is_some() || crate::model::proxy_env_present() } async fn fetch_resource_with_retry( app_state: &Arc, - http_client: &reqwest::Client, url: &Url, - cache_key: Option<&str>, + fetch_policy: ResourceFetchPolicy, resource_url: &str, - request_context: ResourceRequestContext<'_>, + req_headers: &HashMap>, + input: Option<&ConfigInput>, ) -> Option { - let ResourceRequestContext { headers, input, disabled_headers, credential_varying } = request_context; + let cache_key = fetch_policy.cache_key(resource_url); let config = app_state.app_config.config.load(); let default_user_agent = config.default_user_agent.clone(); drop(config); - let provider_config = input.and_then(|i| i.get_resolve_provider(url.as_str())); - let Ok(response) = - send_with_retry_and_provider(&app_state.app_config, url, provider_config.as_ref(), false, |resolved_url| { - request::get_client_request( - http_client, - input.map_or(InputFetchMethod::GET, |i| i.method), - input.map(|i| &i.headers), - resolved_url, - Some(headers), - disabled_headers, - default_user_agent.as_deref(), - ) - }) + let disabled_headers = app_state.get_disabled_headers(); + let mut current_url = url.clone(); + let mut current_headers = req_headers.clone(); + let mut current_input = input; + let mut method = input.map_or(InputFetchMethod::GET, |i| i.method); + + for redirects in 0..=RESOURCE_REDIRECT_LIMIT { + // Every hop is routed on its own, so a redirect cannot move a request to an egress the hop + // itself would not have used. + let hop = classify_resource_hop(current_url.as_str(), proxy_in_use(app_state)).await; + if hop == ResourceHop::Blocked { + debug!("Refused resource destination local to this host: {}", sanitize_sensitive_info(resource_url)); + return None; + } + let use_proxy_aware_client = hop == ResourceHop::ViaProxy; + let provider_config = current_input.and_then(|i| i.get_resolve_provider(current_url.as_str())); + let response = match send_with_retry_and_provider( + &app_state.app_config, + ¤t_url, + provider_config.as_ref(), + // Hand redirects back instead of retrying them: the loop below follows them itself, so that + // every hop is classified and no client-side redirect policy decides where the request ends. + true, + |resolved_url| { + let http_client = if use_proxy_aware_client { + app_state.resource_public_http_client_no_redirect.load() + } else { + app_state.resource_http_client_no_redirect.load() + }; + request::get_client_request( + &http_client, + method, + current_input.map(|i| &i.headers), + resolved_url, + Some(¤t_headers), + disabled_headers.as_ref(), + default_user_agent.as_deref(), + ) + }, + ) .await - else { - return None; - }; + { + Ok(response) => response, + Err(err) => { + debug!( + "Resource fetch failed for {}: {}", + sanitize_sensitive_info(resource_url), + sanitize_sensitive_info(&err.to_string()) + ); + return None; + } + }; - let status = response.status(); - - if status.is_success() { - return Some( - build_resource_stream_response(app_state, cache_key, resource_url, response, credential_varying).await, - ); - } - - // Non-retriable Status -> Upstream Response incl. Body - debug_if_enabled!("Failed to open resource got status {status} for {}", sanitize_sensitive_info(resource_url)); - - let mut response_builder = axum::response::Response::builder().status(status); - for (key, value) in response.headers() { - if !is_hop_by_hop_response_header(key) { - response_builder = response_builder.header(key, value); + let status = response.status(); + if status.is_redirection() { + if redirects == RESOURCE_REDIRECT_LIMIT { + debug!("Resource redirect limit reached for {}", sanitize_sensitive_info(resource_url)); + return None; + } + let next_url = response + .headers() + .get(header::LOCATION) + .and_then(|location| location.to_str().ok()) + .and_then(|location| response.url().join(location).ok()); + let Some(next_url) = next_url else { + debug!("Resource redirect has no usable location for {}", sanitize_sensitive_info(resource_url)); + return None; + }; + let same_origin = response.url().scheme() == next_url.scheme() + && response.url().host_str() == next_url.host_str() + && response.url().port_or_known_default() == next_url.port_or_known_default(); + if !same_origin { + current_headers.retain(|key, _| request::is_safe_cross_origin_redirect_header(key)); + current_input = None; + } + if !matches!(status, StatusCode::TEMPORARY_REDIRECT | StatusCode::PERMANENT_REDIRECT) { + method = InputFetchMethod::GET; + } + current_url = next_url; + continue; } - } - if credential_varying { - if let Some(headers) = response_builder.headers_mut() { - headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("private, no-store")); + + if status.is_success() { + return Some( + build_resource_stream_response(app_state, cache_key, fetch_policy, resource_url, response).await, + ); } + + debug_if_enabled!("Failed to open resource got status {status} for {}", sanitize_sensitive_info(resource_url)); + if !fetch_policy.relays_upstream_response() { + return Some(status.into_response()); + } + let mut response_builder = axum::response::Response::builder().status(status); + for (key, value) in response.headers() { + if !is_hop_by_hop_response_header(key) { + response_builder = response_builder.header(key, value); + } + } + let stream = response.bytes_stream().map_err(|err| StreamError::reqwest(&err)); + return Some(try_unwrap_body!(response_builder.body(axum::body::Body::from_stream(stream)))); } - - let stream = response.bytes_stream().map_err(|err| StreamError::reqwest(&err)); - - Some(try_unwrap_body!(response_builder.body(axum::body::Body::from_stream(stream)))) + None } -/// # Panics +/// How the client-visible answer of a resource route is shaped. +/// +/// The variant mirrors the classification of the destination the route serves: a [`Public`] destination +/// may be handed to the client as it was fetched, a [`NonPublic`] one is relayed with a sanitized +/// answer. Which client performs a hop is decided per hop, not by this policy. +/// +/// [`Public`]: ResourceFetchPolicy::Public +/// [`NonPublic`]: ResourceFetchPolicy::NonPublic +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ResourceFetchPolicy { + Public, + NonPublic, +} + +pub fn resource_input_for_url<'a>(input: Option<&'a ConfigInput>, resource_url: &str) -> Option<&'a ConfigInput> { + let input = input?; + let source = Url::parse(&input.url).ok()?; + let resource = Url::parse(resource_url).ok()?; + (source.scheme() == resource.scheme() + && source.host_str() == resource.host_str() + && source.port_or_known_default() == resource.port_or_known_default()) + .then_some(input) +} + +pub async fn resource_redirect_or_proxy( + app_state: &Arc, + resource_url: &str, + req_headers: &HeaderMap, + input: Option<&ConfigInput>, +) -> axum::response::Response { + let Some(resource_url) = shared::model::persisted_resource_url(resource_url) else { + return StatusCode::BAD_REQUEST.into_response(); + }; + let resource_url = resource_url.as_ref(); + if resource_url.starts_with("/api/v1/library/thumbnail/") { + return redirect(resource_url).into_response(); + } + if resource_url.starts_with("media-server://image/") { + return resource_response(app_state, ResourceFetchPolicy::NonPublic, resource_url, req_headers, None) + .await + .into_response(); + } + let Ok(url) = Url::parse(resource_url) else { + return StatusCode::BAD_REQUEST.into_response(); + }; + if !matches!(url.scheme(), "http" | "https") { + return StatusCode::BAD_REQUEST.into_response(); + } + let Some(host) = url.host_str() else { + return StatusCode::BAD_REQUEST.into_response(); + }; + if classify_resource_destination(host).await == ResourceDestination::Public { + redirect(resource_url).into_response() + } else { + resource_proxy_response(app_state, resource_url, req_headers, resource_input_for_url(input, resource_url)).await + } +} + +pub async fn resource_proxy_response( + app_state: &Arc, + resource_url: &str, + req_headers: &HeaderMap, + input: Option<&ConfigInput>, +) -> axum::response::Response { + let Some(resource_url) = shared::model::persisted_resource_url(resource_url) else { + return StatusCode::BAD_REQUEST.into_response(); + }; + let resource_url = resource_url.as_ref(); + let input = resource_input_for_url(input, resource_url); + match classify_output_resource_url(resource_url).await { + ResourceOutputPolicy::Direct if resource_url.starts_with("/api/v1/library/thumbnail/") => { + redirect(resource_url).into_response() + } + ResourceOutputPolicy::Direct => { + resource_response(app_state, ResourceFetchPolicy::Public, resource_url, req_headers, input) + .await + .into_response() + } + ResourceOutputPolicy::Proxy => { + resource_response(app_state, ResourceFetchPolicy::NonPublic, resource_url, req_headers, input) + .await + .into_response() + } + ResourceOutputPolicy::Blocked => StatusCode::FORBIDDEN.into_response(), + } +} + +impl ResourceFetchPolicy { + const fn cache_key(self, resource_url: &str) -> Option<&str> { + match self { + Self::Public => Some(resource_url), + Self::NonPublic => None, + } + } + + /// Whether an upstream response may be relayed to the client. + /// + /// A public destination was chosen by the user's own configuration, so what it answered is what + /// the client asked for. A resource proxy whose destination is deliberately hidden from the client + /// must not become a reader for that destination: status, headers and body of an upstream error can + /// disclose more about the internal service than the link itself. + const fn relays_upstream_response(self) -> bool { matches!(self, Self::Public) } +} + +/// Callers must pass an already-decoded resource URL (not a `resource://v1/` locator). +/// [`resource_redirect_or_proxy`] and [`resource_proxy_response`] handle the decode before +/// dispatching here; XMLTV and Web-UI endpoints pass URLs from authenticated tokens. pub async fn resource_response( app_state: &Arc, - options: ResourceFetchOptions, + fetch_policy: ResourceFetchPolicy, resource_url: &str, req_headers: &HeaderMap, input: Option<&ConfigInput>, @@ -3861,8 +3981,6 @@ pub async fn resource_response( return StatusCode::NO_CONTENT.into_response(); } - // Dispatched before any HTTP(S) policy decision: these URLs are served by a dedicated - // media-server path and never leave through the resource proxy. if resource_url.starts_with("media-server://image/") { return match open_media_server_image_resource(app_state, resource_url).await { Ok(response) => response, @@ -3877,49 +3995,10 @@ pub async fn resource_response( } }; } - - let authorization = &options.authorization; - let Ok(url) = Url::parse(resource_url) else { - error!("Url is malformed {}", sanitize_sensitive_info(resource_url)); - return StatusCode::BAD_REQUEST.into_response(); - }; - - // Scheme, host presence, and IP literals are checked before the first request. Host names are - // left to the connection-time policy resolver, which is also the only place that sees the real - // address a name resolves to. - if let Err(err) = authorization.policy.validate_initial_url(&url) { - log_resource_rejection(authorization.input_name.as_deref(), &err, resource_url); - return rejection_status(&err).into_response(); - } - - let Some(http_client) = - app_state.resource_clients.load().client(&authorization.policy_digest, options.redirect_mode).cloned() - else { - log_missing_resource_client(authorization.input_name.as_deref(), &authorization.policy_digest); - return StatusCode::BAD_GATEWAY.into_response(); - }; - - let filter: HeaderFilter = Some(Box::new(|key| key != "if-none-match" && key != "if-modified-since")); + let filter: HeaderFilter = Some(Box::new(request::is_safe_cross_origin_redirect_header)); let req_headers = get_headers_from_request(req_headers, &filter); - let configured_input = - authorization.input_name.as_ref().and_then(|input_name| app_state.app_config.get_input_by_name(input_name)); - let effective_input = configured_input.as_deref().or(input); - let credential_context = resource_credential_context(effective_input, &req_headers); - let disabled_headers = app_state.get_disabled_headers(); - let authorization_disabled = - disabled_headers.as_ref().is_some_and(|disabled| disabled.should_remove(header::AUTHORIZATION.as_str())); - let credential_varying = - !authorization_disabled && resource_request_has_authorization(effective_input, &req_headers); - let cache_key = match options.cache_mode { - ResourceCacheMode::Enabled if !credential_varying => Some(resource_cache_key( - authorization.policy_digest.as_str(), - authorization.input_name.as_deref().unwrap_or_default(), - &credential_context, - url.as_str(), - )), - ResourceCacheMode::Enabled | ResourceCacheMode::Disabled => None, - }; - if let (Some(cache_key), Some(cache)) = (cache_key.as_deref(), app_state.cache.load().as_ref()) { + let cache_key = fetch_policy.cache_key(resource_url); + if let (Some(cache_key), Some(cache)) = (cache_key, app_state.cache.load().as_ref()) { let cache_hit = { let mut guard = cache.write().await; guard.get_content(cache_key) @@ -3937,25 +4016,28 @@ pub async fn resource_response( } } trace_if_enabled!("Try to fetch resource {}", sanitize_sensitive_info(resource_url)); - if let Some(resp) = fetch_resource_with_retry( - app_state, - &http_client, - &url, - cache_key.as_deref(), - resource_url, - ResourceRequestContext { - headers: &req_headers, - input: effective_input, - disabled_headers: disabled_headers.as_ref(), - credential_varying, - }, - ) - .await - { - return resp; + if let Ok(url) = Url::parse(resource_url) { + // A resource URL is chosen by playlist or EPG content, so the destination is never fetched + // blindly: an address local to this host (loopback, link-local, cloud metadata) would turn + // the proxy into a reader for the proxy host itself. Private network destinations stay + // allowed, because self-hosted services are the reason this route exists. + let Some(host) = url.host_str() else { + return StatusCode::BAD_REQUEST.into_response(); + }; + if classify_resource_destination(host).await == ResourceDestination::Blocked { + debug!("Refused resource destination local to this host: {}", sanitize_sensitive_info(resource_url)); + return StatusCode::FORBIDDEN.into_response(); + } + if let Some(resp) = + fetch_resource_with_retry(app_state, &url, fetch_policy, resource_url, &req_headers, input).await + { + return resp; + } + // Upstream failure after retries + return StatusCode::BAD_GATEWAY.into_response(); } - // Upstream failure after retries - StatusCode::BAD_GATEWAY.into_response() + error!("Url is malformed {}", sanitize_sensitive_info(resource_url)); + StatusCode::BAD_REQUEST.into_response() } async fn open_media_server_image_resource( diff --git a/backend/app/src/api/api_utils/resource.rs b/backend/app/src/api/api_utils/resource.rs deleted file mode 100644 index 6a2db26f9..000000000 --- a/backend/app/src/api/api_utils/resource.rs +++ /dev/null @@ -1,283 +0,0 @@ -//! Authorization of proxied HTTP(S) resource requests. -//! -//! Every resource request decides which input supplied the concrete URL, looks up that input's -//! policy, and then runs the fetch through the client that belongs to the policy. There is no -//! permissive default: an origin that cannot be resolved is a rejection, and a missing client is a -//! rejection as well. - -use crate::model::AppConfig; -use axum::http::StatusCode; -use log::{debug, error, warn}; -use shared::{model::resolve_resource_value, utils::sanitize_sensitive_info}; -use std::{ - collections::HashMap, - sync::{Arc, LazyLock, Mutex}, - time::{Duration, Instant}, -}; -use tuliprox_core::{ - model::{public_only_policy, PolicyDigest, ResourcePolicy, ResourcePolicyError, ResourceRedirectMode}, - utils::{decode_resource_token, has_resource_token_prefix}, -}; - -/// Whether a resource fetch may be served from (and stored in) the resource cache. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub enum ResourceCacheMode { - Disabled, - Enabled, -} - -/// The authorization a resource request ended up with. -#[derive(Clone, Debug)] -pub struct ResolvedResourceAuthorization { - /// Canonical main input name. `None` only for data without any origin. - pub input_name: Option>, - /// Normalized policy. An empty policy is public-only. - pub policy: Arc, - pub policy_digest: PolicyDigest, -} - -#[derive(Clone, Debug)] -pub struct ResolvedResource { - pub url: Arc, - pub authorization: ResolvedResourceAuthorization, -} - -impl ResolvedResourceAuthorization { - /// Authorization for data without an origin: public destinations only. - pub fn public_only() -> Self { - let policy = public_only_policy(); - Self { input_name: None, policy_digest: policy.digest(), policy } - } - - /// Authorization for an input that is known to be configured. - pub fn for_input(input_name: Arc, policy: Option<&Arc>) -> Self { - let policy = policy.cloned().unwrap_or_else(public_only_policy); - Self { input_name: Some(input_name), policy_digest: policy.digest(), policy } - } -} - -/// Everything the shared resource path needs besides the URL itself. -#[derive(Clone)] -pub struct ResourceFetchOptions { - pub authorization: ResolvedResourceAuthorization, - pub redirect_mode: ResourceRedirectMode, - pub cache_mode: ResourceCacheMode, -} - -impl ResourceFetchOptions { - /// EPG delivery: an upstream redirect is returned to the client and nothing is cached. - pub fn epg(authorization: ResolvedResourceAuthorization) -> Self { - Self { authorization, redirect_mode: ResourceRedirectMode::NoRedirect, cache_mode: ResourceCacheMode::Disabled } - } - - /// Cached resource routes: bounded redirects and cache participation. - pub fn cached(authorization: ResolvedResourceAuthorization) -> Self { - Self { authorization, redirect_mode: ResourceRedirectMode::Bounded, cache_mode: ResourceCacheMode::Enabled } - } -} - -/// Resolves the originating input of a resource value to its current policy. -/// -/// Three outcomes are kept apart on purpose: -/// - no origin at all (legacy records, legacy links) becomes public-only; -/// - an origin naming an unknown or disabled input is rejected; -/// - a valid origin with an empty policy becomes public-only. -pub fn resolve_resource_authorization( - app_config: &AppConfig, - input_name: Option<&Arc>, -) -> Result { - let Some(input_name) = input_name else { - return Ok(ResolvedResourceAuthorization::public_only()); - }; - - let sources = app_config.sources.load(); - // The lookup only contains enabled inputs and their enabled aliases, so an unknown name and a - // disabled input are both rejected here. - let Some(main_name) = sources.group_lookup.get(input_name) else { - return Err(ResourcePolicyError::UnknownOrigin(input_name.to_string())); - }; - let Some(input) = sources.get_input_by_name(main_name) else { - return Err(ResourcePolicyError::UnknownOrigin(input_name.to_string())); - }; - - Ok(ResolvedResourceAuthorization::for_input(Arc::clone(main_name), input.resource_policy.as_ref())) -} - -/// Resolves provenance before the URL reaches any transport client. A valid locator is -/// authoritative and can never fall back to the containing record's input. -pub fn resolve_resource( - app_config: &AppConfig, - value: &str, - legacy_input_name: Option<&Arc>, -) -> Result { - let locator = resolve_resource_value(value).map_err(|_| ResourcePolicyError::UnsupportedScheme)?; - let (url, input_name) = locator.map_or_else( - || (Arc::from(value), legacy_input_name.cloned()), - |locator| (locator.url, Some(locator.input_name)), - ); - let authorization = resolve_resource_authorization(app_config, input_name.as_ref())?; - Ok(ResolvedResource { url, authorization }) -} - -/// Decodes a resource link with the decoder that belongs to the route. -/// -/// New links are authenticated tokens and carry their origin; legacy links were issued before -/// origin tracking and stay public-only. The two encodings are never tried against each other, so -/// a malformed token cannot downgrade to a weaker interpretation. -pub fn decode_resource_link(secret: &[u8; 16], encoded: &str, legacy_decode: F) -> Result -where - F: FnOnce(&[u8; 16], &str) -> Result, -{ - if has_resource_token_prefix(encoded) { - return decode_resource_token(secret, encoded).map(|token| token.resource).map_err(|_| StatusCode::BAD_REQUEST); - } - legacy_decode(secret, encoded).map_err(|()| StatusCode::BAD_REQUEST) -} - -/// Maps a policy rejection to the status the client sees. -/// -/// A rejected upstream is answered like the reverse proxy would answer a broken upstream, which -/// also keeps the internal allowlist out of the response. An unresolvable origin stays a client -/// error because the link or record itself is no longer usable. -pub const fn rejection_status(error: &ResourcePolicyError) -> StatusCode { - match error { - ResourcePolicyError::UnknownOrigin(_) => StatusCode::BAD_REQUEST, - ResourcePolicyError::BlockedAddress - | ResourcePolicyError::HostNotTrusted - | ResourcePolicyError::NetworkNotTrusted - | ResourcePolicyError::TooManyRedirects - | ResourcePolicyError::UnsupportedScheme - | ResourcePolicyError::MissingHost => StatusCode::BAD_GATEWAY, - } -} - -/// At most one warning per input and minute: a hostile playlist can generate unlimited rejections, -/// and the operator only needs the first occurrences to notice. -const REJECTION_LOG_WINDOW: Duration = Duration::from_mins(1); -const REJECTION_LOG_MAX_ENTRIES: usize = 512; - -static REJECTION_LOG: LazyLock>> = LazyLock::new(|| Mutex::new(HashMap::new())); - -/// Logs a rejected resource request with the origin input name. -/// -/// The destination URL is never part of the warning: it is untrusted input and the operator needs -/// the origin to fix the configuration, not the failing URL. -pub fn log_resource_rejection(input_name: Option<&str>, error: &ResourcePolicyError, resource_url: &str) { - debug!( - "Rejected resource request from input '{}': {error} ({})", - input_name.unwrap_or(""), - sanitize_sensitive_info(resource_url) - ); - - let key = input_name.unwrap_or("").to_string(); - let now = Instant::now(); - { - let Ok(mut log_state) = REJECTION_LOG.lock() else { - warn!("Rejected resource request from input '{key}': {error}"); - return; - }; - let visible = match log_state.get(&key) { - Some(last) => now.duration_since(*last) >= REJECTION_LOG_WINDOW, - None => true, - }; - if !visible { - return; - } - if log_state.len() >= REJECTION_LOG_MAX_ENTRIES { - log_state.retain(|_, last| now.duration_since(*last) < REJECTION_LOG_WINDOW); - } - log_state.insert(key.clone(), now); - } - - warn!("Rejected resource request from input '{key}': {error}"); -} - -/// Reports a request whose policy is not served by any current client. -/// -/// This happens when the configuration changed between resolving the origin and using it. The -/// request is rejected instead of falling back to a broader client. -pub fn log_missing_resource_client(input_name: Option<&str>, digest: &PolicyDigest) { - error!( - "No resource client for policy {} of input '{}'; rejecting instead of falling back", - digest.as_str(), - input_name.unwrap_or("") - ); -} - -#[cfg(test)] -mod tests { - use super::{rejection_status, ResourceCacheMode, ResourceFetchOptions}; - use axum::http::StatusCode; - use tuliprox_core::model::{public_only_policy, ResourcePolicyError}; - - #[test] - fn epg_and_cached_options_differ_in_redirect_and_cache_mode() { - let authorization = super::ResolvedResourceAuthorization::public_only(); - let epg = ResourceFetchOptions::epg(authorization.clone()); - let cached = ResourceFetchOptions::cached(authorization); - - assert_eq!(epg.cache_mode, ResourceCacheMode::Disabled); - assert_eq!(cached.cache_mode, ResourceCacheMode::Enabled); - assert_ne!(epg.redirect_mode, cached.redirect_mode); - } - - #[test] - fn rejection_status_separates_origin_errors_from_destination_errors() { - assert_eq!(rejection_status(&ResourcePolicyError::UnknownOrigin("gone".to_string())), StatusCode::BAD_REQUEST); - assert_eq!(rejection_status(&ResourcePolicyError::HostNotTrusted), StatusCode::BAD_GATEWAY); - assert_eq!(rejection_status(&ResourcePolicyError::BlockedAddress), StatusCode::BAD_GATEWAY); - } - - #[test] - fn originless_authorization_is_public_only() { - let authorization = super::ResolvedResourceAuthorization::public_only(); - assert_eq!(authorization.input_name, None); - assert_eq!(authorization.policy_digest, public_only_policy().digest()); - assert!(authorization.policy.is_empty()); - } - - #[test] - fn decode_resource_link_prefers_the_authenticated_token() { - let secret = [5u8; 16]; - let encoded = tuliprox_core::utils::encode_resource_token( - &secret, - &tuliprox_core::utils::resource_token("resource://v1/example"), - ) - .expect("encode"); - - let decoded = super::decode_resource_link(&secret, &encoded, |_, _| Err(())).expect("decode"); - - assert_eq!(decoded, "resource://v1/example"); - } - - #[test] - fn decode_resource_link_never_falls_back_to_the_legacy_decoder() { - let secret = [5u8; 16]; - // A malformed token must not be handed to the route's legacy decoder. - let malformed = format!("{}not-a-token", tuliprox_core::utils::RESOURCE_TOKEN_PREFIX); - - assert_eq!( - super::decode_resource_link(&secret, &malformed, |_, _| panic!("legacy decoder")), - Err(StatusCode::BAD_REQUEST) - ); - } - - #[test] - fn decode_resource_link_uses_the_route_decoder_for_legacy_values() { - let secret = [5u8; 16]; - - let legacy = - super::decode_resource_link(&secret, "legacy", |_, _| Ok("https://cdn.example.com/a.png".to_string())) - .expect("legacy decode"); - - assert_eq!(legacy, "https://cdn.example.com/a.png"); - assert_eq!(super::decode_resource_link(&secret, "broken", |_, _| Err(())).err(), Some(StatusCode::BAD_REQUEST)); - } - - #[test] - fn input_authorization_without_policy_stays_public_only_but_keeps_the_name() { - let authorization = super::ResolvedResourceAuthorization::for_input("local".into(), None); - assert_eq!(authorization.input_name.as_deref(), Some("local")); - assert_eq!(authorization.policy_digest, public_only_policy().digest()); - } -} diff --git a/backend/app/src/api/api_utils/tests.rs b/backend/app/src/api/api_utils/tests.rs index f07e6a303..325e0b0e4 100644 --- a/backend/app/src/api/api_utils/tests.rs +++ b/backend/app/src/api/api_utils/tests.rs @@ -1,9 +1,9 @@ use super::*; use crate::{ api::model::{ - empty_resource_client_set, ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, ConnectionManager, - EventManager, MetadataUpdateManager, PlaylistStorageState, ProviderConfig as RuntimeProviderConfig, - ProviderConfigConnection, ResourceClientSet, SharedStreamManager, + ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, ConnectionManager, EventManager, + MetadataUpdateManager, PlaylistStorageState, ProviderConfig as RuntimeProviderConfig, ProviderConfigConnection, + SharedStreamManager, }, auth::Fingerprint, model::{ @@ -27,10 +27,9 @@ use shared::{ defaults::{default_catchup_session_ttl_secs, default_hls_session_ttl_secs}, foundation::Filter, model::{ - provider_saturation::build_group_lookup, AdmissionStrategy, ClusterFlags, ConfigPaths, ConfigProviderDto, - ConfigTargetOptions, GeoIpUnavailablePolicy, InputFetchMethod, InputType, PlaylistItem, PlaylistItemHeader, - PlaylistItemType, ProcessingOrder, ProviderUrlSelectionPolicy, ProxyType, ResourcePolicyDto, StreamChannel, - TargetType, XtreamCluster, + AdmissionStrategy, ClusterFlags, ConfigPaths, ConfigProviderDto, ConfigTargetOptions, GeoIpUnavailablePolicy, + InputFetchMethod, InputType, PlaylistItem, PlaylistItemHeader, PlaylistItemType, ProcessingOrder, + ProviderUrlSelectionPolicy, ProxyType, StreamChannel, TargetType, XtreamCluster, }, utils::Internable, }; @@ -40,10 +39,7 @@ use tokio::{ net::TcpListener, sync::{mpsc, RwLock}, }; -use tuliprox_core::{ - model::{public_only_policy, ResourceClientKey, ResourcePolicy, ResourcePolicyError, ResourceRedirectMode}, - utils::{resource_cache_key, response_compression::should_compress_response}, -}; +use tuliprox_core::utils::response_compression::should_compress_response; use tuliprox_session::{ admission::{evaluate_remaining_strategies_after_grace, get_effective_admission_strategies}, AdmissionRejectionReason, GraceResolutionContext, @@ -4459,83 +4455,6 @@ fn create_test_app_config() -> AppConfig { } } -/// App config whose only input carries `policy` and one enabled alias. -fn create_policy_app_config(policy: ResourcePolicyDto) -> AppConfig { - let config = create_test_app_config(); - let policy = Arc::new(ResourcePolicy::from_dto(&policy).expect("test policy")); - let input = Arc::new(ConfigInput { - id: 1, - name: "main-input".intern(), - input_type: InputType::M3u, - url: "https://provider.example/playlist.m3u".to_string(), - enabled: true, - resource_policy: Some(policy), - aliases: Some(vec![ConfigInputAlias { - id: 2, - name: "aliased-input".intern(), - url: "https://provider.example/alias.m3u".to_string(), - username: None, - password: None, - priority: 0, - max_connections: 1, - exp_date: None, - enabled: true, - stalker: None, - }]), - ..ConfigInput::default() - }); - let inputs = vec![Arc::clone(&input)]; - let sources = SourcesConfig { inputs, group_lookup: build_group_lookup(&[input]), ..SourcesConfig::default() }; - config.sources.store(Arc::new(sources)); - config -} - -#[test] -fn resource_authorization_uses_the_main_input_policy_for_an_alias() { - let app_config = create_policy_app_config(ResourcePolicyDto { - allowed_hosts: vec!["media.home.arpa".to_string()], - allowed_networks: vec!["192.168.50.20/32".to_string()], - }); - let input_name = Arc::from("aliased-input"); - let authorization = resolve_resource_authorization(&app_config, Some(&input_name)).expect("authorized"); - - assert_eq!(authorization.input_name.as_deref(), Some("main-input")); - assert!(authorization.policy.allows_host("media.home.arpa")); - assert_ne!(authorization.policy_digest, public_only_policy().digest()); -} - -#[test] -fn resource_authorization_rejects_an_unknown_or_disabled_origin() { - let app_config = create_policy_app_config(ResourcePolicyDto::default()); - let input_name = Arc::from("removed-input"); - let error = resolve_resource_authorization(&app_config, Some(&input_name)).expect_err("rejected"); - - assert_eq!(error, ResourcePolicyError::UnknownOrigin("removed-input".to_string())); - assert_eq!(rejection_status(&error), StatusCode::BAD_REQUEST); -} - -#[test] -fn resource_authorization_follows_a_config_change_without_regenerating_links() { - let app_config = create_policy_app_config(ResourcePolicyDto { - allowed_hosts: vec!["first.home.arpa".to_string()], - allowed_networks: vec!["192.168.50.20/32".to_string()], - }); - let input_name = Arc::from("main-input"); - let before = resolve_resource_authorization(&app_config, Some(&input_name)).expect("authorized"); - - let reloaded = create_policy_app_config(ResourcePolicyDto { - allowed_hosts: vec!["second.home.arpa".to_string()], - allowed_networks: vec!["10.0.0.0/8".to_string()], - }); - app_config.sources.store(Arc::clone(&reloaded.sources.load())); - - let after = resolve_resource_authorization(&app_config, Some(&input_name)).expect("authorized"); - - assert_ne!(before.policy_digest, after.policy_digest); - assert!(after.policy.allows_host("second.home.arpa")); - assert!(!after.policy.allows_host("first.home.arpa")); -} - fn create_test_provider_app_config() -> AppConfig { let input = Arc::new(ConfigInput { id: 1, @@ -4685,7 +4604,8 @@ fn create_test_app_state_for_config(app_cfg: Arc) -> Arc { http_client: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(crate::api::model::DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, @@ -4716,30 +4636,8 @@ fn create_test_fingerprint(addr: std::net::SocketAddr) -> Fingerprint { Fingerprint::new(format!("fp-{addr}"), addr.ip().to_string(), addr) } -#[test] -fn resource_credential_context_separates_configured_and_inbound_credentials() { - let mut input = ConfigInput { - name: "input".into(), - username: Some("user".to_string()), - password: Some("password-a".to_string()), - ..ConfigInput::default() - }; - let no_headers = HashMap::new(); - let configured_a = resource_credential_context(Some(&input), &no_headers); - input.password = Some("password-b".to_string()); - let configured_b = resource_credential_context(Some(&input), &no_headers); - assert_ne!(configured_a, configured_b); - - let mut request_headers = HashMap::new(); - request_headers.insert("authorization".to_string(), b"Bearer a".to_vec()); - let inbound_a = resource_credential_context(None, &request_headers); - request_headers.insert("authorization".to_string(), b"Bearer b".to_vec()); - let inbound_b = resource_credential_context(None, &request_headers); - assert_ne!(inbound_a, inbound_b); -} - #[tokio::test] -async fn resource_cache_is_scoped_to_the_authorizing_policy() { +async fn resource_cache_is_used_only_by_matching_public_fetch_policy() { const CACHED_BODY: &[u8] = b"cached image"; const UPSTREAM_BODY: &[u8] = b"upstream image"; @@ -4748,18 +4646,9 @@ async fn resource_cache_is_scoped_to_the_authorizing_policy() { let cache_dir = temp_dir.path().to_string_lossy(); let mut cache = LRUResourceCache::new(1024, cache_dir.as_ref()); let resource_url = "http://1.1.1.1/icon.png"; - - // An entry stored under the public-only scope: it must never answer a request authorized by a - // policy that is allowed to reach private destinations. - let public_only_scope = resource_cache_key( - public_only_policy().digest().as_str(), - "", - &resource_credential_context(None, &HashMap::new()), - resource_url, - ); - let cached_path = cache.store_path(&public_only_scope, Some("image/png")); + let cached_path = cache.store_path(resource_url, Some("image/png")); tokio::fs::write(&cached_path, CACHED_BODY).await.expect("write cached image"); - cache.add_content(&public_only_scope, Some("image/png".to_string()), CACHED_BODY.len()).expect("cache entry"); + cache.add_content(resource_url, Some("image/png".to_string()), CACHED_BODY.len()).expect("register cached image"); app_state.cache.store(Some(Arc::new(RwLock::new(cache)))); let response_head = format!( @@ -4768,162 +4657,442 @@ async fn resource_cache_is_scoped_to_the_authorizing_policy() { ); let (upstream_addr, upstream_task) = spawn_legacy_hls_test_origin(response_head, UPSTREAM_BODY.to_vec()).await; let proxy = reqwest::Proxy::http(format!("http://{upstream_addr}")).expect("mock proxy URL"); - let mock_client = reqwest::Client::builder().proxy(proxy).build().expect("mock upstream client"); + let mock_client = Arc::new(reqwest::Client::builder().proxy(proxy).build().expect("mock upstream client")); + // The destination is public, so the hop is fetched through the public resource client; both resource + // clients are mocked so the assertion below is about the cache, not about which client performed the + // request. + app_state.resource_public_http_client_no_redirect.store(Arc::clone(&mock_client)); + app_state.resource_http_client_no_redirect.store(mock_client); - let public_only = ResolvedResourceAuthorization::public_only(); - let private_policy = Arc::new( - ResourcePolicy::from_dto(&shared::model::ResourcePolicyDto { - allowed_hosts: vec!["media.home.arpa".to_string()], - allowed_networks: vec!["192.168.50.20/32".to_string()], - }) - .expect("policy"), - ); - let scoped = ResolvedResourceAuthorization::for_input("private".into(), Some(&private_policy)); - assert_ne!(public_only.policy_digest, scoped.policy_digest); - - let mut clients = HashMap::new(); - for key in [ - ResourceClientKey::new(public_only.policy_digest.clone(), ResourceRedirectMode::Bounded), - ResourceClientKey::new(scoped.policy_digest.clone(), ResourceRedirectMode::Bounded), - ] { - clients.insert(key, mock_client.clone()); - } - app_state.resource_clients.store(Arc::new(ResourceClientSet::from_clients(clients))); - - let cached_response = resource_response( - &app_state, - ResourceFetchOptions::cached(public_only.clone()), - resource_url, - &HeaderMap::new(), - None, - ) - .await - .into_response(); - assert_eq!(cached_response.status(), StatusCode::OK); - let cached_body = cached_response.into_body().collect().await.expect("read cached image").to_bytes(); - assert_eq!(cached_body, Bytes::from_static(CACHED_BODY)); - - let fresh_response = - resource_response(&app_state, ResourceFetchOptions::cached(scoped), resource_url, &HeaderMap::new(), None) + let public_response = + resource_response(&app_state, ResourceFetchPolicy::Public, resource_url, &HeaderMap::new(), None) .await .into_response(); - assert_eq!(fresh_response.status(), StatusCode::OK); - let fresh_body = fresh_response.into_body().collect().await.expect("read upstream image").to_bytes(); - assert_eq!(fresh_body, Bytes::from_static(UPSTREAM_BODY)); - assert_ne!(fresh_body, Bytes::from_static(CACHED_BODY)); + assert_eq!(public_response.status(), StatusCode::OK); + let public_body = public_response.into_body().collect().await.expect("read cached image").to_bytes(); + assert_eq!(public_body, Bytes::from_static(CACHED_BODY)); + + let response = resource_response(&app_state, ResourceFetchPolicy::NonPublic, resource_url, &HeaderMap::new(), None) + .await + .into_response(); + + assert_eq!(response.status(), StatusCode::OK); + let response_body = response.into_body().collect().await.expect("read upstream image").to_bytes(); + assert_eq!(response_body, Bytes::from_static(UPSTREAM_BODY)); + assert_ne!(response_body, Bytes::from_static(CACHED_BODY)); let upstream_request = upstream_task.await.expect("mock upstream task completes"); assert!(upstream_request.starts_with("GET http://1.1.1.1/icon.png HTTP/1.1\r\n")); } #[tokio::test] -async fn resource_response_bypasses_shared_cache_for_forwarded_authorization() { - const CACHED_BODY: &[u8] = b"cached private image"; - const UPSTREAM_BODY: &[u8] = b"upstream private image"; - +async fn no_redirect_resource_ignores_configured_proxy() { let app_state = create_test_app_state(); - let temp_dir = tempfile::tempdir().expect("tempdir"); - let cache_dir = temp_dir.path().to_string_lossy(); - let mut cache = LRUResourceCache::new(1024, cache_dir.as_ref()); - let resource_url = "http://1.1.1.1/private-icon.png"; - let authorization = ResolvedResourceAuthorization::public_only(); - let credential_headers = HashMap::from([("authorization".to_string(), b"Bearer private".to_vec())]); - let cache_key = resource_cache_key( - authorization.policy_digest.as_str(), - "", - &resource_credential_context(None, &credential_headers), - resource_url, + let response_head = + "HTTP/1.1 302 Found\r\nLocation: http://10.0.0.2/other.png\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), Vec::new()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { url: format!("http://{proxy_addr}"), username: None, password: None }), + ..Config::default() + })); + let client = crate::api::model::create_resource_http_client_no_redirect(&app_state.app_config) + .expect("resource HTTP client"); + // 192.0.2.1 (TEST-NET-1) is not routable, so a direct attempt fails by timing out or refusing the + // connection. A response would mean the request was answered by the proxy after all. + let result = client.get("http://192.0.2.1/icon.png").timeout(std::time::Duration::from_millis(100)).send().await; + assert!( + result.as_ref().is_err_and(|err| err.is_timeout() || err.is_connect()), + "the direct path must be attempted: {result:?}" ); - let cached_path = cache.store_path(&cache_key, Some("image/png")); - tokio::fs::write(&cached_path, CACHED_BODY).await.expect("write cached image"); - cache.add_content(&cache_key, Some("image/png".to_string()), CACHED_BODY.len()).expect("cache entry"); - app_state.cache.store(Some(Arc::new(RwLock::new(cache)))); + assert!( + tokio::time::timeout(std::time::Duration::from_millis(100), proxy_task).await.is_err(), + "resource client must not send requests through the configured proxy" + ); +} - let response_head = format!( - "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: {}\r\nCache-Control: public, max-age=60\r\nConnection: close\r\n\r\n", - UPSTREAM_BODY.len() - ); - let (upstream_addr, upstream_task) = spawn_legacy_hls_test_origin(response_head, UPSTREAM_BODY.to_vec()).await; - let proxy = reqwest::Proxy::http(format!("http://{upstream_addr}")).expect("mock proxy URL"); - let mock_client = reqwest::Client::builder().proxy(proxy).build().expect("mock upstream client"); - let mut clients = HashMap::new(); - clients.insert( - ResourceClientKey::new(authorization.policy_digest.clone(), ResourceRedirectMode::Bounded), - mock_client, - ); - app_state.resource_clients.store(Arc::new(ResourceClientSet::from_clients(clients))); +#[tokio::test] +async fn public_resource_client_honours_the_configured_proxy() { + // Counterpart of `no_redirect_resource_ignores_configured_proxy`: a resource hop that can leave the + // local network must go through the configured proxy, or the fetch discloses this host's address. + let app_state = create_test_app_state(); + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { url: format!("http://{proxy_addr}"), username: None, password: None }), + ..Config::default() + })); + let client = crate::api::model::create_resource_public_http_client_no_redirect(&app_state.app_config) + .expect("resource HTTP client"); - let mut request_headers = HeaderMap::new(); - request_headers.insert(header::AUTHORIZATION, HeaderValue::from_static("Bearer private")); - let response = resource_response( - &app_state, - ResourceFetchOptions::cached(authorization), - resource_url, - &request_headers, - None, - ) - .await - .into_response(); + let response = client.get("http://8.8.8.8/icon.png").send().await.expect("request reaches the proxy"); assert_eq!(response.status(), StatusCode::OK); - assert_eq!( - response.headers().get(header::CACHE_CONTROL).and_then(|value| value.to_str().ok()), - Some("private, no-store") - ); - let body = response.into_body().collect().await.expect("read upstream image").to_bytes(); - assert_eq!(body, Bytes::from_static(UPSTREAM_BODY)); - assert_ne!(body, Bytes::from_static(CACHED_BODY)); - - let upstream_request = tokio::time::timeout(std::time::Duration::from_secs(1), upstream_task) + let request = tokio::time::timeout(std::time::Duration::from_secs(2), proxy_task) .await - .expect("credential-varying request was not sent") - .expect("mock upstream task completes"); - assert!(upstream_request.to_ascii_lowercase().contains("authorization: bearer private\r\n")); + .expect("request must reach proxy") + .expect("proxy task"); + assert!(request.starts_with("GET http://8.8.8.8/icon.png HTTP/1.1\r\n"), "{request}"); } #[tokio::test] -async fn resource_response_rejects_blocked_ip_literal_before_request() { +async fn an_unresolved_resource_name_is_fetched_through_the_configured_proxy() { + // A resource name that does not resolve locally is still resolvable through a configured proxy (for + // example with remote DNS), so the fetch must not be pinned to the direct client. let app_state = create_test_app_state(); - let response_head = "HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n".to_string(); - let (origin_addr, mut origin_task) = spawn_legacy_hls_test_origin(response_head, Vec::new()).await; - let authorization = ResolvedResourceAuthorization::public_only(); - let mut clients = HashMap::new(); - clients.insert( - ResourceClientKey::new(authorization.policy_digest.clone(), ResourceRedirectMode::Bounded), - reqwest::Client::builder().no_proxy().build().expect("test resource client"), + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { url: format!("http://{proxy_addr}"), username: None, password: None }), + ..Config::default() + })); + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock proxy client"); + app_state.resource_public_http_client_no_redirect.store(Arc::new(client)); + + let response = + resource_proxy_response(&app_state, "http://unresolved.invalid/logo.png", &HeaderMap::new(), None).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(response.into_body().collect().await.expect("image body").to_bytes(), Bytes::from_static(b"png")); + let request = tokio::time::timeout(std::time::Duration::from_secs(2), proxy_task) + .await + .expect("request must reach proxy") + .expect("proxy task"); + assert!(request.starts_with("GET http://unresolved.invalid/logo.png HTTP/1.1\r\n"), "{request}"); +} + +#[tokio::test] +async fn public_resource_client_reaches_a_proxy_on_loopback() { + // A proxy is commonly configured on the loopback interface (`http://localhost:8118`), so the + // connect-time guard of the public resource client must not reject the proxy host itself. + let app_state = create_test_app_state(); + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { + url: format!("http://localhost:{}", proxy_addr.port()), + username: None, + password: None, + }), + ..Config::default() + })); + let client = crate::api::model::create_resource_public_http_client_no_redirect(&app_state.app_config) + .expect("resource HTTP client"); + + let response = client.get("http://8.8.8.8/icon.png").send().await.expect("request reaches the proxy"); + + assert_eq!(response.status(), StatusCode::OK); + let request = tokio::time::timeout(std::time::Duration::from_secs(2), proxy_task) + .await + .expect("request must reach proxy") + .expect("proxy task"); + assert!(request.starts_with("GET http://8.8.8.8/icon.png HTTP/1.1\r\n"), "{request}"); +} + +#[tokio::test] +async fn no_redirect_resource_refuses_destinations_local_to_this_host() { + let app_state = create_test_app_state(); + let (proxy_addr, proxy_task) = + spawn_legacy_hls_test_origin("HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n".to_string(), Vec::new()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { url: format!("http://{proxy_addr}"), username: None, password: None }), + ..Config::default() + })); + let client = crate::api::model::create_resource_http_client_no_redirect(&app_state.app_config) + .expect("resource HTTP client"); + app_state.resource_http_client_no_redirect.store(Arc::new(client)); + + for local_only in ["http://127.0.0.1/icon.png", "http://169.254.169.254/latest/meta-data/", "http://[::1]/icon.png"] + { + let response = + resource_response(&app_state, ResourceFetchPolicy::NonPublic, local_only, &HeaderMap::new(), None) + .await + .into_response(); + + assert_eq!(response.status(), StatusCode::FORBIDDEN, "{local_only}"); + } + + // The guard rejects before a request is built, so nothing may reach the configured proxy. + assert!( + tokio::time::timeout(std::time::Duration::from_millis(100), proxy_task).await.is_err(), + "no request may reach the proxy for local-only destinations" ); - app_state.resource_clients.store(Arc::new(ResourceClientSet::from_clients(clients))); - - let response = resource_response( - &app_state, - ResourceFetchOptions::cached(authorization), - &format!("http://{origin_addr}/icon.png"), - &HeaderMap::new(), - None, - ) - .await - .into_response(); - - assert_eq!(response.status(), StatusCode::BAD_GATEWAY); - let origin_result = tokio::time::timeout(std::time::Duration::from_millis(250), &mut origin_task).await; - origin_task.abort(); - assert!(origin_result.is_err(), "blocked loopback request reached the test origin"); } #[tokio::test] -async fn resource_response_rejects_a_policy_without_a_current_client() { +async fn resource_redirect_hides_private_destination_and_upstream_location() { let app_state = create_test_app_state(); + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nLocation: http://10.0.0.2/private\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock upstream client"); + app_state.resource_http_client_no_redirect.store(Arc::new(client)); + + let private = resource_redirect_or_proxy(&app_state, "http://10.0.0.1/icon.png", &HeaderMap::new(), None).await; + assert_eq!(private.status(), StatusCode::OK); + assert!(private.headers().get("location").is_none()); + assert_eq!(private.into_body().collect().await.expect("private icon body").to_bytes(), Bytes::from_static(b"png")); + let request = proxy_task.await.expect("mock request"); + assert!(request.starts_with("GET http://10.0.0.1/icon.png HTTP/1.1\r\n")); + + let public = resource_redirect_or_proxy(&app_state, "http://8.8.8.8/icon.png", &HeaderMap::new(), None).await; + assert_eq!(public.status(), StatusCode::FOUND); + assert_eq!(public.headers().get("location").and_then(|value| value.to_str().ok()), Some("http://8.8.8.8/icon.png")); + + let local_thumbnail = + resource_redirect_or_proxy(&app_state, "/api/v1/library/thumbnail/item", &HeaderMap::new(), None).await; + assert_eq!(local_thumbnail.status(), StatusCode::FOUND); + assert_eq!( + local_thumbnail.headers().get("location").and_then(|value| value.to_str().ok()), + Some("/api/v1/library/thumbnail/item") + ); + + let blocked = resource_redirect_or_proxy(&app_state, "http://127.0.0.1/icon.png", &HeaderMap::new(), None).await; + assert_eq!(blocked.status(), StatusCode::FORBIDDEN); + assert!(blocked.headers().get("location").is_none()); +} + +#[tokio::test] +async fn proxied_m3u_logo_follows_private_redirect_without_forwarding_credentials() { + let app_state = create_test_app_state(); + let listener = TcpListener::bind("127.0.0.1:0").await.expect("mock proxy binds"); + let proxy_addr = listener.local_addr().expect("mock proxy address"); + let proxy_task = tokio::spawn(async move { + let mut requests = Vec::new(); + for response in [ + "HTTP/1.1 302 Found\r\nLocation: http://10.0.0.2/icon.png\r\nContent-Length: 0\r\nConnection: close\r\n\r\n", + "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\npng", + ] { + let (mut socket, _) = listener.accept().await.expect("mock proxy accepts request"); + let mut request = Vec::new(); + while !request.windows(4).any(|window| window == b"\r\n\r\n") { + let mut chunk = [0_u8; 1024]; + let read = socket.read(&mut chunk).await.expect("mock proxy reads request"); + if read == 0 { + break; + } + request.extend_from_slice(&chunk[..read]); + } + socket.write_all(response.as_bytes()).await.expect("mock proxy writes response"); + requests.push(String::from_utf8_lossy(&request).into_owned()); + } + requests + }); + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("mock proxy URL")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock resource client"); + app_state.resource_http_client_no_redirect.store(Arc::new(client)); + let input = ConfigInput { + url: "http://10.0.0.1/playlist.m3u".to_string(), + headers: HashMap::from([("Authorization".to_string(), "Bearer secret".to_string())]), + ..ConfigInput::default() + }; + let mut headers = HeaderMap::new(); + headers.insert(header::AUTHORIZATION, HeaderValue::from_static("Bearer player-secret")); + + let response = resource_proxy_response(&app_state, "http://10.0.0.1/logo.png", &headers, Some(&input)).await; + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(response.into_body().collect().await.expect("image body").to_bytes(), Bytes::from_static(b"png")); + + let requests = proxy_task.await.expect("mock proxy task"); + assert!(requests[0].starts_with("GET http://10.0.0.1/logo.png HTTP/1.1\r\n")); + assert!(requests[0].to_ascii_lowercase().contains("authorization: bearer secret")); + assert!(!requests[0].contains("player-secret")); + assert!(requests[1].starts_with("GET http://10.0.0.2/icon.png HTTP/1.1\r\n")); + assert!(!requests[1].to_ascii_lowercase().contains("authorization:")); +} + +#[test] +fn configured_resource_headers_are_limited_to_the_input_origin() { + let input = ConfigInput { url: "http://10.0.0.1:8080/playlist.m3u".to_string(), ..ConfigInput::default() }; + assert!(resource_input_for_url(Some(&input), "http://10.0.0.1:8080/logo.png").is_some()); + assert!(resource_input_for_url(Some(&input), "http://10.0.0.2:8080/logo.png").is_none()); + assert!(resource_input_for_url(Some(&input), "http://10.0.0.1:8081/logo.png").is_none()); + assert!(resource_input_for_url(Some(&input), "https://10.0.0.1:8080/logo.png").is_none()); +} + +#[tokio::test] +async fn public_resource_is_fetched_through_the_configured_proxy() { + // Regression lock: a resource fetch that leaves the local network must use the configured proxy, + // otherwise it discloses the operator's address to the resource host. The Web UI resource route + // reaches this path, because it wraps every icon of a public destination into a proxy link. + let app_state = create_test_app_state(); + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("test proxy")) + .build() + .expect("mock proxy client"); + app_state.resource_public_http_client_no_redirect.store(Arc::new(client)); + + let response = resource_proxy_response(&app_state, "http://8.8.8.8/logo.png", &HeaderMap::new(), None).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(response.into_body().collect().await.expect("image body").to_bytes(), Bytes::from_static(b"png")); + let request = tokio::time::timeout(std::time::Duration::from_secs(2), proxy_task) + .await + .expect("request must reach proxy") + .expect("proxy task"); + assert!(request.starts_with("GET http://8.8.8.8/logo.png HTTP/1.1\r\n"), "{request}"); +} + +#[tokio::test] +async fn proxied_resource_follows_a_redirect_into_the_public_network_through_the_configured_proxy() { + // Regression lock: the hop decides which client is used. A network-internal destination that + // redirects to a public host must not make the follow-up request directly, or the redirect would + // disclose the operator's address to that host despite a configured proxy. + let app_state = create_test_app_state(); + let redirect_head = + "HTTP/1.1 302 Found\r\nLocation: http://8.8.8.8/logo.png\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"; + let (private_proxy_addr, private_proxy_task) = + spawn_legacy_hls_test_origin(redirect_head.to_string(), Vec::new()).await; + let private_client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{private_proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock private client"); + app_state.resource_http_client_no_redirect.store(Arc::new(private_client)); + + let response_head = "HTTP/1.1 200 OK\r\nContent-Type: image/png\r\nContent-Length: 3\r\nConnection: close\r\n\r\n"; + let (public_proxy_addr, public_proxy_task) = + spawn_legacy_hls_test_origin(response_head.to_string(), b"png".to_vec()).await; + let public_client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{public_proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock public client"); + app_state.resource_public_http_client_no_redirect.store(Arc::new(public_client)); + + let response = resource_proxy_response(&app_state, "http://10.0.0.1/logo.png", &HeaderMap::new(), None).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(response.into_body().collect().await.expect("image body").to_bytes(), Bytes::from_static(b"png")); + let first_hop = private_proxy_task.await.expect("first hop request"); + assert!(first_hop.starts_with("GET http://10.0.0.1/logo.png HTTP/1.1\r\n"), "{first_hop}"); + let second_hop = tokio::time::timeout(std::time::Duration::from_secs(2), public_proxy_task) + .await + .expect("public hop must be fetched") + .expect("public hop request"); + assert!(second_hop.starts_with("GET http://8.8.8.8/logo.png HTTP/1.1\r\n"), "{second_hop}"); +} + +#[tokio::test] +async fn public_resource_redirect_to_a_destination_local_to_this_host_is_refused() { + // Regression lock: a public host that redirects to loopback, link-local, or a cloud metadata + // endpoint must not turn the resource route into a reader for this host. The redirect target is + // never requested, and its location is never relayed to the client. + let app_state = create_test_app_state(); + let listener = TcpListener::bind("127.0.0.1:0").await.expect("mock proxy binds"); + let proxy_addr = listener.local_addr().expect("mock proxy address"); + let proxy_task = tokio::spawn(async move { + let mut requests = Vec::new(); + loop { + let Ok(accepted) = tokio::time::timeout(std::time::Duration::from_millis(500), listener.accept()).await + else { + break; + }; + let Ok((mut socket, _)) = accepted else { break }; + let mut request = Vec::new(); + while !request.windows(4).any(|window| window == b"\r\n\r\n") { + let mut chunk = [0_u8; 1024]; + match socket.read(&mut chunk).await { + Ok(0) | Err(_) => break, + Ok(read) => request.extend_from_slice(&chunk[..read]), + } + } + requests.push(String::from_utf8_lossy(&request).into_owned()); + let head = "HTTP/1.1 302 Found\r\nLocation: http://127.0.0.1/secret\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"; + let _ = socket.write_all(head.as_bytes()).await; + } + requests + }); + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock client"); + app_state.resource_public_http_client_no_redirect.store(Arc::new(client)); + + let response = resource_proxy_response(&app_state, "http://8.8.8.8/logo.png", &HeaderMap::new(), None).await; + + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + assert!(response.headers().get("location").is_none(), "the local destination must not be relayed"); + let requests = proxy_task.await.expect("mock proxy task"); + assert_eq!(requests.len(), 1, "the local destination must not be requested: {requests:?}"); +} + +#[tokio::test] +async fn no_redirect_resource_does_not_relay_upstream_error_details() { + let app_state = create_test_app_state(); + let body = b"internal service error".to_vec(); + let response_head = + format!("HTTP/1.1 404 Not Found\r\nContent-Type: text/html\r\nContent-Length: {}\r\n\r\n", body.len()); + let (proxy_addr, proxy_task) = spawn_legacy_hls_test_origin(response_head, body.clone()).await; + app_state.app_config.config.store(Arc::new(Config { + proxy: Some(crate::model::ProxyConfig { url: format!("http://{proxy_addr}"), username: None, password: None }), + ..Config::default() + })); + let client = reqwest::Client::builder() + .proxy(reqwest::Proxy::all(format!("http://{proxy_addr}")).expect("test proxy")) + .redirect(reqwest::redirect::Policy::none()) + .build() + .expect("mock upstream client"); + app_state.resource_http_client_no_redirect.store(Arc::new(client)); + let response = resource_response( &app_state, - ResourceFetchOptions::cached(ResolvedResourceAuthorization::public_only()), - "http://1.1.1.1/icon.png", + ResourceFetchPolicy::NonPublic, + "http://10.0.0.1/icon.png", &HeaderMap::new(), None, ) .await .into_response(); + // The client learns that the fetch failed, not what the destination answered. assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + let relayed = response.into_body().collect().await.expect("response body").to_bytes(); + assert!(relayed.is_empty(), "upstream error body must not be relayed: {relayed:?}"); + + let request = tokio::time::timeout(std::time::Duration::from_secs(2), proxy_task) + .await + .expect("request must reach proxy") + .expect("proxy task"); + assert!(request.starts_with("GET http://10.0.0.1/icon.png HTTP/1.1\r\n"), "{request}"); +} + +#[tokio::test] +async fn resource_client_refuses_names_that_resolve_to_local_addresses() { + let app_state = create_test_app_state(); + let (origin_addr, _origin_task) = + spawn_legacy_hls_test_origin("HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n".to_string(), Vec::new()).await; + let guarded = crate::api::model::create_resource_http_client_no_redirect(&app_state.app_config) + .expect("resource HTTP client"); + + // `localhost` resolves to loopback on the client's DNS layer, so the request must fail while the + // name is resolved and never reach the origin that is listening on that loopback address. + let error = guarded + .get(format!("http://localhost:{}/icon.png", origin_addr.port())) + .send() + .await + .expect_err("a name resolving to a local address must be refused"); + + // reqwest only reports the outer failure, so the cause chain has to be walked to see the refusal. + let mut causes = error.to_string(); + let mut source = std::error::Error::source(&error); + while let Some(cause) = source { + causes.push_str(&format!(" | {cause}")); + source = cause.source(); + } + assert!(causes.contains("local to this host"), "{causes}"); } fn create_test_fingerprint_with_user_agent(addr: std::net::SocketAddr, user_agent: &str) -> Fingerprint { diff --git a/backend/app/src/api/endpoints/api_playlist_utils.rs b/backend/app/src/api/endpoints/api_playlist_utils.rs index 4a588ac67..b9aaab91a 100644 --- a/backend/app/src/api/endpoints/api_playlist_utils.rs +++ b/backend/app/src/api/endpoints/api_playlist_utils.rs @@ -4,7 +4,6 @@ use crate::{ empty_json_list_response, json_or_bin_response, stream_json_or_bin_response_stream, stream_json_or_bin_response_try_stream, }, - endpoints::xmltv_api::encode_resource_link, model::AppState, }, iptv::{m3u, xtream}, @@ -18,11 +17,8 @@ use crate::{ use axum::response::IntoResponse; use serde_json::json; use shared::{ - model::{ - resolve_resource_value, InputPersistence, M3uPlaylistItem, ResourceLocator, TargetType, UiPlaylistItem, - XtreamCluster, XtreamPlaylistItem, - }, - utils::{concat_path, concat_path_leading_slash, interner_gc, obfuscate_text, Internable}, + model::{InputPersistence, M3uPlaylistItem, TargetType, UiPlaylistItem, XtreamCluster, XtreamPlaylistItem}, + utils::{concat_path, concat_path_leading_slash, interner_gc, seal_web_ui_resource_url, Internable}, }; use std::sync::Arc; use tokio_stream::StreamExt; @@ -104,38 +100,29 @@ pub(in crate::api::endpoints) async fn get_playlist_for_target( pub(in crate::api::endpoints) fn rewrite_resource_url( encrypt_secret: &[u8; 16], resource_url: &str, - mut item: UiPlaylistItem, + item: UiPlaylistItem, ) -> UiPlaylistItem { - if item.logo.is_empty() || item.logo.starts_with('/') { + if item.logo.is_empty() { return item; } - let resource = match resolve_resource_value(&item.logo) { - Ok(Some(_)) => Arc::clone(&item.logo), - Ok(None) => { - let Ok(resource) = ResourceLocator::new(Arc::clone(&item.input_name), Arc::clone(&item.logo)) - .and_then(|locator| locator.encode()) - else { - item.logo = Arc::from(""); - return item; - }; - resource - } - Err(_) => { - item.logo = Arc::from(""); - return item; - } - }; - let encoded = encode_resource_link(encrypt_secret, &resource).unwrap_or_else(|| { - let external = resolve_resource_value(&resource) - .ok() - .flatten() - .map_or_else(String::new, |locator| locator.url.to_string()); - obfuscate_text(encrypt_secret, &external) - }); - item.logo = concat_path(resource_url, &encoded).intern(); + let mut item = item; + if item.logo.starts_with('/') { + return item; + } + item.logo = concat_path(resource_url, &seal_web_ui_resource_url(encrypt_secret, &item.logo)).intern(); item } +/// Item as handed to the Web UI: icon wrapped into a proxy link, playback URL routed through the +/// resource route. +/// +/// Both rewrites live in one place, so a preview path cannot ship the icon rewrite and forget the +/// other way around: an icon that is not wrapped can name a destination internal to this instance. +fn preview_item(encrypt_secret: &[u8; 16], resource_url: &str, input_id: u16, item: UiPlaylistItem) -> UiPlaylistItem { + let item = rewrite_resource_url(encrypt_secret, resource_url, item); + rewrite_stalker_playback_url(encrypt_secret, resource_url, input_id, item) +} + fn rewrite_stalker_playback_url( encrypt_secret: &[u8; 16], resource_url: &str, @@ -144,7 +131,7 @@ fn rewrite_stalker_playback_url( ) -> UiPlaylistItem { let locator = format!("{STALKER_RESOURCE_SCHEME}{input_id}/{}/{}", item.xtream_cluster.as_stream_type(), item.provider_id); - item.url = concat_path(resource_url, &obfuscate_text(encrypt_secret, &locator)).intern(); + item.url = concat_path(resource_url, &seal_web_ui_resource_url(encrypt_secret, &locator)).intern(); item } @@ -155,6 +142,8 @@ pub(in crate::api::endpoints) async fn get_playlist_for_input( accept: Option<&str>, ) -> impl IntoResponse + Send { if let Some(input) = cfg_input { + // Icon URLs are wrapped into a proxy link, so the browser never receives a destination that + // may be internal to this Tuliprox instance. let config = app_state.app_config.config.load(); let web_ui_path = config.web_ui.as_ref().and_then(|web_ui| web_ui.path.as_ref()).map_or("", String::as_str); let resource_url = concat_path_leading_slash(web_ui_path, "api/v1/playlist/resource"); @@ -201,17 +190,11 @@ pub(in crate::api::endpoints) async fn get_playlist_for_input( return (axum::http::StatusCode::BAD_REQUEST, axum::Json(json!({"error": error_strings.join(", ")}))) .into_response(); } - let config = app_state.app_config.config.load(); - let web_ui_path = config.web_ui.as_ref().and_then(|web_ui| web_ui.path.as_ref()).map_or("", String::as_str); - let resource_url = concat_path_leading_slash(web_ui_path, "api/v1/playlist/resource"); - let encrypt_secret = app_state.get_encrypt_secret(); let channels: Vec = fetch .groups .iter() .flat_map(|group| group.channels.iter()) - .map(UiPlaylistItem::from) - .map(|item| rewrite_resource_url(&encrypt_secret, &resource_url, item)) - .map(|item| rewrite_stalker_playback_url(&encrypt_secret, &resource_url, input.id, item)) + .map(|item| preview_item(&encrypt_secret, &resource_url, input.id, UiPlaylistItem::from(item))) .collect(); interner_gc(); return json_or_bin_response(accept, &channels).into_response(); @@ -298,8 +281,7 @@ pub(in crate::api::endpoints) async fn get_playlist_for_custom_provider( let input_id = input.id; let converted_stream = tokio_stream::iter(result.into_iter().flat_map(|g| g.channels).map(move |pli| { - let item = rewrite_resource_url(&encrypt_secret, &resource_url, UiPlaylistItem::from(&pli)); - rewrite_stalker_playback_url(&encrypt_secret, &resource_url, input_id, item) + preview_item(&encrypt_secret, &resource_url, input_id, UiPlaylistItem::from(&pli)) })); stream_json_or_bin_response_stream(accept, converted_stream).into_response() } @@ -312,19 +294,11 @@ pub(in crate::api::endpoints) async fn get_playlist_for_custom_provider( #[cfg(test)] mod tests { - use super::{rewrite_resource_url, stalker_refresh_pending_response}; + use super::{preview_item, rewrite_resource_url, stalker_refresh_pending_response}; use shared::{ - model::{PlaylistItemType, ResourceLocator, UiPlaylistItem, XtreamCluster}, - utils::Internable, + model::{PlaylistItemType, UiPlaylistItem, XtreamCluster}, + utils::{open_web_ui_resource_url, seal_web_ui_resource_url, Internable}, }; - use tuliprox_core::utils::MAX_RESOURCE_TOKEN_BYTES; - - /// Decodes a rewritten link with the same route-specific decoding the resource route uses. - fn decode_link(secret: &[u8; 16], link: &str) -> ResourceLocator { - let encoded = link.rsplit('/').next().expect("encoded part"); - let token = tuliprox_core::utils::decode_resource_token(secret, encoded).expect("token decodes"); - ResourceLocator::decode(&token.resource).expect("locator decodes") - } fn sample_item(logo: &str) -> UiPlaylistItem { UiPlaylistItem { @@ -375,25 +349,41 @@ mod tests { } #[test] - fn rewrite_resource_url_wraps_external_urls_with_their_origin() { + fn rewrite_resource_url_wraps_external_urls() { let secret = [7u8; 16]; let item = sample_item("https://example.com/poster.jpg"); let rewritten = rewrite_resource_url(&secret, "/api/v1/playlist/resource", item); + let expected_suffix = seal_web_ui_resource_url(&secret, "https://example.com/poster.jpg"); - let locator = decode_link(&secret, rewritten.logo.as_ref()); - assert_eq!(locator.url.as_ref(), "https://example.com/poster.jpg"); - assert_eq!(locator.input_name.as_ref(), "test"); + assert_eq!(rewritten.logo.as_ref(), format!("/api/v1/playlist/resource/{expected_suffix}")); } #[test] - fn rewrite_resource_url_rejects_oversized_urls() { + fn preview_item_wraps_icon_and_routes_playback() { let secret = [7u8; 16]; - let long_logo = format!("https://example.com/{}", "a".repeat(MAX_RESOURCE_TOKEN_BYTES)); - let item = sample_item(&long_logo); + let item = sample_item("http://192.168.1.20/logo.png"); + + let previewed = preview_item(&secret, "/api/v1/playlist/resource", 5, item); + + assert!(previewed.logo.starts_with("/api/v1/playlist/resource/"), "{}", previewed.logo); + assert!(!previewed.logo.contains("192.168.1.20"), "{}", previewed.logo); + let playback = previewed.url.trim_start_matches("/api/v1/playlist/resource/"); + assert_ne!(playback, previewed.url.as_ref(), "playback URL must be wrapped too"); + assert_eq!( + open_web_ui_resource_url(&secret, playback).expect("decodable playback link"), + "stalker://5/live/provider" + ); + } + + #[test] + fn rewrite_resource_url_hides_private_destinations_from_the_client() { + let secret = [7u8; 16]; + let item = sample_item("http://192.168.1.20/logo.png"); let rewritten = rewrite_resource_url(&secret, "/api/v1/playlist/resource", item); - assert!(rewritten.logo.is_empty()); + assert!(rewritten.logo.starts_with("/api/v1/playlist/resource/"), "{}", rewritten.logo); + assert!(!rewritten.logo.contains("192.168.1.20"), "{}", rewritten.logo); } } diff --git a/backend/app/src/api/endpoints/custom_video_stream_api.rs b/backend/app/src/api/endpoints/custom_video_stream_api.rs index 56ba221a4..ba9348d73 100644 --- a/backend/app/src/api/endpoints/custom_video_stream_api.rs +++ b/backend/app/src/api/endpoints/custom_video_stream_api.rs @@ -507,12 +507,11 @@ mod tests { use super::{cvs_api_register, parse_cvs_standalone_hls_segment_file}; use crate::{ api::model::{ - build_hls_standalone_custom_plan, empty_resource_client_set, - hls_custom_video_manifest_response_for_access_lease, ActiveProviderManager, ActiveUserManager, AppState, - CancelTokens, ConnectionManager, CustomVideoStreamType, DownloadQueue, EventManager, HlsAccessLease, - HlsAccessLeaseId, HlsPlaybackFamilyKey, HlsProvisioningState, HlsProxyManager, HlsRuntimeCustomTailReason, - HlsStandaloneCustomAccess, MetadataUpdateManager, PlaylistStorageState, ProxySessionId, - SharedStreamManager, TransportStreamBuffer, UpdateGuard, + build_hls_standalone_custom_plan, hls_custom_video_manifest_response_for_access_lease, + ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, ConnectionManager, CustomVideoStreamType, + DownloadQueue, EventManager, HlsAccessLease, HlsAccessLeaseId, HlsPlaybackFamilyKey, HlsProvisioningState, + HlsProxyManager, HlsRuntimeCustomTailReason, HlsStandaloneCustomAccess, MetadataUpdateManager, + PlaylistStorageState, ProxySessionId, SharedStreamManager, TransportStreamBuffer, UpdateGuard, }, model::{ ApiProxyConfig, ApiProxyServerInfo, AppConfig, Config, ConfigInput, ConfigSource, ConfigTarget, @@ -726,7 +725,8 @@ mod tests { http_client: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, diff --git a/backend/app/src/api/endpoints/download_api/tests.rs b/backend/app/src/api/endpoints/download_api/tests.rs index d0106e916..f942e60d4 100644 --- a/backend/app/src/api/endpoints/download_api/tests.rs +++ b/backend/app/src/api/endpoints/download_api/tests.rs @@ -9,10 +9,9 @@ use super::{ }; use crate::{ api::model::{ - empty_resource_client_set, recording_notification::LifecycleEvent, ActiveProviderManager, ActiveUserManager, - AppState, CancelTokens, ConnectionManager, DownloadControl, DownloadKind, DownloadQueue, DownloadState, - EventManager, EventMessage, FileDownload, MetadataUpdateManager, PlaylistStorageState, SharedStreamManager, - UpdateGuard, + recording_notification::LifecycleEvent, ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, + ConnectionManager, DownloadControl, DownloadKind, DownloadQueue, DownloadState, EventManager, EventMessage, + FileDownload, MetadataUpdateManager, PlaylistStorageState, SharedStreamManager, UpdateGuard, }, model::{ ApiProxyConfig, ApiProxyServerInfo, AppConfig, Config, ConfigInput, MediaToolCapabilities, MessageContent, @@ -578,7 +577,8 @@ fn create_test_app_state_with_downloads(downloads: Arc) -> Arc axum::http::StatusCode::NOT_FOUND.into_response(), - Some(value) => { - let resolved = match resolve_resource(&app_state.app_config, &value, Some(&m3u_item.input_name)) { - Ok(resolved) => resolved, - Err(err) => { - log_resource_rejection(Some(m3u_item.input_name.as_ref()), &err, &value); - return rejection_status(&err).into_response(); - } - }; - let url = resolved.url; + Some(url) => { if user.proxy.is_redirect(m3u_item.item_type) || target.is_force_redirect(m3u_item.item_type) { - let input = resolved - .authorization - .input_name - .as_ref() - .and_then(|input_name| app_state.app_config.get_input_by_name(input_name)); + let input = app_state.app_config.get_input_by_name(&m3u_item.input_name); let redirect_url = crate::api::api_utils::resolve_redirect_location(input.as_deref(), &url); match redirect_url { Ok(redirect_url) => { - debug!("Redirecting stream request to {}", sanitize_sensitive_info(redirect_url.as_ref())); - redirect(redirect_url.as_ref()).into_response() + resource_redirect_or_proxy(&app_state, redirect_url.as_ref(), &req_headers, input.as_deref()) + .await } Err(err) => { error!("Failed to resolve redirect url: {}", sanitize_sensitive_info(&err.to_string())); @@ -964,15 +953,10 @@ async fn m3u_api_resource( } } } else { - resource_response( - &app_state, - ResourceFetchOptions::cached(resolved.authorization), - &url, - &req_headers, - None, - ) - .await - .into_response() + // The response narrows the input to the resource origin itself; passing the input of + // the item keeps that decision in one place. + let input = app_state.app_config.get_input_by_name(&m3u_item.input_name); + resource_proxy_response(&app_state, &url, &req_headers, input.as_deref()).await } } } diff --git a/backend/app/src/api/endpoints/v1_api_playlist.rs b/backend/app/src/api/endpoints/v1_api_playlist.rs index dc5bacf45..96ec449e4 100644 --- a/backend/app/src/api/endpoints/v1_api_playlist.rs +++ b/backend/app/src/api/endpoints/v1_api_playlist.rs @@ -3,9 +3,8 @@ use crate::{ api::{ api_utils::{ - create_api_proxy_user, decode_resource_link, json_or_bin_response, log_resource_rejection, - rejection_status, resolve_resource, resource_response, try_option_bad_request, try_result_bad_request, - try_unwrap_body, ResourceFetchOptions, + create_api_proxy_user, json_or_bin_response, resource_proxy_response, try_option_bad_request, + try_result_bad_request, try_unwrap_body, }, auth_middleware::{check_permission, permission_layer, VerifiedClaims}, endpoints::{ @@ -55,15 +54,15 @@ use shared::{ error::TuliproxError, foundation::{get_filter_detailed, Filter, ValueProvider}, model::{ - ingest_resource_value, permission::Permission, stalker::StalkerStreamKind, EpgChannel, - InputPlaylistUpdateStatusDto, InputType, InputUpdateAction, InputUpdateCapabilities, InputUpdateRequest, - OperationRunAccepted, PersistedPlaylistUpdateClusterState, PersistedPlaylistUpdateClusterStatusDto, + permission::Permission, stalker::StalkerStreamKind, EpgChannel, InputPlaylistUpdateStatusDto, InputType, + InputUpdateAction, InputUpdateCapabilities, InputUpdateRequest, OperationRunAccepted, + PersistedPlaylistUpdateClusterState, PersistedPlaylistUpdateClusterStatusDto, PersistedPlaylistUpdateInputResult, PlaylistEpgRequest, PlaylistItem, PlaylistRequest, PlaylistUpdateRequestDto, PlaylistUpdateRequestPayload, PlaylistUpdateRunId, PlaylistUpdateState, PlaylistUpdateStatusDto, PlaylistUrlResolveRequest, ProxyType, TargetType, UiPlaylistItem, VirtualId, XtreamCluster, }, - utils::{concat_path_leading_slash, deobfuscate_text, sanitize_sensitive_info, Internable}, + utils::{concat_path_leading_slash, open_web_ui_resource_url, sanitize_sensitive_info, Internable}, }; use std::{path::Path, str::FromStr, sync::Arc}; use tokio_stream::StreamExt; @@ -467,17 +466,7 @@ async fn load_epg_channels_for_input( }; let source_channels = match source_result { - Ok(channels) => channels - .into_iter() - .map(|mut channel| { - if let Some(icon) = &mut channel.icon { - if ingest_resource_value(icon, &input.name).is_err() { - channel.icon = None; - } - } - channel - }) - .collect(), + Ok(channels) => channels, Err(err) => { debug!( "Skipping EPG source {}: {}", @@ -1100,36 +1089,17 @@ async fn playlist_resource( axum::extract::State(app_state): axum::extract::State>, ) -> impl IntoResponse + Send { let encrypt_secret = app_state.get_encrypt_secret(); - // This route decodes only its own two formats: the authenticated token for new links, and the - // XOR-based `deobfuscate_text` encoding for links issued before origin tracking. - let decoded = decode_resource_link(&encrypt_secret, &resource, |secret, value| { - deobfuscate_text(secret, value).map_err(|_| ()) - }); - - let resource_value = match decoded { - Ok(decoded) => decoded, - Err(status) => return status.into_response(), - }; - - // Stalker locators are playback references resolved by a dedicated path, not resource URLs. - if let Some((input_id, cluster, provider_id)) = parse_stalker_resource(&resource_value) { - return stalker_resource_response(&app_state, input_id, cluster, provider_id).await; - } - - match resolve_resource(&app_state.app_config, &resource_value, None) { - Ok(resolved) => resource_response( - &app_state, - ResourceFetchOptions::cached(resolved.authorization), - &resolved.url, - &req_headers, - None, - ) - .await - .into_response(), - Err(err) => { - log_resource_rejection(None, &err, &resource_value); - rejection_status(&err).into_response() + if let Ok(resource_url) = open_web_ui_resource_url(&encrypt_secret, &resource) { + if let Some((input_id, cluster, provider_id)) = parse_stalker_resource(&resource_url) { + return stalker_resource_response(&app_state, input_id, cluster, provider_id).await; } + // This route serves every icon the Web UI shows, so it is classified like the player routes: + // a public destination goes out through the proxy-aware fetch, which honours a configured + // proxy, while one that is not provably public is fetched directly so the request cannot leave + // through a proxy that has no route to it. + resource_proxy_response(&app_state, &resource_url, &req_headers, None).await.into_response() + } else { + axum::http::StatusCode::BAD_REQUEST.into_response() } } @@ -1423,11 +1393,13 @@ async fn playlist_episode_item( ) .await { + // The icon is wrapped like on every other Web UI path: the item carries the + // destination as it was stored, which may name a host internal to this instance. let config = app_state.app_config.config.load(); let web_ui_path = config.web_ui.as_ref().and_then(|web_ui| web_ui.path.as_ref()).map_or("", String::as_str); - let resource_url = - shared::utils::concat_path_leading_slash(web_ui_path, "api/v1/playlist/resource"); + let resource_url = concat_path_leading_slash(web_ui_path, "api/v1/playlist/resource"); + drop(config); let item = rewrite_resource_url( &app_state.get_encrypt_secret(), &resource_url, @@ -1447,9 +1419,9 @@ mod tests { use super::resolve_provider_url_for_request; use crate::{ api::model::{ - empty_resource_client_set, recording::recording_source_resolution::resolve_recording_config, - ActiveProviderManager, ActiveUserManager, AppState, ConnectionManager, DownloadQueue, EventManager, - MetadataUpdateManager, PlaylistStorageState, SharedStreamManager, + recording::recording_source_resolution::resolve_recording_config, ActiveProviderManager, ActiveUserManager, + AppState, ConnectionManager, DownloadQueue, EventManager, MetadataUpdateManager, PlaylistStorageState, + SharedStreamManager, }, model::{ AppConfig, Config, ConfigInput, ConfigInputOptions, ConfigInputUpdateQuality, ConfigProvider, ConfigSource, @@ -2672,7 +2644,8 @@ mod tests { http_client: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(crate::api::model::DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, @@ -3534,11 +3507,22 @@ mod tests { .expect("malformed response"); assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let legacy = shared::utils::obfuscate_text(&app_state.get_encrypt_secret(), "http://10.0.0.1/icon.png"); + let response = router + .clone() + .into_service::() + .oneshot( + Request::get(format!("/playlist/resource/{legacy}")).body(Body::empty()).expect("legacy token request"), + ) + .await + .expect("legacy token response"); + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + for locator in [ format!("{}99/live/101", super::STALKER_RESOURCE_SCHEME), format!("{}8/live/101", super::STALKER_RESOURCE_SCHEME), ] { - let resource = shared::utils::obfuscate_text(&app_state.get_encrypt_secret(), &locator); + let resource = shared::utils::seal_web_ui_resource_url(&app_state.get_encrypt_secret(), &locator); let response = router .clone() .into_service::() @@ -3552,7 +3536,7 @@ mod tests { assert_eq!(response.status(), StatusCode::NOT_FOUND, "{locator}"); } - let resource = shared::utils::obfuscate_text( + let resource = shared::utils::seal_web_ui_resource_url( &app_state.get_encrypt_secret(), &format!("{}7/live/102", super::STALKER_RESOURCE_SCHEME), ); diff --git a/backend/app/src/api/endpoints/xmltv_api.rs b/backend/app/src/api/endpoints/xmltv_api.rs index a66458415..896164baa 100644 --- a/backend/app/src/api/endpoints/xmltv_api.rs +++ b/backend/app/src/api/endpoints/xmltv_api.rs @@ -1,10 +1,9 @@ use crate::{ api::{ api_utils::{ - coalesce_byte_stream, create_api_proxy_user, decode_resource_link, empty_json_response_as_array, - get_user_target, get_user_target_by_credentials, internal_server_error, log_resource_rejection, - rejection_status, resolve_resource, resource_response, stream_json_or_bin_response_try_stream, - try_unwrap_body, ResourceFetchOptions, + coalesce_byte_stream, create_api_proxy_user, empty_json_response_as_array, get_user_target, + get_user_target_by_credentials, internal_server_error, resource_response, + stream_json_or_bin_response_try_stream, try_unwrap_body, ResourceFetchPolicy, }, model::{AppState, UserApiRequest, UserApiRequestQueryOrBody}, static_headers::CT_XML, @@ -21,23 +20,24 @@ use crate::{ }, utils, utils::{ - canonicalize_output_epg_id, canonicalize_untrusted_epg_id, deobscure_text, encode_resource_token, - file_exists_async, format_xmltv_time_utc, get_epg_processing_options, lowercase_xmltv_text, obscure_text, + canonicalize_output_epg_id, canonicalize_untrusted_epg_id, deobscure_authenticated_bytes, file_exists_async, + format_xmltv_time_utc, get_epg_processing_options, lowercase_xmltv_text, obscure_authenticated_bytes, + request::{classify_resource_destination, ResourceDestination}, EpgIdOutputCase, EpgProcessingOptions, EpgTimeShift, }, }; use axum::response::IntoResponse; use chrono::{DateTime, TimeZone}; -use log::{debug, error, trace}; +use log::{error, trace}; use quick_xml::events::{BytesEnd, BytesStart, BytesText, Event}; use shared::{ concat_string, + error::TuliproxError, model::{ - resolve_resource_value, ConfigTargetOptions, EpgChannel, EpgProgramme, EpgProgrammeDto, ResourceToken, - ShortEpgDto, ShortEpgResultDto, StreamEpgEntry, StreamEpgItemRequest, StreamEpgRequest, StreamEpgResponse, - TargetType, + ConfigTargetOptions, EpgChannel, EpgProgramme, EpgProgrammeDto, ShortEpgDto, ShortEpgResultDto, StreamEpgEntry, + StreamEpgItemRequest, StreamEpgRequest, StreamEpgResponse, TargetType, }, - utils::{concat_path, concat_path_leading_slash, obfuscate_text, Internable}, + utils::{concat_path, concat_path_leading_slash, seal_web_ui_resource_url, Internable}, }; use std::{ borrow::Cow, @@ -52,6 +52,27 @@ use tokio::{ }; use tokio_stream::StreamExt; use tokio_util::io::ReaderStream; +use url::Url; + +const XMLTV_RESOURCE_TOKEN_DOMAIN: &[u8] = b"tuliprox.xmltv.resource.v1"; +/// Largest accepted token, checked against the encoded form before it is decoded. +const MAX_XMLTV_RESOURCE_TOKEN_LEN: usize = 16_384; + +/// Seals an icon URL for the proxy route. +/// +/// The token is the only place the destination appears in a client-visible URL: it is +/// authenticated and encrypted with the instance secret, so a client cannot read or forge it. +fn encode_xmltv_resource_url(secret: &[u8; 16], url: &str) -> Result { + obscure_authenticated_bytes(secret, XMLTV_RESOURCE_TOKEN_DOMAIN, url.as_bytes()) +} + +fn decode_xmltv_resource_url(secret: &[u8; 16], encoded: &str) -> Result { + if encoded.len() > MAX_XMLTV_RESOURCE_TOKEN_LEN { + return Err(TuliproxError::Crypto("XMLTV resource token is too long".to_string())); + } + let url = deobscure_authenticated_bytes(secret, XMLTV_RESOURCE_TOKEN_DOMAIN, encoded)?; + String::from_utf8(url).map_err(|_| TuliproxError::Crypto("Invalid XMLTV resource URL".to_string())) +} pub fn get_empty_epg_response() -> axum::response::Response { try_unwrap_body!(axum::response::Response::builder() @@ -201,42 +222,28 @@ pub fn rewrite_epg_channel_resource_url( resource_url: &str, mut channel: EpgChannel, ) -> EpgChannel { - let Some(icon) = channel.icon.as_ref() else { - return channel; - }; - if icon.is_empty() || icon.starts_with('/') { - return channel; + // Programme icons are part of the same response: a channel without an icon must not keep its + // programmes exposed either. + for programme in &mut channel.programmes { + programme.icon = rewrite_epg_resource_icon(encrypt_secret, resource_url, programme.icon.take()); } - let encoded = encode_resource_link(encrypt_secret, icon).unwrap_or_else(|| { - let external = external_resource_url(icon); - obfuscate_text(encrypt_secret, external.as_ref()) - }); - channel.icon = Some(concat_path(resource_url, &encoded).intern()); + channel.icon = rewrite_epg_resource_icon(encrypt_secret, resource_url, channel.icon.take()); channel } -/// Encodes a resource link that carries its origin. +/// Wraps an EPG icon into a proxy link, so the client never receives the destination itself. /// -/// `None` means the link could not carry the origin, for example because the URL is longer than a -/// token may be. Those links keep the legacy encoding and are therefore public-only, which fails -/// closed instead of authorizing an unchecked destination. -pub fn encode_resource_link(encrypt_secret: &[u8; 16], resource: &str) -> Option { - let token = ResourceToken { resource: resource.to_string() }; - match encode_resource_token(encrypt_secret, &token) { - Ok(encoded) => Some(encoded), - Err(err) => { - debug!("Falling back to a legacy resource link: {err}"); - None - } - } -} - -fn external_resource_url(value: &str) -> Cow<'_, str> { - match resolve_resource_value(value) { - Ok(Some(locator)) => Cow::Owned(locator.url.to_string()), - Ok(None) => Cow::Borrowed(value), - Err(_) => Cow::Borrowed(""), +/// Empty and instance-relative paths are already served by this instance and stay untouched. +fn rewrite_epg_resource_icon( + encrypt_secret: &[u8; 16], + resource_url: &str, + icon: Option>, +) -> Option> { + let icon = shared::model::persisted_resource_arc(&icon?); + if icon.is_empty() || icon.starts_with('/') { + return Some(icon); } + Some(concat_path(resource_url, &seal_web_ui_resource_url(encrypt_secret, &icon)).intern()) } macro_rules! continue_on_err { @@ -250,8 +257,7 @@ macro_rules! continue_on_err { async fn write_programme_metadata_tags( writer: &mut quick_xml::Writer, programme: &EpgProgramme, - epg_processing_options: &EpgProcessingOptions, - base_url: Option<&str>, + icons: &EpgIconOutput<'_>, ) -> Result<(), quick_xml::Error> { for category in &programme.categories { let mut elem = BytesStart::new(EPG_TAG_CATEGORY); @@ -264,10 +270,11 @@ async fn write_programme_metadata_tags( } if let Some(icon_url) = programme.icon.as_deref() { - let icon = rewrite_xmltv_icon_url(epg_processing_options, base_url, icon_url); - let mut elem = BytesStart::new(EPG_TAG_ICON); - elem.push_attribute((EPG_ATTRIB_SRC, icon.as_ref())); - writer.write_event_async(Event::Empty(elem)).await?; + if let Some(icon) = icons.url(icon_url).await { + let mut elem = BytesStart::new(EPG_TAG_ICON); + elem.push_attribute((EPG_ATTRIB_SRC, icon.as_ref())); + writer.write_event_async(Event::Empty(elem)).await?; + } } if programme.is_live { @@ -279,22 +286,62 @@ async fn write_programme_metadata_tags( Ok(()) } -fn rewrite_xmltv_icon_url<'a>( - epg_processing_options: &EpgProcessingOptions, - base_url: Option<&str>, - icon_url: &'a str, -) -> Cow<'a, str> { - if epg_processing_options.rewrite_urls { - if let Some(base) = base_url { - if let Some(encoded) = encode_resource_link(&epg_processing_options.encrypt_secret, icon_url) { - return Cow::Owned(concat_string!(base, "/", &encoded)); - } - if let Ok(enc) = obscure_text(&epg_processing_options.encrypt_secret, icon_url) { - return Cow::Owned(concat_string!(base, "/", &enc)); +/// Everything needed to turn an EPG icon into the URL that is written to the output. +/// +/// Bundled so that the writer chain does not gain a parameter per concern. +struct EpgIconOutput<'a> { + options: &'a EpgProcessingOptions, + base_url: Option<&'a str>, +} + +/// Where an EPG icon URL can be used. +enum IconDestination { + /// Provably public: the client may fetch the URL itself. + Public, + /// Not provably public (private network, reserved range, unresolved name): the URL must never + /// reach a client, because it can name an internal destination. Only the proxy fetches it. + Proxied, + /// Not an http(s) URL, or local to the proxy host (loopback, link-local, metadata endpoints): + /// there is nothing the client could use and nothing the proxy would fetch. + Unusable, +} + +async fn classify_icon_destination(icon_url: &str) -> IconDestination { + let Ok(url) = Url::parse(icon_url) else { + return IconDestination::Unusable; + }; + if !matches!(url.scheme(), "http" | "https") { + return IconDestination::Unusable; + } + match url.host_str() { + Some(host) => match classify_resource_destination(host).await { + ResourceDestination::Public => IconDestination::Public, + ResourceDestination::Private => IconDestination::Proxied, + ResourceDestination::Blocked => IconDestination::Unusable, + }, + None => IconDestination::Unusable, + } +} + +impl EpgIconOutput<'_> { + /// Builds the URL written into the EPG for an icon, or `None` when the icon must be dropped. + /// + /// Only a provably public destination may be handed to the client as-is, and only while the client + /// fetches resources itself (no reverse-proxy rewriting). Every other destination is replaced by an + /// authenticated proxy link, which keeps icons from internal hosts working without exposing their + /// address. Without a client-visible base URL the icon is dropped: writing the original URL would + /// leak the internal destination, which is exactly what must not happen. + async fn url<'i>(&self, icon_url: &'i str) -> Option> { + let icon_url = shared::model::persisted_resource_url(icon_url)?; + match classify_icon_destination(icon_url.as_ref()).await { + IconDestination::Unusable => None, + IconDestination::Public if !self.options.rewrite_urls => Some(icon_url), + IconDestination::Public | IconDestination::Proxied => { + let encoded = encode_xmltv_resource_url(&self.options.encrypt_secret, icon_url.as_ref()).ok()?; + self.base_url.map(|base| Cow::Owned(concat_string!(base, "/", &encoded))) } } } - external_resource_url(icon_url) } #[allow(clippy::too_many_lines)] @@ -314,22 +361,17 @@ async fn serve_epg_with_rewrites( target.options.as_ref().is_some_and(ConfigTargetOptions::lowercase_xmltv_display_names); let server_info = app_state.app_config.get_user_server_info(user); - let base_url = - if !matches!(epg_processing_options.time_shift, EpgTimeShift::None) || epg_processing_options.rewrite_urls { - server_info.as_ref().map(|si| { - concat_string!( - &si.get_base_url(), - "/", - storage_const::EPG_RESOURCE_PATH, - "/", - &user.username, - "/", - &user.password - ) - }) - } else { - None - }; + let base_url = server_info.as_ref().map(|si| { + concat_string!( + &si.get_base_url(), + "/", + storage_const::EPG_RESOURCE_PATH, + "/", + &user.username, + "/", + &user.password + ) + }); let generator_info = server_info.as_ref().map(ApiProxyServerInfo::get_base_url).unwrap_or_default(); @@ -385,6 +427,7 @@ async fn serve_epg_with_rewrites( let (mut tx, rx) = tokio::io::duplex(8192); tokio::spawn(async move { + let icons = EpgIconOutput { options: &epg_processing_options, base_url: base_url.as_deref() }; // Work-Around BytesText DocType escape, see below if let Err(err) = tx.write_all(XML_PREAMBLE.as_ref()).await { error!("EPG: Failed to write xml header {err}"); @@ -433,12 +476,12 @@ async fn serve_epg_with_rewrites( continue_on_err!(writer.write_event_async(Event::End(elem)).await); if let Some(icon_url) = &channel.icon { - let icon = rewrite_xmltv_icon_url(&epg_processing_options, base_url.as_deref(), icon_url.as_ref()); - - let mut elem = BytesStart::new("icon"); - elem.push_attribute(("src", icon.as_ref())); - if (writer.write_event_async(Event::Empty(elem)).await).is_err() { - // ignore + if let Some(icon) = icons.url(icon_url.as_ref()).await { + let mut elem = BytesStart::new("icon"); + elem.push_attribute(("src", icon.as_ref())); + if (writer.write_event_async(Event::Empty(elem)).await).is_err() { + // ignore + } } } @@ -476,14 +519,7 @@ async fn serve_epg_with_rewrites( continue_on_err!(writer.write_event_async(Event::End(BytesEnd::new("desc"))).await); } - if let Err(err) = write_programme_metadata_tags( - &mut writer, - programme, - &epg_processing_options, - base_url.as_deref(), - ) - .await - { + if let Err(err) = write_programme_metadata_tags(&mut writer, programme, &icons).await { error!("EPG programme metadata write failed: {err}"); } @@ -948,33 +984,15 @@ async fn epg_api_resource( return e.into_player_response(auth_status); } - let encrypt_secret = app_state.get_encrypt_secret(); - // This route decodes only its own two formats: the authenticated token for new links, and the - // AES-based `obscure_text` encoding for links issued before origin tracking. - let decoded = - decode_resource_link(&encrypt_secret, &resource, |secret, value| deobscure_text(secret, value).map_err(|_| ())); - - let resource_value = match decoded { - Ok(decoded) => decoded, - Err(status) => return status.into_response(), + let Ok(resource_url) = decode_xmltv_resource_url(&app_state.get_encrypt_secret(), &resource) else { + return axum::http::StatusCode::BAD_REQUEST.into_response(); }; - - // Legacy links and EPG icons without a recorded origin stay public-only. - match resolve_resource(&app_state.app_config, &resource_value, None) { - Ok(resolved) => resource_response( - &app_state, - ResourceFetchOptions::epg(resolved.authorization), - &resolved.url, - &req_headers, - None, - ) - .await - .into_response(), - Err(err) => { - log_resource_rejection(None, &err, &resource_value); - rejection_status(&err).into_response() - } + if !Url::parse(&resource_url).is_ok_and(|url| matches!(url.scheme(), "http" | "https")) { + return axum::http::StatusCode::BAD_REQUEST.into_response(); } + resource_response(&app_state, ResourceFetchPolicy::NonPublic, &resource_url, &req_headers, None) + .await + .into_response() } /// Registers the XMLTV EPG API routes for handling HTTP GET requests. @@ -1001,10 +1019,11 @@ pub fn xmltv_api_register() -> axum::Router> { #[cfg(test)] mod tests { use super::{ - empty_stream_epg_entries, from_programme, get_epg_path_for_target, get_epg_path_for_target_by_type, - group_stream_epg_items, prepare_stream_epg_request, rewrite_epg_channel_resource_url, serve_epg, - serve_epg_web_ui, serve_short_epg, serve_stream_epg, stream_epg_api, stream_epg_programmes_for_channel, - write_programme_metadata_tags, MAX_STREAM_EPG_CHANNEL_ID_BYTES, MAX_STREAM_EPG_ITEMS, + decode_xmltv_resource_url, empty_stream_epg_entries, encode_xmltv_resource_url, from_programme, + get_epg_path_for_target, get_epg_path_for_target_by_type, group_stream_epg_items, prepare_stream_epg_request, + rewrite_epg_channel_resource_url, serve_epg, serve_epg_web_ui, serve_short_epg, serve_stream_epg, + stream_epg_api, stream_epg_programmes_for_channel, write_programme_metadata_tags, EpgIconOutput, + MAX_STREAM_EPG_CHANNEL_ID_BYTES, MAX_STREAM_EPG_ITEMS, }; use crate::{ api::model::{create_test_app_state, AppState}, @@ -1014,7 +1033,10 @@ mod tests { }, processing::parser::ics::parse_ics_file_to_channel, repository::{epg_write_file, BPlusTree}, - utils::{lowercase_xmltv_text, EpgIdOutputCase, EpgProcessingOptions, EpgTimeShift}, + utils::{ + lowercase_xmltv_text, obscure_authenticated_bytes, obscure_text, EpgIdOutputCase, EpgProcessingOptions, + EpgTimeShift, + }, }; use arc_swap::ArcSwapOption; use axum::response::IntoResponse; @@ -1023,9 +1045,9 @@ mod tests { foundation::Filter, model::{ ConfigTargetOptions, EpgCategory, EpgChannel, EpgOutputOptions, EpgProgramme, ProcessingOrder, - ResourceLocator, StreamEpgItemRequest, StreamEpgRequest, TargetType, + StreamEpgItemRequest, StreamEpgRequest, TargetType, }, - utils::Internable, + utils::{concat_path, seal_web_ui_resource_url, Internable}, }; use std::{ collections::HashMap, @@ -1036,7 +1058,6 @@ mod tests { }; use tempfile::tempdir; use tokio::io::AsyncWrite; - use tuliprox_core::utils::{decode_resource_token, has_resource_token_prefix}; struct ErroringWriter; @@ -1513,7 +1534,7 @@ mod tests { ]; programme.is_live = true; programme.is_new = true; - programme.icon = Some("https://example.com/programme.jpg".intern()); + programme.icon = Some("https://8.8.8.8/programme.jpg".intern()); write_test_epg_db( &epg_path, EpgChannel { @@ -1539,7 +1560,7 @@ mod tests { assert!(!xml.contains("news & updates")); assert!(xml.contains(r#"News & Analysis"#)); assert!(xml.contains("Sports")); - assert!(xml.contains(r#""#)); + assert!(xml.contains(r#""#)); assert!(xml.contains("")); assert!(xml.contains("")); @@ -1556,6 +1577,12 @@ mod tests { assert!(live_pos < new_pos); } + /// Icon output for tests. Destination verdicts come from the process-wide memo, so the icons used + /// here are IP literals that are decided without resolving a name. + fn icon_output<'a>(options: &'a EpgProcessingOptions, base_url: Option<&'a str>) -> EpgIconOutput<'a> { + EpgIconOutput { options, base_url } + } + #[tokio::test] async fn programme_metadata_writer_propagates_io_errors() { let mut programme = EpgProgramme::new(100, 200, "channel".intern()); @@ -1564,7 +1591,7 @@ mod tests { let options = EpgProcessingOptions { rewrite_urls: false, time_shift: EpgTimeShift::None, encrypt_secret: [0; 16] }; - assert!(write_programme_metadata_tags(&mut writer, &programme, &options, None).await.is_err()); + assert!(write_programme_metadata_tags(&mut writer, &programme, &icon_output(&options, None)).await.is_err()); } #[tokio::test] @@ -1575,21 +1602,157 @@ mod tests { let base_url = "http://localhost/epg/user/password"; let original_url = "https://example.com/programme.jpg"; let mut programme = EpgProgramme::new(100, 200, "channel".intern()); - programme.icon = Some( - ResourceLocator::new("epg-input".into(), original_url.into()).expect("locator").encode().expect("encode"), - ); + programme.icon = Some(original_url.intern()); let mut writer = quick_xml::Writer::new(Vec::new()); - write_programme_metadata_tags(&mut writer, &programme, &options, Some(base_url)).await?; + write_programme_metadata_tags(&mut writer, &programme, &icon_output(&options, Some(base_url))).await?; let xml = String::from_utf8(writer.into_inner())?; let resource = xml .strip_prefix(&format!(r#""#)) .unwrap_or_default(); - let token = decode_resource_token(&secret, resource).expect("token decodes"); - let locator = ResourceLocator::decode(&token.resource).expect("locator decodes"); - assert_eq!(locator.url.as_ref(), original_url); - assert_eq!(locator.input_name.as_ref(), "epg-input"); + assert_eq!(decode_xmltv_resource_url(&secret, resource)?, original_url); + Ok(()) + } + + #[tokio::test] + async fn xmltv_icon_output_keeps_public_url_and_proxies_private_url() -> Result<(), Box> { + let secret = [9; 16]; + let options = + EpgProcessingOptions { rewrite_urls: false, time_shift: EpgTimeShift::None, encrypt_secret: secret }; + let base = "https://user.example/resource/epg/user/password"; + + let public = icon_output(&options, Some(base)).url("https://8.8.8.8/icon.png").await; + assert_eq!(public.as_deref(), Some("https://8.8.8.8/icon.png")); + + for private_url in ["http://10.0.0.1/icon.png", "http://192.168.1.20/icon.png", "http://[fd00::1]/icon.png"] { + let rewritten = icon_output(&options, Some(base)).url(private_url).await; + let rewritten = rewritten.expect("private icon is proxied"); + let encoded = rewritten.strip_prefix(&format!("{base}/")).expect("user server URL"); + assert_eq!(decode_xmltv_resource_url(&secret, encoded)?, private_url); + assert!(!rewritten.contains("10.0.0.1") && !rewritten.contains("192.168.1.20"), "{rewritten}"); + } + Ok(()) + } + + #[tokio::test] + async fn xmltv_icon_output_drops_destinations_that_are_never_usable() { + let options = + EpgProcessingOptions { rewrite_urls: false, time_shift: EpgTimeShift::None, encrypt_secret: [9; 16] }; + let base = "https://user.example/resource/epg/user/password"; + + for unusable in [ + "http://127.0.0.1/icon.png", + "http://169.254.169.254/latest/meta-data/", + "http://[::1]/icon.png", + "http://0.0.0.0/icon.png", + "ftp://example.com/icon.png", + "/local/icon.png", + ] { + assert!( + icon_output(&options, Some(base)).url(unusable).await.is_none(), + "icon must be dropped: {unusable}" + ); + } + } + + #[tokio::test] + async fn xmltv_icon_output_never_writes_a_private_url_without_client_base_url() { + let private_url = "http://192.168.1.20/icon.png"; + let rewrite_mode = + EpgProcessingOptions { rewrite_urls: true, time_shift: EpgTimeShift::None, encrypt_secret: [9; 16] }; + let redirect_mode = + EpgProcessingOptions { rewrite_urls: false, time_shift: EpgTimeShift::None, encrypt_secret: [9; 16] }; + + assert!(icon_output(&rewrite_mode, None).url(private_url).await.is_none()); + assert!(icon_output(&redirect_mode, None).url(private_url).await.is_none()); + } + + #[tokio::test] + async fn xmltv_icon_output_proxies_public_url_in_reverse_mode() { + let options = + EpgProcessingOptions { rewrite_urls: true, time_shift: EpgTimeShift::None, encrypt_secret: [9; 16] }; + let rewritten = icon_output(&options, Some("https://user.example/resource/epg/user/password")) + .url("https://8.8.8.8/icon.png") + .await; + + let rewritten = rewritten.expect("public icon is proxied in reverse mode"); + assert!(rewritten.starts_with("https://user.example/resource/epg/user/password/")); + assert!(!rewritten.contains("8.8.8.8"), "{rewritten}"); + } + + #[test] + fn epg_channel_resource_rewrite_wraps_channel_and_programme_icons() { + let secret = [7u8; 16]; + let base = "/api/v1/playlist/resource"; + let icon_url = "http://192.168.1.20/programme.png"; + let mut programme_with_icon = EpgProgramme::new(100, 200, "example.channel".intern()); + programme_with_icon.icon = Some(icon_url.intern()); + + let channel = EpgChannel { + id: "example.channel".intern(), + title: None, + icon: None, + programmes: vec![programme_with_icon, EpgProgramme::new(200, 300, "example.channel".intern())], + }; + + let rewritten = rewrite_epg_channel_resource_url(&secret, base, channel); + + let programme_icon = rewritten.programmes[0].icon.as_deref().expect("programme icon is wrapped"); + assert!(programme_icon.starts_with(base), "{programme_icon}"); + assert!(!programme_icon.contains("192.168.1.20"), "{programme_icon}"); + assert!(programme_icon.ends_with(&seal_web_ui_resource_url(&secret, icon_url)), "{programme_icon}"); + assert!(rewritten.programmes[1].icon.is_none()); + assert!(rewritten.icon.is_none(), "a channel without an icon stays without one"); + } + + #[tokio::test] + async fn xmltv_icon_output_never_contains_a_private_icon_destination() { + let dir = tempdir().expect("temp dir"); + let epg_path = dir.path().join("epg.db"); + let mut programme = EpgProgramme::new(100, 200, "example.channel".intern()); + programme.icon = Some("http://10.0.0.1/programme.png".intern()); + write_test_epg_db( + &epg_path, + EpgChannel { + id: "example.channel".intern(), + title: Some("Example Network".intern()), + icon: Some("http://10.0.0.1/channel.png".intern()), + programmes: vec![programme], + }, + ); + let app_state = test_app_state(); + let target = Arc::new(test_target_with_epg_options(true, true)); + + let response = serve_epg(&app_state, &epg_path, &ProxyUserCredentials::default(), &target, None).await; + let xml = response_body_text(response).await; + + assert!(!xml.contains("10.0.0.1"), "internal destination leaked into XMLTV output: {xml}"); + assert!(!xml.contains(" Result<(), Box> { + let secret = [9; 16]; + let icon_url = "http://10.0.0.1/icon.png"; + + let legacy = obscure_text(&secret, icon_url)?; + assert!(decode_xmltv_resource_url(&secret, &legacy).is_err(), "legacy obfuscation must be rejected"); + + let mut authenticated = encode_xmltv_resource_url(&secret, icon_url)?; + authenticated.replace_range(4..5, if &authenticated[4..5] == "A" { "B" } else { "A" }); + assert!(decode_xmltv_resource_url(&secret, &authenticated).is_err(), "tampered token must be rejected"); + + let foreign_domain = obscure_authenticated_bytes(&secret, b"tuliprox.other.domain.v1", icon_url.as_bytes())?; + assert!( + decode_xmltv_resource_url(&secret, &foreign_domain).is_err(), + "a token of another domain must not decode" + ); + + let other_secret = encode_xmltv_resource_url(&[8; 16], icon_url)?; + assert!(decode_xmltv_resource_url(&secret, &other_secret).is_err(), "a foreign secret must be rejected"); + + assert_eq!(decode_xmltv_resource_url(&secret, &encode_xmltv_resource_url(&secret, icon_url)?)?, icon_url); Ok(()) } @@ -1907,26 +2070,20 @@ mod tests { } #[test] - fn rewrite_epg_channel_resource_url_wraps_external_icon_with_its_origin() { + fn rewrite_epg_channel_resource_url_wraps_external_icon() { let secret = [9u8; 16]; let resource_url = "/api/v1/playlist/resource"; - let mut channel = sample_channel(None); - channel.icon = Some( - ResourceLocator::new("epg-input".into(), "https://cdn.example.com/logo.png".into()) - .expect("locator") - .encode() - .expect("encode"), - ); + let channel = sample_channel(Some("https://cdn.example.com/logo.png")); let rewritten = rewrite_epg_channel_resource_url(&secret, resource_url, channel); - let link = rewritten.icon.as_deref().expect("rewritten icon"); - let encoded = link.rsplit('/').next().expect("encoded part"); - assert!(has_resource_token_prefix(encoded)); - let token = decode_resource_token(&secret, encoded).expect("token decodes"); - let locator = ResourceLocator::decode(&token.resource).expect("locator decodes"); - assert_eq!(locator.url.as_ref(), "https://cdn.example.com/logo.png"); - assert_eq!(locator.input_name.as_ref(), "epg-input"); + assert_eq!( + rewritten.icon.as_deref(), + Some( + concat_path(resource_url, &seal_web_ui_resource_url(&secret, "https://cdn.example.com/logo.png")) + .as_str() + ) + ); } #[test] diff --git a/backend/app/src/api/endpoints/xtream_api/mod.rs b/backend/app/src/api/endpoints/xtream_api/mod.rs index 2faa7de74..0cae1b7f3 100644 --- a/backend/app/src/api/endpoints/xtream_api/mod.rs +++ b/backend/app/src/api/endpoints/xtream_api/mod.rs @@ -10,10 +10,10 @@ use crate::{ empty_json_response_as_array, empty_json_response_as_object, force_provider_stream_response, get_session_reservation_ttl_secs, get_user_target, get_user_target_by_credentials, internal_server_error, is_seekable_media_request, is_session_based_playback, is_stream_share_enabled, local_stream_response, - log_resource_rejection, redirect, redirect_response, reentry_suppressed_response, rejection_status, - resolve_initial_stalker_playback_url, resolve_resource, resource_response, separate_number_and_remainder, + redirect, redirect_response, reentry_suppressed_response, resolve_initial_stalker_playback_url, + resource_proxy_response, resource_redirect_or_proxy, separate_number_and_remainder, should_allow_exhausted_shared_reconnect, stream_response, try_option_bad_request, try_result_bad_request, - try_unwrap_body, RedirectParams, ResourceFetchOptions, + try_unwrap_body, RedirectParams, }, endpoints::{ hls_api::{ @@ -41,7 +41,10 @@ use crate::{ get_target_id_mapping, get_target_storage_path, storage_const, user_get_bouquet_filter, xtream_get_collection_path, xtream_get_item_for_stream_id, xtream_load_rewrite_playlist, VirtualIdRecord, }, - utils::{apply_timeshift, debug_if_enabled, file_exists_async, parse_timeshift, request, trace_if_enabled}, + utils::{ + apply_timeshift, debug_if_enabled, file_exists_async, parse_timeshift, prepare_xtream_resource_hosts, request, + trace_if_enabled, + }, }; use axum::{http::HeaderMap, response::IntoResponse}; use bytes::Bytes; @@ -1077,33 +1080,19 @@ async fn xtream_player_api_resource( if !shared::model::is_resource_field_name(resource) { return axum::http::StatusCode::NOT_FOUND.into_response(); } + let stream_url = pli.resolve_resource_url(resource); match stream_url { None => axum::http::StatusCode::NOT_FOUND.into_response(), - Some(value) => { - let resolved = match resolve_resource(&app_state.app_config, &value, Some(&pli.input_name)) { - Ok(resolved) => resolved, - Err(err) => { - log_resource_rejection(Some(pli.input_name.as_ref()), &err, &value); - return rejection_status(&err).into_response(); - } - }; - let url = resolved.url; + Some(url) => { if user.proxy.is_redirect(pli.item_type) || target.is_force_redirect(pli.item_type) { - let input = resolved - .authorization - .input_name - .as_ref() - .and_then(|input_name| app_state.app_config.get_input_by_name(input_name)); + let input = app_state.app_config.get_input_by_name(&pli.input_name); let redirect_url = api_utils::resolve_redirect_location(input.as_deref(), &url); match redirect_url { Ok(redirect_url) => { - trace_if_enabled!( - "Redirecting resource request to {}", - sanitize_sensitive_info(redirect_url.as_ref()) - ); - redirect(redirect_url.as_ref()).into_response() + resource_redirect_or_proxy(app_state, redirect_url.as_ref(), req_headers, input.as_deref()) + .await } Err(err) => { error!("Failed to resolve redirect url: {}", sanitize_sensitive_info(&err.to_string())); @@ -1112,15 +1101,10 @@ async fn xtream_player_api_resource( } } else { trace_if_enabled!("Resource request to {}", sanitize_sensitive_info(&url)); - resource_response( - app_state, - ResourceFetchOptions::cached(resolved.authorization), - &url, - req_headers, - None, - ) - .await - .into_response() + // The response narrows the input to the resource origin itself; passing the input of + // the item keeps that decision in one place. + let input = app_state.app_config.get_input_by_name(&pli.input_name); + resource_proxy_response(app_state, &url, req_headers, input.as_deref()).await } } } @@ -1354,6 +1338,7 @@ pub async fn xtream_get_stream_info_response( return axum::http::StatusCode::SERVICE_UNAVAILABLE.into_response(); } }; + prepare_xtream_resource_hosts(&pli, &options).await; return axum::Json(pli.to_info_document(&options)).into_response(); } diff --git a/backend/app/src/api/main_api.rs b/backend/app/src/api/main_api.rs index df175338f..b5f849d8b 100644 --- a/backend/app/src/api/main_api.rs +++ b/backend/app/src/api/main_api.rs @@ -19,7 +19,8 @@ use crate::{ http_layers::create_cors_layer, model::{ create_cache, create_http_client, create_http_client_no_redirect, create_public_http_client_no_redirect, - create_resource_client_set, exec_provider_dns, exec_qos_aggregation, load_playlists_into_memory_cache, + create_resource_http_client_no_redirect, create_resource_public_http_client_no_redirect, exec_provider_dns, + exec_qos_aggregation, load_playlists_into_memory_cache, recording_rule_scheduler::spawn_recording_rule_scheduler, recording_supervisor::start_recording_supervisors, ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, ConnectionManager, DownloadQueue, EventManager, EventMessage, HdHomerunAppState, @@ -430,7 +431,8 @@ async fn create_shared_data( let client = create_http_client(app_config)?; let client_no_redirect = create_http_client_no_redirect(app_config)?; let public_client_no_redirect = create_public_http_client_no_redirect(app_config)?; - let resource_clients = create_resource_client_set(app_config)?; + let resource_client_no_redirect = create_resource_http_client_no_redirect(app_config)?; + let resource_public_client_no_redirect = create_resource_public_http_client_no_redirect(app_config)?; let tokens = CancelTokens::default(); let metadata_manager = Arc::new(MetadataUpdateManager::new(tokens.metadata.clone())); @@ -451,7 +453,8 @@ async fn create_shared_data( http_client: Arc::new(ArcSwap::from_pointee(client)), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(client_no_redirect)), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(public_client_no_redirect)), - resource_clients: Arc::new(ArcSwap::from_pointee(resource_clients)), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(resource_client_no_redirect)), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(resource_public_client_no_redirect)), downloads: Arc::new(DownloadQueue::new_with_state_file(Some(downloads_state_file))), cache: Arc::new(ArcSwapOption::from(cache)), shared_stream_manager, @@ -1123,10 +1126,9 @@ mod tests { use super::super::ready; use crate::{ api::model::{ - empty_resource_client_set, ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, - ConnectionKind, ConnectionManager, DownloadQueue, EventManager, HlsProvisioningState, HlsProxyManager, - ManualPlaylistUpdateRequest, MetadataUpdateManager, PlaylistStorageState, ProviderHandle, - SharedStreamManager, UpdateGuard, + ActiveProviderManager, ActiveUserManager, AppState, CancelTokens, ConnectionKind, ConnectionManager, + DownloadQueue, EventManager, HlsProvisioningState, HlsProxyManager, ManualPlaylistUpdateRequest, + MetadataUpdateManager, PlaylistStorageState, ProviderHandle, SharedStreamManager, UpdateGuard, }, model::{AppConfig, Config, ConfigInput, MediaToolCapabilities, ProcessTargets, SourcesConfig}, repository::GeoIp, @@ -1225,7 +1227,8 @@ mod tests { http_client: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, diff --git a/backend/app/src/api/model/app_state.rs b/backend/app/src/api/model/app_state.rs index b327fae60..9169037d3 100644 --- a/backend/app/src/api/model/app_state.rs +++ b/backend/app/src/api/model/app_state.rs @@ -16,7 +16,7 @@ use crate::{ repository::{get_geoip_path, GeoIp}, utils::{ reload_logger, - request::{create_client, create_client_with_redirect, create_resource_http_client, PublicIpResolver}, + request::{create_client, create_client_with_redirect, PublicIpResolver, ResourceDestinationResolver}, LRUResourceCache, }, }; @@ -36,7 +36,6 @@ use std::{ }; use tokio::sync::{mpsc, RwLock}; use tokio_util::sync::CancellationToken; -use tuliprox_core::model::{public_only_policy, PolicyDigest, ResourceClientKey, ResourcePolicy, ResourceRedirectMode}; use tuliprox_hls::api::HlsProxyManager; use tuliprox_metadata::manager::MetadataUpdateManager; use tuliprox_repository::{identity_registry::IdentityRegistry, token_revocations::TokenRevocations}; @@ -284,7 +283,46 @@ pub fn create_http_client(app_config: &AppConfig) -> Result Result { - let builder = create_client_with_redirect(app_config, reqwest::redirect::Policy::none()).http1_only(); + create_no_redirect_client(app_config, None) +} + +/// Creates the no-redirect client used for a resource hop that can be reached only from the local +/// network. +/// +/// It connects directly, because a configured proxy has no route to such a destination, and its DNS +/// layer refuses addresses local to this host, so a destination cannot resolve to a local address +/// while the connection is built. Private network destinations stay allowed, because self-hosted +/// media servers are the reason this route exists. +pub fn create_resource_http_client_no_redirect(app_config: &AppConfig) -> Result { + let config = app_config.config.load(); + let mut builder = create_client_with_redirect(app_config, reqwest::redirect::Policy::none()) + .no_proxy() + .dns_resolver(ResourceDestinationResolver::default()) + .http1_only(); + if config.connect_timeout_secs > 0 { + builder = builder.connect_timeout(Duration::from_secs(u64::from(config.connect_timeout_secs))); + } + builder.build().map_err(|err| TuliproxError::Config(format!("Failed to create resource HTTP client: {err}"))) +} + +/// Creates the no-redirect client used for a resource hop that can leave the local network. +/// +/// It honours the configured proxy, because a resource fetch must not disclose this host's address to +/// a destination that a proxy was configured to hide it from. Its DNS layer additionally refuses +/// addresses local to this host, so a name that was classified as public cannot resolve to a local +/// address while the connection is built. +pub fn create_resource_public_http_client_no_redirect(app_config: &AppConfig) -> Result { + create_no_redirect_client(app_config, Some(Arc::new(ResourceDestinationResolver::allowing_proxy_hosts(app_config)))) +} + +fn create_no_redirect_client( + app_config: &AppConfig, + resolver: Option>, +) -> Result { + let mut builder = create_client_with_redirect(app_config, reqwest::redirect::Policy::none()).http1_only(); + if let Some(resolver) = resolver.clone() { + builder = builder.dns_resolver(resolver); + } let config = app_config.config.load(); build_http_client_with_fallback( builder, @@ -292,14 +330,15 @@ pub fn create_http_client_no_redirect(app_config: &AppConfig) -> Result Result, - policies: HashMap>, -} - -impl ResourceClientSet { - /// Client for a policy digest and redirect mode, or `None` when the policy is not configured - /// any more. A miss is answered with a rejection, never with a broader fallback. - pub fn client(&self, digest: &PolicyDigest, mode: ResourceRedirectMode) -> Option<&Client> { - self.clients.get(&ResourceClientKey::new(digest.clone(), mode)) - } - - pub fn policy(&self, digest: &PolicyDigest) -> Option<&Arc> { self.policies.get(digest) } - - pub fn is_empty(&self) -> bool { self.clients.is_empty() } - - /// Set with pre-built clients, for tests that inject a mock transport. - #[cfg(test)] - pub fn from_clients(clients: HashMap) -> Self { - Self { clients, policies: HashMap::new() } - } -} - -/// Empty client set, used by states that never exercise the resource proxy. -pub fn empty_resource_client_set() -> Arc> { - Arc::new(ArcSwap::from_pointee(ResourceClientSet::default())) -} - -/// Builds the resource client set for the currently configured inputs. -/// -/// Only primary inputs carry a policy; aliases inherit it, and every other input shares the -/// public-only client. -pub fn create_resource_client_set(app_config: &AppConfig) -> Result { - let config = app_config.config.load(); - tuliprox_core::utils::request::warn_resource_proxy_bypass(&config); - drop(config); - - let sources = app_config.sources.load(); - let mut policies: HashMap> = HashMap::new(); - let public_only = public_only_policy(); - policies.insert(public_only.digest(), Arc::clone(&public_only)); - for input in &sources.inputs { - if let Some(policy) = input.resource_policy.as_ref().filter(|policy| !policy.is_empty()) { - policies.entry(policy.digest()).or_insert_with(|| Arc::clone(policy)); - } - } - drop(sources); - - let mut clients = HashMap::with_capacity(policies.len() * 2); - for policy in policies.values() { - for redirect_mode in [ResourceRedirectMode::NoRedirect, ResourceRedirectMode::Bounded] { - let client = create_resource_http_client(app_config, Arc::clone(policy), redirect_mode)?; - clients.insert(ResourceClientKey::new(policy.digest(), redirect_mode), client); - } - } - Ok(ResourceClientSet { clients, policies }) -} - fn build_http_client_with_fallback( mut builder: reqwest::ClientBuilder, config: &Arc, @@ -496,8 +470,12 @@ pub struct AppState { pub http_client: Arc>, pub http_client_no_redirect: Arc>, pub public_http_client_no_redirect: Arc>, - /// Policy- and redirect-mode-keyed clients used by the resource proxy. - pub resource_clients: Arc>, + /// No-redirect client for a resource hop reachable only from the local network: connects directly + /// and refuses addresses local to this host. + pub resource_http_client_no_redirect: Arc>, + /// No-redirect client for a resource hop that can leave the local network: honours the configured + /// proxy and refuses addresses local to this host. + pub resource_public_http_client_no_redirect: Arc>, pub downloads: Arc, pub cache: Arc>>, pub shared_stream_manager: Arc, @@ -599,7 +577,8 @@ pub(crate) fn create_test_app_state(config: Config) -> Arc { http_client: Arc::new(ArcSwap::from_pointee(Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), - resource_clients: Arc::new(ArcSwap::from_pointee(ResourceClientSet::default())), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), downloads: Arc::new(DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, @@ -677,7 +656,10 @@ impl AppState { self.http_client_no_redirect.store(Arc::new(client_no_redirect)); let public_client_no_redirect = create_public_http_client_no_redirect(&self.app_config)?; self.public_http_client_no_redirect.store(Arc::new(public_client_no_redirect)); - self.resource_clients.store(Arc::new(create_resource_client_set(&self.app_config)?)); + let resource_client_no_redirect = create_resource_http_client_no_redirect(&self.app_config)?; + self.resource_http_client_no_redirect.store(Arc::new(resource_client_no_redirect)); + let public_resource_client_no_redirect = create_resource_public_http_client_no_redirect(&self.app_config)?; + self.resource_public_http_client_no_redirect.store(Arc::new(public_resource_client_no_redirect)); // cache let config = self.app_config.config.load(); @@ -720,7 +702,6 @@ impl AppState { } self.app_config.set_sources(sources)?; self.active_provider.update_config(&self.app_config); - self.resource_clients.store(Arc::new(create_resource_client_set(&self.app_config)?)); shared::model::REGEX_CACHE.sweep(); Ok(changes) diff --git a/backend/app/src/api/model/streams/active_client_stream.rs b/backend/app/src/api/model/streams/active_client_stream.rs index 766aa747c..b14a5746e 100644 --- a/backend/app/src/api/model/streams/active_client_stream.rs +++ b/backend/app/src/api/model/streams/active_client_stream.rs @@ -1587,11 +1587,11 @@ mod tests { }; use crate::{ api::model::{ - connection_manager::PROVIDER_END_NOT_SET, empty_resource_client_set, ActiveProviderManager, - ActiveUserManager, AppState, BoxedProviderStream, CancelTokens, ConnectionManager, CreateUserSessionParams, - CustomVideoStreamType, DownloadQueue, EventManager, GraceResolutionContext, MetadataUpdateManager, - PlaylistStorageState, ProviderContentRepresentationMode, ProviderHandle, SharedStreamManager, - StreamDetails, StreamError, UpdateGuard, + connection_manager::PROVIDER_END_NOT_SET, ActiveProviderManager, ActiveUserManager, AppState, + BoxedProviderStream, CancelTokens, ConnectionManager, CreateUserSessionParams, CustomVideoStreamType, + DownloadQueue, EventManager, GraceResolutionContext, MetadataUpdateManager, PlaylistStorageState, + ProviderContentRepresentationMode, ProviderHandle, SharedStreamManager, StreamDetails, StreamError, + UpdateGuard, }, auth::Fingerprint, model::{ @@ -1719,7 +1719,8 @@ mod tests { http_client: Arc::new(ArcSwap::from_pointee(Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, @@ -1799,7 +1800,8 @@ mod tests { http_client: Arc::new(ArcSwap::from_pointee(Client::new())), http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(Client::new())), - resource_clients: empty_resource_client_set(), + resource_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), + resource_public_http_client_no_redirect: Arc::new(ArcSwap::from_pointee(reqwest::Client::new())), downloads: Arc::new(DownloadQueue::new()), cache: Arc::new(ArcSwapOption::default()), shared_stream_manager, diff --git a/backend/core/Cargo.toml b/backend/core/Cargo.toml index ff9ca3d46..bb91f2558 100644 --- a/backend/core/Cargo.toml +++ b/backend/core/Cargo.toml @@ -31,6 +31,7 @@ futures.workspace = true indexmap.workspace = true ipnet = "2" log.workspace = true +lru = "0.16.4" parking_lot = "0.12.5" paste.workspace = true path-clean.workspace = true diff --git a/backend/core/src/model/config/input.rs b/backend/core/src/model/config/input.rs index 03626c9f1..19d2efbd7 100644 --- a/backend/core/src/model/config/input.rs +++ b/backend/core/src/model/config/input.rs @@ -1,5 +1,5 @@ use crate::{ - model::{macros, ConfigInputUpdateQuality, ConfigProvider, EpgConfig, PanelApiConfig, ResourcePolicy}, + model::{macros, ConfigInputUpdateQuality, ConfigProvider, EpgConfig, PanelApiConfig}, utils::get_csv_file_path, }; use chrono::Utc; @@ -506,9 +506,6 @@ pub struct ConfigInput { pub provider_configs: Option>>, /// Resolved Stalker device identity + portal hints. pub stalker: Option, - /// Trusted private destinations for resource URLs supplied by this input. `None` and an - /// empty policy are equivalent: public destinations only. - pub resource_policy: Option>, } impl ConfigInput { @@ -897,8 +894,6 @@ impl ConfigInput { let password = alias.password.clone().or_else(|| cfg.password.clone()); StalkerInputConfig { username, password, ..cfg } }), - // An alias serves the same resources as its main input, so it inherits the policy. - resource_policy: self.resource_policy.clone(), } } @@ -991,13 +986,6 @@ impl From<&ConfigInputDto> for ConfigInput { .stalker .as_ref() .map(|s| StalkerInputConfig::from(s).with_credentials(dto.username.as_ref(), dto.password.as_ref())), - // An invalid policy is rejected while the sources config is built; dropping it here - // fails closed to public-only instead of silently trusting a partial allowlist. - resource_policy: dto - .resource_policy - .as_ref() - .and_then(|policy| ResourcePolicy::from_dto(policy).ok()) - .map(Arc::new), } } } @@ -1155,45 +1143,6 @@ mod tests { assert_update_quality_is_preserved(&alias_input); } - #[test] - fn alias_conversion_inherits_the_resource_policy() { - let dto = ConfigInputDto { - input_type: InputType::M3u, - url: "https://provider.example/playlist.m3u".to_string(), - resource_policy: Some(shared::model::ResourcePolicyDto { - allowed_hosts: vec!["media.home.arpa".to_string()], - allowed_networks: vec!["192.168.50.20/32".to_string()], - }), - aliases: Some(vec![ConfigInputAliasDto { - name: "alias".into(), - url: "https://alias.example.invalid".to_string(), - ..ConfigInputAliasDto::default() - }]), - ..ConfigInputDto::default() - }; - let input = ConfigInput::from(&dto); - let alias = input.aliases.as_ref().and_then(|aliases| aliases.first()).expect("runtime alias"); - - let alias_input = input.as_input(alias); - - let policy = alias_input.resource_policy.expect("alias inherits the policy"); - assert!(policy.allows_host("media.home.arpa")); - } - - #[test] - fn an_invalid_resource_policy_converts_to_public_only() { - // The load path rejects this configuration; the conversion itself must fail closed. - let dto = ConfigInputDto { - resource_policy: Some(shared::model::ResourcePolicyDto { - allowed_hosts: vec!["not a host".to_string()], - allowed_networks: Vec::new(), - }), - ..ConfigInputDto::default() - }; - - assert!(ConfigInput::from(&dto).resource_policy.is_none()); - } - #[test] fn batch_conversion_preserves_update_quality_when_promoting_alias() { let mut input = ConfigInput::from(&ConfigInputDto { diff --git a/backend/core/src/model/config/mod.rs b/backend/core/src/model/config/mod.rs index 57900f606..1d9221c36 100644 --- a/backend/core/src/model/config/mod.rs +++ b/backend/core/src/model/config/mod.rs @@ -24,7 +24,6 @@ mod proxy; mod qos_aggregation; mod rate_limit; mod rename; -mod resource_policy; mod reverse_proxy; mod schedule; mod sort; @@ -61,7 +60,6 @@ pub use proxy::*; pub use qos_aggregation::*; pub use rate_limit::*; pub use rename::*; -pub use resource_policy::*; pub use reverse_proxy::*; pub use schedule::*; pub use sort::*; diff --git a/backend/core/src/model/config/resource_policy.rs b/backend/core/src/model/config/resource_policy.rs deleted file mode 100644 index e10f3adf0..000000000 --- a/backend/core/src/model/config/resource_policy.rs +++ /dev/null @@ -1,486 +0,0 @@ -use crate::utils::{ - encode_base64_string, - network::request::{canonicalize_ip, classify_ip, AddressClass}, -}; -use ipnet::IpNet; -use shared::{error::TuliproxError, model::ResourcePolicyDto}; -use std::{ - fmt, - hash::{Hash, Hasher}, - net::IpAddr, - sync::{Arc, LazyLock}, -}; -use url::{Host, Url}; - -/// Domain separator for policy digests. Bumping it invalidates every persisted cache entry and -/// client identity that was derived from a policy. -const POLICY_DIGEST_DOMAIN: &[u8] = b"tuliprox.resource-policy.v1"; - -/// The networks a policy may authorize. Anything outside this list is either public or always -/// blocked, so accepting a broader range here could never grant access โ€” it would only widen the -/// address space an operator can accidentally authorize. -const ALLOWED_PRIVATE_NETWORKS: &[&str] = &["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7"]; - -/// Which destinations the resource proxy accepts for one input. -/// -/// An empty policy is public-only and is the normalized representation of "no exception -/// configured". Normalization is part of the contract: two configurations that normalize to the -/// same hosts and networks are equal, share one HTTP client, and share one cache identity. -#[derive(Debug, Clone, Default)] -pub struct ResourcePolicy { - pub allowed_hosts: Arc<[Arc]>, - pub allowed_networks: Arc<[IpNet]>, -} - -impl PartialEq for ResourcePolicy { - fn eq(&self, other: &Self) -> bool { self.digest() == other.digest() } -} - -impl Eq for ResourcePolicy {} - -impl Hash for ResourcePolicy { - fn hash(&self, state: &mut H) { self.digest().hash(state); } -} - -impl ResourcePolicy { - #[inline] - pub fn is_empty(&self) -> bool { self.allowed_hosts.is_empty() && self.allowed_networks.is_empty() } - - /// Stable identity of the normalized policy: the client key and the cache scope. - pub fn digest(&self) -> PolicyDigest { - let mut hasher = blake3::Hasher::new(); - hasher.update(POLICY_DIGEST_DOMAIN); - hasher.update(&(self.allowed_hosts.len() as u64).to_be_bytes()); - for host in self.allowed_hosts.iter() { - hasher.update(&(host.len() as u64).to_be_bytes()); - hasher.update(host.as_bytes()); - } - hasher.update(&(self.allowed_networks.len() as u64).to_be_bytes()); - for network in self.allowed_networks.iter() { - let network = network.to_string(); - hasher.update(&(network.len() as u64).to_be_bytes()); - hasher.update(network.as_bytes()); - } - PolicyDigest(encode_base64_string(&hasher.finalize().as_bytes()[..16]).into()) - } - - /// Digest of the public-only policy. Every request without an exception uses it, so the - /// cache scope of legacy and unconfigured inputs is one shared identity. - pub fn public_only_digest() -> PolicyDigest { public_only_policy().digest() } - - /// Exact host name match, case-insensitive because host names are stored normalized. - pub fn allows_host(&self, host: &str) -> bool { self.allowed_hosts.iter().any(|allowed| allowed.as_ref() == host) } - - pub fn allows_network(&self, address: IpAddr) -> bool { - let address = canonicalize_ip(address); - self.allowed_networks.iter().any(|network| network.contains(&address)) - } - - /// Decision for an address that came out of DNS resolution for `host`. - pub fn authorize_resolved(&self, host: &str, address: IpAddr) -> Result<(), ResourcePolicyError> { - match classify_ip(address) { - AddressClass::Blocked => Err(ResourcePolicyError::BlockedAddress), - AddressClass::Public => Ok(()), - AddressClass::Private => { - if !self.allows_host(host) { - return Err(ResourcePolicyError::HostNotTrusted); - } - if !self.allows_network(address) { - return Err(ResourcePolicyError::NetworkNotTrusted); - } - Ok(()) - } - } - } - - /// Decision for an IP literal inside a URL. A literal has no host name to match, so only the - /// network policy can authorize it and the literal itself is the destination. - pub fn authorize_literal(&self, address: IpAddr) -> Result<(), ResourcePolicyError> { - match classify_ip(address) { - AddressClass::Blocked => Err(ResourcePolicyError::BlockedAddress), - AddressClass::Public => Ok(()), - AddressClass::Private if self.allows_network(address) => Ok(()), - AddressClass::Private => Err(ResourcePolicyError::NetworkNotTrusted), - } - } - - /// Validates scheme, host presence, and IP literals of a URL before the first request. - /// - /// Host names are deliberately not resolved here: `PolicyIpResolver` is the single - /// connection-time enforcement point, so DNS rebinding protection and the single lookup stay - /// intact. An IP literal never reaches the resolver, which is why it is classified here. - pub fn validate_initial_url(&self, url: &Url) -> Result<(), ResourcePolicyError> { - if !matches!(url.scheme(), "http" | "https") { - return Err(ResourcePolicyError::UnsupportedScheme); - } - match url.host().ok_or(ResourcePolicyError::MissingHost)? { - Host::Ipv4(address) => self.authorize_literal(IpAddr::V4(address)), - Host::Ipv6(address) => self.authorize_literal(IpAddr::V6(address)), - Host::Domain(_) => Ok(()), - } - } - - /// Normalizes and validates a configured policy. - pub fn from_dto(dto: &ResourcePolicyDto) -> Result { - let allowed_hosts = normalize_hosts(&dto.allowed_hosts)?; - let allowed_networks = normalize_networks(&dto.allowed_networks)?; - Ok(Self { allowed_hosts: allowed_hosts.into(), allowed_networks: allowed_networks.into() }) - } - - /// Convenience for callers that hold a list of configured inputs. - pub fn from_optional_dto(dto: Option<&ResourcePolicyDto>) -> Result { - dto.map_or_else(|| Ok(Self::default()), Self::from_dto) - } -} - -/// The shared public-only policy instance. -pub fn public_only_policy() -> Arc { - static PUBLIC_ONLY: LazyLock> = LazyLock::new(|| Arc::new(ResourcePolicy::default())); - Arc::clone(&PUBLIC_ONLY) -} - -fn normalize_hosts(hosts: &[String]) -> Result>, TuliproxError> { - let mut normalized: Vec = Vec::with_capacity(hosts.len()); - for host in hosts { - let trimmed = host.trim(); - if trimmed.is_empty() { - return Err(policy_error("host entry must not be empty")); - } - if trimmed.contains("://") || trimmed.contains('/') { - return Err(policy_error(format!("'{trimmed}' is not a host name; remove scheme and path"))); - } - if trimmed.contains('*') { - return Err(policy_error(format!("wildcards are not supported in '{trimmed}'"))); - } - if trimmed.contains(':') { - return Err(policy_error(format!("'{trimmed}' must not contain a port"))); - } - // A DNS name only. IP literals are authorized through `allowed_networks`, and accepting - // them here would create a second, unchecked way to name a destination. - match Host::parse(trimmed).map_err(|_| policy_error(format!("'{trimmed}' is not a valid host name")))? { - Host::Domain(domain) => { - let domain = domain.trim_end_matches('.').to_ascii_lowercase(); - if domain.is_empty() { - return Err(policy_error(format!("'{trimmed}' is not a valid host name"))); - } - normalized.push(domain); - } - Host::Ipv4(_) | Host::Ipv6(_) => { - return Err(policy_error(format!("'{trimmed}' is an IP address; list it in allowed_networks instead"))) - } - } - } - normalized.sort_unstable(); - normalized.dedup(); - Ok(normalized.into_iter().map(Arc::from).collect()) -} - -fn normalize_networks(networks: &[String]) -> Result, TuliproxError> { - let allowed: Vec = - ALLOWED_PRIVATE_NETWORKS.iter().filter_map(|network| network.parse::().ok()).collect(); - let mut normalized: Vec = Vec::with_capacity(networks.len()); - for network in networks { - let trimmed = network.trim(); - let parsed = - trimmed.parse::().map_err(|_| policy_error(format!("'{trimmed}' is not a CIDR network")))?; - let parsed = parsed.trunc(); - // The whole network must be contained in a private range, so a broad entry such as - // 0.0.0.0/0 cannot silently authorize the public internet. - if !is_within_private_range(&parsed, &allowed) { - return Err(policy_error(format!( - "'{trimmed}' is not contained in a private range ({})", - ALLOWED_PRIVATE_NETWORKS.join(", ") - ))); - } - normalized.push(parsed); - } - normalized.sort_unstable(); - normalized.dedup(); - Ok(normalized) -} - -fn is_within_private_range(network: &IpNet, allowed: &[IpNet]) -> bool { - allowed.iter().any(|range| match (range, network) { - (IpNet::V4(range), IpNet::V4(net)) => net.prefix_len() >= range.prefix_len() && range.contains(&net.network()), - (IpNet::V6(range), IpNet::V6(net)) => net.prefix_len() >= range.prefix_len() && range.contains(&net.network()), - _ => false, - }) -} - -fn policy_error(message: impl Into) -> TuliproxError { - TuliproxError::ConfigInput(format!("invalid resource_policy: {}", message.into())) -} - -/// Short, filesystem-safe identity of a normalized policy. -#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)] -pub struct PolicyDigest(Arc); - -impl PolicyDigest { - #[inline] - pub fn as_str(&self) -> &str { &self.0 } -} - -impl fmt::Display for PolicyDigest { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.write_str(&self.0) } -} - -/// Which redirect strategy a resource client uses. Both modes are needed at the same time: the -/// EPG route returns upstream redirects, the cached resource routes follow a bounded number. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] -pub enum ResourceRedirectMode { - NoRedirect, - Bounded, -} - -/// Identifies one resource HTTP client. -#[derive(Debug, Clone, PartialEq, Eq, Hash)] -pub struct ResourceClientKey { - pub policy_digest: PolicyDigest, - pub redirect_mode: ResourceRedirectMode, -} - -impl ResourceClientKey { - pub fn new(policy_digest: PolicyDigest, redirect_mode: ResourceRedirectMode) -> Self { - Self { policy_digest, redirect_mode } - } -} - -/// Why a resource request was refused. The variant is kept for diagnostics; the HTTP layer maps -/// every policy rejection to the same status. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum ResourcePolicyError { - /// The origin names an input that does not exist or is disabled. - UnknownOrigin(String), - BlockedAddress, - HostNotTrusted, - NetworkNotTrusted, - TooManyRedirects, - UnsupportedScheme, - MissingHost, -} - -impl fmt::Display for ResourcePolicyError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::UnknownOrigin(name) => write!(f, "origin input '{name}' is unknown or disabled"), - Self::BlockedAddress => f.write_str("destination is an always-blocked address"), - Self::HostNotTrusted => f.write_str("private destination host is not listed in allowed_hosts"), - Self::NetworkNotTrusted => f.write_str("destination address is not inside allowed_networks"), - Self::TooManyRedirects => f.write_str("redirect limit exceeded"), - Self::UnsupportedScheme => f.write_str("unsupported resource URL scheme"), - Self::MissingHost => f.write_str("resource URL has no host"), - } - } -} - -impl std::error::Error for ResourcePolicyError {} - -#[cfg(test)] -mod tests { - use super::{public_only_policy, ResourceClientKey, ResourcePolicy, ResourcePolicyDto, ResourceRedirectMode}; - use std::net::IpAddr; - - fn policy(hosts: &[&str], networks: &[&str]) -> ResourcePolicy { - ResourcePolicy::from_dto(&ResourcePolicyDto { - allowed_hosts: hosts.iter().map(|h| (*h).to_string()).collect(), - allowed_networks: networks.iter().map(|n| (*n).to_string()).collect(), - }) - .expect("policy should be valid") - } - - fn ip(value: &str) -> IpAddr { value.parse().expect("ip address") } - - #[test] - fn identical_policies_share_one_digest() { - let first = policy(&["Media.Home.Arpa."], &["192.168.50.20/32", "10.0.0.0/8"]); - let second = policy(&["media.home.arpa"], &["10.0.0.0/8", "192.168.50.20/32"]); - let third = policy(&["media.home.arpa"], &["10.0.0.0/8", "192.168.50.16/28"]); - - assert_eq!(first.digest(), second.digest()); - assert_eq!(first, second); - assert_ne!(first.digest(), third.digest()); - } - - #[test] - fn empty_policy_is_public_only() { - let empty = ResourcePolicy::default(); - assert!(empty.is_empty()); - assert_eq!(empty.digest(), public_only_policy().digest()); - assert_eq!(empty.authorize_resolved("any.host", ip("8.8.8.8")), Ok(())); - assert_eq!( - empty.authorize_resolved("any.host", ip("10.1.2.3")), - Err(super::ResourcePolicyError::HostNotTrusted) - ); - assert_eq!(empty.authorize_literal(ip("192.168.1.1")), Err(super::ResourcePolicyError::NetworkNotTrusted)); - } - - #[test] - fn private_dns_destination_requires_host_and_network() { - let policy = policy(&["media.home.arpa"], &["192.168.50.20/32"]); - - assert_eq!(policy.authorize_resolved("media.home.arpa", ip("192.168.50.20")), Ok(())); - assert_eq!( - policy.authorize_resolved("media.home.arpa", ip("192.168.50.21")), - Err(super::ResourcePolicyError::NetworkNotTrusted) - ); - assert_eq!( - policy.authorize_resolved("other.home.arpa", ip("192.168.50.20")), - Err(super::ResourcePolicyError::HostNotTrusted) - ); - } - - #[test] - fn ip_literals_are_authorized_by_network_only() { - let policy = policy(&["media.home.arpa"], &["192.168.50.20/32"]); - assert_eq!(policy.authorize_literal(ip("192.168.50.20")), Ok(())); - assert_eq!(policy.authorize_literal(ip("192.168.50.21")), Err(super::ResourcePolicyError::NetworkNotTrusted)); - assert_eq!(policy.authorize_literal(ip("127.0.0.1")), Err(super::ResourcePolicyError::BlockedAddress)); - assert_eq!(policy.authorize_literal(ip("169.254.169.254")), Err(super::ResourcePolicyError::BlockedAddress)); - } - - #[test] - fn mapped_ipv6_is_classified_as_its_ipv4_address() { - let policy = policy(&["media.home.arpa"], &["192.168.50.20/32"]); - assert_eq!(policy.authorize_literal(ip("::ffff:192.168.50.20")), Ok(())); - assert_eq!(policy.authorize_literal(ip("::ffff:10.0.0.1")), Err(super::ResourcePolicyError::NetworkNotTrusted)); - } - - #[test] - fn public_destinations_stay_allowed() { - let policy = policy(&["media.home.arpa"], &["192.168.50.20/32"]); - assert_eq!(policy.authorize_resolved("cdn.example.com", ip("93.184.216.34")), Ok(())); - assert_eq!(policy.authorize_literal(ip("1.1.1.1")), Ok(())); - } - - #[test] - fn blocked_ranges_are_rejected_even_with_a_matching_network() { - // 10.0.0.0/8 covers none of these, but the check must not depend on the policy at all. - let policy = policy(&[], &["10.0.0.0/8"]); - for blocked in ["127.0.0.1", "169.254.1.1", "0.0.0.0", "224.0.0.1", "255.255.255.255", "100.64.0.1"] { - assert_eq!( - policy.authorize_literal(ip(blocked)), - Err(super::ResourcePolicyError::BlockedAddress), - "{blocked} must stay blocked" - ); - } - for blocked in ["::1", "fe80::1", "ff02::1", "2001:db8::1"] { - assert_eq!( - policy.authorize_literal(ip(blocked)), - Err(super::ResourcePolicyError::BlockedAddress), - "{blocked} must stay blocked" - ); - } - } - - #[test] - fn ula_addresses_are_private() { - let policy = policy(&["nas.home.arpa"], &["fd00::/64"]); - assert_eq!(policy.authorize_resolved("nas.home.arpa", ip("fd00::5")), Ok(())); - assert_eq!( - policy.authorize_resolved("nas.home.arpa", ip("fd00:1::5")), - Err(super::ResourcePolicyError::NetworkNotTrusted) - ); - } - - #[test] - fn initial_url_validation_checks_scheme_host_and_literals() { - let policy = policy(&["media.home.arpa"], &["192.168.50.20/32"]); - - let allowed = url::Url::parse("http://192.168.50.20/logo.png").expect("url"); - assert_eq!(policy.validate_initial_url(&allowed), Ok(())); - - let rejected = url::Url::parse("http://192.168.1.1/logo.png").expect("url"); - assert_eq!(policy.validate_initial_url(&rejected), Err(super::ResourcePolicyError::NetworkNotTrusted)); - - let blocked = url::Url::parse("http://[::1]/logo.png").expect("url"); - assert_eq!(policy.validate_initial_url(&blocked), Err(super::ResourcePolicyError::BlockedAddress)); - - // Host names are resolved later, by the connection-time resolver. - let name = url::Url::parse("https://media.home.arpa/logo.png").expect("url"); - assert_eq!(policy.validate_initial_url(&name), Ok(())); - - let scheme = url::Url::parse("ftp://media.home.arpa/logo.png").expect("url"); - assert_eq!(policy.validate_initial_url(&scheme), Err(super::ResourcePolicyError::UnsupportedScheme)); - } - - #[test] - fn host_entries_are_validated_and_normalized() { - let dto = ResourcePolicyDto { - allowed_hosts: vec!["Media.Home.Arpa.".to_string(), " media.home.arpa ".to_string()], - allowed_networks: Vec::new(), - }; - let policy = ResourcePolicy::from_dto(&dto).expect("policy"); - assert_eq!(policy.allowed_hosts.len(), 1); - assert!(policy.allows_host("media.home.arpa")); - - for invalid in ["", "https://host/path", "host:8080", "*.home.arpa", "192.168.1.1"] { - let dto = ResourcePolicyDto { allowed_hosts: vec![invalid.to_string()], allowed_networks: Vec::new() }; - assert!(ResourcePolicy::from_dto(&dto).is_err(), "'{invalid}' must be rejected"); - } - } - - #[test] - fn networks_outside_private_ranges_are_rejected() { - for invalid in ["0.0.0.0/0", "8.8.8.0/24", "172.32.0.0/16", "2001:db8::/32", "not-a-network"] { - let dto = ResourcePolicyDto { allowed_hosts: Vec::new(), allowed_networks: vec![invalid.to_string()] }; - assert!(ResourcePolicy::from_dto(&dto).is_err(), "'{invalid}' must be rejected"); - } - - for valid in ["10.0.0.0/8", "172.16.5.0/24", "192.168.50.20/32", "fc00::/7", "fd00::/64"] { - let dto = ResourcePolicyDto { allowed_hosts: Vec::new(), allowed_networks: vec![valid.to_string()] }; - assert!(ResourcePolicy::from_dto(&dto).is_ok(), "'{valid}' must be accepted"); - } - } - - #[tokio::test] - async fn the_resolver_keeps_only_authorized_addresses() { - use crate::utils::network::request::resolve_policy_socket_addrs; - - // A public literal is authorized by every policy, including the empty one. - let addresses = resolve_policy_socket_addrs("1.1.1.1", 80, &ResourcePolicy::default()) - .await - .expect("public literal resolves"); - assert_eq!(addresses.len(), 1); - assert_eq!(addresses[0].port(), 80); - - // Loopback stays blocked even though the policy resolver is the only check here. - let loopback = resolve_policy_socket_addrs("127.0.0.1", 80, &ResourcePolicy::default()).await; - assert_eq!( - loopback.err().map(|err| err.kind()), - Some(std::io::ErrorKind::PermissionDenied), - "loopback must be refused" - ); - - let private = resolve_policy_socket_addrs("192.168.1.1", 80, &ResourcePolicy::default()).await; - assert_eq!( - private.err().map(|err| err.kind()), - Some(std::io::ErrorKind::PermissionDenied), - "a private literal without a policy must be refused" - ); - } - - #[test] - fn loopback_can_never_be_authorized() { - // Loopback is outside every allowed private range, so a policy that tries to authorize it - // is rejected while the configuration is loaded instead of silently accepting it. - let dto = ResourcePolicyDto { - allowed_hosts: vec!["localhost".to_string()], - allowed_networks: vec!["127.0.0.1/32".to_string()], - }; - assert!(ResourcePolicy::from_dto(&dto).is_err()); - } - - #[test] - fn client_keys_separate_policies_and_redirect_modes() { - let first = ResourceClientKey::new(policy(&["a.home.arpa"], &[]).digest(), ResourceRedirectMode::Bounded); - let same = ResourceClientKey::new(policy(&["a.home.arpa"], &[]).digest(), ResourceRedirectMode::Bounded); - let other_mode = - ResourceClientKey::new(policy(&["a.home.arpa"], &[]).digest(), ResourceRedirectMode::NoRedirect); - let other_policy = - ResourceClientKey::new(policy(&["b.home.arpa"], &[]).digest(), ResourceRedirectMode::Bounded); - - assert_eq!(first, same); - assert_ne!(first, other_mode); - assert_ne!(first, other_policy); - } -} diff --git a/backend/core/src/model/config/source.rs b/backend/core/src/model/config/source.rs index 6a78b68f2..74843210e 100644 --- a/backend/core/src/model/config/source.rs +++ b/backend/core/src/model/config/source.rs @@ -1,4 +1,4 @@ -use crate::model::{macros, ConfigInput, ConfigTarget, ProcessTargets, ResourcePolicy}; +use crate::model::{macros, ConfigInput, ConfigTarget, ProcessTargets}; use indexmap::IndexMap; use parking_lot::RwLock; use shared::{ @@ -433,12 +433,6 @@ impl TryFrom<&SourcesConfigDto> for SourcesConfig { .unwrap_or_default(); for input_dto in &dto.inputs { - // Reject an invalid policy instead of falling back to a partially normalized - // allowlist; a typo in a host or CIDR must not silently widen or narrow access. - if let Some(policy_dto) = &input_dto.resource_policy { - ResourcePolicy::from_dto(policy_dto) - .map_err(|err| TuliproxError::ConfigInput(format!("input '{}': {err}", input_dto.name)))?; - } let mut input = ConfigInput::from(input_dto); // Prepare input if let Some(path) = input.prepare(&provider)? { @@ -448,8 +442,6 @@ impl TryFrom<&SourcesConfigDto> for SourcesConfig { inputs.push(input); } - check_unique_member_names(&inputs)?; - // Resolve staged playlist inputs to their configured download type. // The provider link is kept for overlay routing and must not affect staged playlist fetching. if inputs.iter().any(|input| input.input_type.is_staged()) { @@ -461,6 +453,8 @@ impl TryFrom<&SourcesConfigDto> for SourcesConfig { } } + check_unique_member_names(&inputs)?; + let inputs: Vec> = inputs.into_iter().map(Arc::new).collect(); let group_lookup = build_group_lookup(&inputs); @@ -481,9 +475,11 @@ impl TryFrom<&SourcesConfigDto> for SourcesConfig { /// Rejects duplicate input and alias names. /// -/// The canonical input name is the identity a resource origin resolves to, and aliases share that -/// namespace through `group_lookup`. A duplicate would silently resolve to whichever entry was -/// inserted last, which would authorize a resource against the wrong policy, so it is a load error. +/// Input names and alias names share one namespace: both name a member in `group_lookup`, and that map +/// decides which input a member belongs to. A duplicate would silently resolve to whichever member was +/// inserted last, so a request would run against another input's playlist, credentials, and limits. +/// Such a configuration is a load error, and it must be reported while the sources are read - a reload +/// that returns an error leaves the running configuration in place. fn check_unique_member_names(inputs: &[ConfigInput]) -> Result<(), TuliproxError> { let mut seen: HashMap<&str, &str> = HashMap::with_capacity(inputs.len()); for input in inputs { @@ -612,57 +608,30 @@ impl SourcesConfig { mod tests { use super::{ConfigProvider, SourcesConfig}; use shared::model::{ - ConfigInputDto, ConfigProviderDto, ProviderUrlSelectionPolicy, ResourcePolicyDto, SourcesConfigDto, + ConfigInputAliasDto, ConfigInputDto, ConfigProviderDto, ProviderUrlSelectionPolicy, SourcesConfigDto, }; use std::net::IpAddr; - fn sources_dto(resource_policy: Option) -> SourcesConfigDto { + fn sources_dto() -> SourcesConfigDto { SourcesConfigDto { templates: None, provider: None, inputs: vec![ConfigInputDto { - name: "policy-input".into(), + name: "first-input".into(), url: "https://provider.example/playlist.m3u".to_string(), - resource_policy, ..ConfigInputDto::default() }], sources: Vec::new(), } } - #[test] - fn a_valid_resource_policy_loads_into_the_runtime_input() { - let dto = sources_dto(Some(ResourcePolicyDto { - allowed_hosts: vec!["Media.Home.Arpa".to_string()], - allowed_networks: vec!["192.168.50.20/32".to_string()], - })); - - let sources = SourcesConfig::try_from(&dto).expect("policy should be accepted"); - let input = sources.inputs.first().expect("input"); - let policy = input.resource_policy.as_ref().expect("policy"); - - assert!(policy.allows_host("media.home.arpa")); - assert_eq!(policy.allowed_networks.len(), 1); - } - - #[test] - fn an_invalid_resource_policy_fails_the_config_load() { - for invalid in [ - ResourcePolicyDto { allowed_hosts: vec!["https://host/path".to_string()], allowed_networks: Vec::new() }, - ResourcePolicyDto { allowed_hosts: Vec::new(), allowed_networks: vec!["0.0.0.0/0".to_string()] }, - ResourcePolicyDto { allowed_hosts: Vec::new(), allowed_networks: vec!["public.example/24".to_string()] }, - ] { - let dto = sources_dto(Some(invalid.clone())); - let error = SourcesConfig::try_from(&dto).expect_err("invalid policy must be rejected"); - assert!(error.to_string().contains("resource_policy"), "error should name the policy: {error}"); - } - } - + /// A duplicate member name silently resolves to whichever entry was inserted last, so a request + /// would run against another input. #[test] fn duplicate_input_names_fail_the_config_load() { - let mut dto = sources_dto(None); + let mut dto = sources_dto(); dto.inputs.push(ConfigInputDto { - name: "policy-input".into(), + name: "first-input".into(), url: "https://other.example/playlist.m3u".to_string(), ..ConfigInputDto::default() }); @@ -674,19 +643,19 @@ mod tests { #[test] fn duplicate_alias_names_fail_the_config_load() { - let mut dto = sources_dto(None); - dto.inputs[0].aliases = Some(vec![shared::model::ConfigInputAliasDto { + let mut dto = sources_dto(); + dto.inputs[0].aliases = Some(vec![ConfigInputAliasDto { name: "shared-alias".into(), url: "https://provider.example/alias.m3u".to_string(), - ..shared::model::ConfigInputAliasDto::default() + ..ConfigInputAliasDto::default() }]); dto.inputs.push(ConfigInputDto { name: "second-input".into(), url: "https://second.example/playlist.m3u".to_string(), - aliases: Some(vec![shared::model::ConfigInputAliasDto { + aliases: Some(vec![ConfigInputAliasDto { name: "shared-alias".into(), url: "https://second.example/alias.m3u".to_string(), - ..shared::model::ConfigInputAliasDto::default() + ..ConfigInputAliasDto::default() }]), ..ConfigInputDto::default() }); @@ -698,14 +667,14 @@ mod tests { #[test] fn an_alias_name_colliding_with_an_input_name_fails_the_config_load() { - let mut dto = sources_dto(None); + let mut dto = sources_dto(); dto.inputs.push(ConfigInputDto { name: "second-input".into(), url: "https://second.example/playlist.m3u".to_string(), - aliases: Some(vec![shared::model::ConfigInputAliasDto { - name: "policy-input".into(), + aliases: Some(vec![ConfigInputAliasDto { + name: "first-input".into(), url: "https://second.example/alias.m3u".to_string(), - ..shared::model::ConfigInputAliasDto::default() + ..ConfigInputAliasDto::default() }]), ..ConfigInputDto::default() }); @@ -716,12 +685,24 @@ mod tests { } #[test] - fn an_input_without_a_resource_policy_has_none() { - let dto = sources_dto(None); + fn distinct_member_names_load() { + let mut dto = sources_dto(); + dto.inputs.push(ConfigInputDto { + name: "second-input".into(), + url: "https://second.example/playlist.m3u".to_string(), + aliases: Some(vec![ConfigInputAliasDto { + name: "second-alias".into(), + url: "https://second.example/alias.m3u".to_string(), + ..ConfigInputAliasDto::default() + }]), + ..ConfigInputDto::default() + }); - let sources = SourcesConfig::try_from(&dto).expect("load"); + let sources = SourcesConfig::try_from(&dto).expect("distinct member names must load"); - assert!(sources.inputs.first().expect("input").resource_policy.is_none()); + assert_eq!(sources.inputs.len(), 2); + assert!(sources.group_lookup.contains_key("first-input")); + assert!(sources.group_lookup.contains_key("second-input")); } #[test] diff --git a/backend/core/src/model/provider.rs b/backend/core/src/model/provider.rs index 40157e322..97c7d6660 100644 --- a/backend/core/src/model/provider.rs +++ b/backend/core/src/model/provider.rs @@ -584,7 +584,6 @@ mod tests { provider_configs: None, cache_duration_seconds: 0, stalker: None, - resource_policy: None, }; let conn = Arc::new(RwLock::new(ProviderConfigConnection::default())); let counter = Arc::new(AtomicUsize::new(0)); diff --git a/backend/core/src/model/xmltv.rs b/backend/core/src/model/xmltv.rs index 8ac31cfae..66c3e0ede 100644 --- a/backend/core/src/model/xmltv.rs +++ b/backend/core/src/model/xmltv.rs @@ -97,8 +97,6 @@ pub struct PersistedEpgSource { pub priority: i16, pub logo_override: bool, pub kind: PersistedEpgSourceKind, - /// Canonical input that owns resource values read from this source. - pub input_name: Option>, } fn filter_channels_and_programmes(channels: &mut Vec, programmes: &mut Vec) { diff --git a/backend/core/src/model/xtream.rs b/backend/core/src/model/xtream.rs index 216cb470e..bb4d5eabd 100644 --- a/backend/core/src/model/xtream.rs +++ b/backend/core/src/model/xtream.rs @@ -90,6 +90,7 @@ pub fn xtream_mapping_option_from_target_options( base_url, web_ui_request: user.t_is_api_user, encrypt_secret, + resource_host_policies: Arc::default(), }) } diff --git a/backend/core/src/utils/crypto_utils.rs b/backend/core/src/utils/crypto_utils.rs index 674b05c91..bc0d3326f 100644 --- a/backend/core/src/utils/crypto_utils.rs +++ b/backend/core/src/utils/crypto_utils.rs @@ -13,8 +13,11 @@ use rand::{rngs::OsRng, RngCore, TryRngCore}; use shared::{error::TuliproxError, utils::encode_base64_string}; type Aes128Ctr = ctr::Ctr128BE; +#[cfg(test)] const AUTH_TOKEN_VERSION: u8 = 1; +#[cfg(test)] const AES_128_CTR_IV_LEN: usize = 16; +#[cfg(test)] const BLAKE3_MAC_LEN: usize = 32; pub fn encode_base64_hash(text: &str) -> String { @@ -29,6 +32,7 @@ fn apply_aes_128_ctr(secret: &[u8; 16], iv: &[u8], data: &mut [u8]) -> Result<() Ok(()) } +#[cfg(test)] fn derive_authenticated_token_mac_key(secret: &[u8; 16], domain: &[u8]) -> [u8; BLAKE3_MAC_LEN] { let mut hasher = blake3::Hasher::new(); hasher.update(b"tuliprox.authenticated-token.mac.v1"); @@ -38,84 +42,13 @@ fn derive_authenticated_token_mac_key(secret: &[u8; 16], domain: &[u8]) -> [u8; *hasher.finalize().as_bytes() } +#[cfg(test)] fn authenticated_token_mac(secret: &[u8; 16], domain: &[u8], data: &[u8]) -> blake3::Hash { let mac_key = derive_authenticated_token_mac_key(secret, domain); blake3::keyed_hash(&mac_key, data) } -/// Derives the AES-CTR IV synthetically (SIV) from the secret, the domain and the plaintext, -/// so that encoding the same payload twice yields the same token. -/// -/// The IV stays unpredictable to anyone without `secret`, and distinct payloads (or domains) -/// still get distinct IVs, so the CTR keystream is never reused across different plaintexts. -/// Encoding the same payload twice deliberately produces an identical token: these tokens are -/// stable per-item URLs (STRM files, M3U catchup URLs), and equality of two tokens reveals only -/// that they point at the same item โ€” which their surrounding filename/playlist entry already says. -fn derive_synthetic_iv(secret: &[u8; 16], domain: &[u8], plaintext: &[u8]) -> [u8; AES_128_CTR_IV_LEN] { - let mut hasher = blake3::Hasher::new(); - hasher.update(b"tuliprox.authenticated-token.siv.v1"); - hasher.update(&(domain.len() as u64).to_be_bytes()); - hasher.update(domain); - hasher.update(secret); - hasher.update(plaintext); - let mut iv = [0u8; AES_128_CTR_IV_LEN]; - iv.copy_from_slice(&hasher.finalize().as_bytes()[..AES_128_CTR_IV_LEN]); - iv -} - -pub fn obscure_authenticated_bytes( - secret: &[u8; 16], - domain: &[u8], - plaintext: &[u8], -) -> Result { - let iv = derive_synthetic_iv(secret, domain, plaintext); - - let data_len = 1 + AES_128_CTR_IV_LEN + plaintext.len(); - let mut out = Vec::with_capacity(data_len + BLAKE3_MAC_LEN); - out.push(AUTH_TOKEN_VERSION); - out.extend_from_slice(&iv); - out.extend_from_slice(plaintext); - apply_aes_128_ctr(secret, &iv, &mut out[1 + AES_128_CTR_IV_LEN..])?; - - let mac = authenticated_token_mac(secret, domain, &out); - out.extend_from_slice(mac.as_bytes()); - Ok(general_purpose::URL_SAFE_NO_PAD.encode(out)) -} - -pub fn deobscure_authenticated_bytes( - secret: &[u8; 16], - domain: &[u8], - encoded: &str, -) -> Result, TuliproxError> { - let data = general_purpose::URL_SAFE_NO_PAD - .decode(encoded) - .map_err(|_| TuliproxError::Crypto("Can't decode authenticated token".to_string()))?; - - let min_len = 1 + AES_128_CTR_IV_LEN + BLAKE3_MAC_LEN; - if data.len() < min_len { - return Err(TuliproxError::Crypto("Authenticated token is too short".to_string())); - } - if data[0] != AUTH_TOKEN_VERSION { - return Err(TuliproxError::Crypto("Unsupported authenticated token version".to_string())); - } - - let mac_offset = data.len() - BLAKE3_MAC_LEN; - let (authenticated_data, token_mac) = data.split_at(mac_offset); - let expected_mac = authenticated_token_mac(secret, domain, authenticated_data); - let token_mac = <[u8; BLAKE3_MAC_LEN]>::try_from(token_mac) - .map(blake3::Hash::from_bytes) - .map_err(|_| TuliproxError::Crypto("Invalid authenticated token MAC".to_string()))?; - if expected_mac != token_mac { - return Err(TuliproxError::Crypto("Authenticated token MAC mismatch".to_string())); - } - - let iv_end = 1 + AES_128_CTR_IV_LEN; - let iv = &authenticated_data[1..iv_end]; - let ciphertext = &authenticated_data[iv_end..]; - let mut plaintext = ciphertext.to_vec(); - apply_aes_128_ctr(secret, iv, &mut plaintext)?; - Ok(plaintext) -} +pub use shared::utils::{deobscure_authenticated_bytes, obscure_authenticated_bytes}; pub fn obscure_text(secret: &[u8; 16], url: &str) -> Result { let mut iv = [0u8; 16]; diff --git a/backend/core/src/utils/lru_cache.rs b/backend/core/src/utils/lru_cache.rs index 44efa731f..e7e356fb7 100644 --- a/backend/core/src/utils/lru_cache.rs +++ b/backend/core/src/utils/lru_cache.rs @@ -12,25 +12,6 @@ const CACHEDIR_TAG: &str = "CACHEDIR.TAG"; #[inline] fn encode_cache_key(key: &str) -> String { encode_base64_hash(key) } -/// Cache identity of a proxied resource. -/// -/// One URL can be fetched under different destination policies, input identities, and credential -/// contexts. All three scopes belong to the key so a body fetched in one authorization context can -/// never answer another. The version domain invalidates entries created with an older layout. -pub fn resource_cache_key( - policy_digest: &str, - canonical_input: &str, - credential_context: &str, - canonical_url: &str, -) -> String { - format!( - "tuliprox.resource.v2|{}:{policy_digest}|{}:{canonical_input}|{}:{credential_context}|{canonical_url}", - policy_digest.len(), - canonical_input.len(), - credential_context.len(), - ) -} - /// `LRUResourceCache` /// /// A least-recently-used (LRU) file-based resource cache that stores files in a directory on disk, @@ -265,20 +246,3 @@ impl LRUResourceCache { } } } - -#[cfg(test)] -mod cache_key_tests { - use super::resource_cache_key; - - #[test] - fn resource_cache_key_separates_policies() { - let url = "https://cdn.example.com/logo.png"; - let public_only = resource_cache_key("public", "input", "credentials", url); - let private = resource_cache_key("private", "input", "credentials", url); - - assert_ne!(public_only, private); - assert_ne!(public_only, resource_cache_key("public", "other-input", "credentials", url)); - assert_ne!(public_only, resource_cache_key("public", "input", "other-credentials", url)); - assert_eq!(public_only, resource_cache_key("public", "input", "credentials", url)); - } -} diff --git a/backend/core/src/utils/mod.rs b/backend/core/src/utils/mod.rs index db313c6ce..7cdd49c9d 100644 --- a/backend/core/src/utils/mod.rs +++ b/backend/core/src/utils/mod.rs @@ -17,7 +17,7 @@ mod ordinal; mod provider_resolve_token; mod recording_paths; pub mod request_headers; -mod resource_token; +mod resource_output; pub mod response_compression; pub mod runtime_liveness; mod step_measure; @@ -84,7 +84,7 @@ pub use self::{ ordinal::*, provider_resolve_token::*, recording_paths::*, - resource_token::*, + resource_output::*, step_measure::*, sys_utils::*, telegram::*, diff --git a/backend/core/src/utils/network/request.rs b/backend/core/src/utils/network/request.rs index b7d964e33..50d19a8ef 100644 --- a/backend/core/src/utils/network/request.rs +++ b/backend/core/src/utils/network/request.rs @@ -8,8 +8,7 @@ pub const STREAM_IDLE_TIMEOUT: u64 = 60; use crate::{ model::{ resolve_provider_scheme_url_with_provider_index, AppConfig, Config, ConfigInput, ConfigProvider, InputSource, - ResourcePolicy, ResourcePolicyError, ResourceRedirectMode, ResourceRetryConfig, - ReverseProxyDisabledHeaderConfig, + ResourceRetryConfig, ReverseProxyDisabledHeaderConfig, }, utils::{ async_file_reader, async_file_writer, @@ -27,6 +26,7 @@ use crate::{ }; use futures::StreamExt; use log::{debug, error, log_enabled, trace, warn, Level}; +use lru::LruCache; use regex::Regex; use reqwest::{ header::{HeaderMap, HeaderName, HeaderValue, HOST}, @@ -43,16 +43,17 @@ use std::{ collections::{HashMap, HashSet}, io::{Error, ErrorKind}, net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}, + num::NonZeroUsize, path::{Path, PathBuf}, - sync::{Arc, Once}, - time::Duration, + sync::{Arc, Mutex, Once, OnceLock, PoisonError}, + time::{Duration, Instant}, }; use tokio::{ fs::File, io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt}, - time::sleep, + time::{sleep, timeout}, }; -use url::{Host, Url}; +use url::Url; static PROXY_DIAGNOSTICS_ONCE: Once = Once::new(); @@ -71,123 +72,156 @@ impl reqwest::dns::Resolve for PublicIpResolver { } } -/// Resolves a host and keeps only the addresses the policy authorizes. +pub async fn resolve_public_socket_addrs(host: &str, port: u16) -> std::io::Result> { + let addresses = lookup_socket_addrs(host, port).await?; + if addresses.is_empty() || addresses.iter().any(|address| !is_public_ip(address.ip())) { + return Err(Error::new(ErrorKind::PermissionDenied, "destination resolves to a non-public address")); + } + Ok(addresses) +} + +/// Resolves a resource destination, refusing addresses that are local to this host. /// -/// The resolver is the connection-time enforcement point for host names. It runs on every -/// connection, so a DNS answer that changes between requests is re-evaluated instead of being -/// trusted from a one-time pre-check. IP literals in a URL bypass DNS entirely, which is why -/// `ResourcePolicy::validate_initial_url` and the redirect policy classify them separately. -#[derive(Debug, Clone)] -pub struct PolicyIpResolver { - policy: Arc, +/// Private network destinations stay allowed: self-hosted services are a normal resource source. +pub async fn resolve_resource_socket_addrs(host: &str, port: u16) -> std::io::Result> { + let addresses = lookup_socket_addrs(host, port).await?; + if addresses.is_empty() { + return Err(Error::new(ErrorKind::PermissionDenied, "destination does not resolve")); + } + if addresses.iter().any(|address| classify_ip(address.ip()) == ResourceDestination::Blocked) { + return Err(Error::new(ErrorKind::PermissionDenied, "destination is local to this host")); + } + Ok(addresses) } -impl PolicyIpResolver { - pub fn new(policy: Arc) -> Self { Self { policy } } +/// Resolves a host name or IP literal, bracketed IPv6 included. An unresolvable name yields an empty +/// result so that every policy decides on its own how to report it. +async fn lookup_socket_addrs(host: &str, port: u16) -> std::io::Result> { + if let Ok(address) = host_literal(host).parse::() { + return Ok(vec![SocketAddr::new(address, port)]); + } + Ok(tokio::net::lookup_host((host, port)).await?.collect()) } -impl reqwest::dns::Resolve for PolicyIpResolver { +/// Hosts of the proxies an outbound request may be routed through. +/// +/// A resource client that honours the configured proxy asks its resolver about the proxy host as well, +/// and a proxy commonly lives on the loopback interface (`http://localhost:8118`). That name is exempt +/// from the local-address guard, because a proxy on this host is a valid destination for this host. +fn proxy_hosts(app_config: &AppConfig) -> Vec> { + const ENV_KEYS: [&str; 3] = ["HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY"]; + let mut hosts = Vec::new(); + let config = app_config.config.load(); + if let Some(proxy) = config.proxy.as_ref() { + if let Some(host) = + crate::model::parse_proxy_url_with_http_fallback(&proxy.url).and_then(|url| url.host_str().map(Arc::from)) + { + hosts.push(host); + } + } + drop(config); + for (key, value) in std::env::vars_os() { + let (Some(key), Some(value)) = (key.to_str(), value.to_str()) else { + continue; + }; + if !ENV_KEYS.iter().any(|candidate| candidate.eq_ignore_ascii_case(key)) { + continue; + } + if let Some(host) = + crate::model::parse_proxy_url_with_http_fallback(value).and_then(|url| url.host_str().map(Arc::from)) + { + hosts.push(host); + } + } + hosts +} + +/// DNS layer of the proxied resource client. +/// +/// Classifying a destination leaves a window: a name can resolve to a public address while it is +/// classified and to a local one while the connection is built. This resolver closes that window, +/// because the addresses it validates are the addresses reqwest connects to. It deliberately does not +/// consult [`DestinationCache`]: a remembered verdict would reopen exactly that window. +/// +/// The proxy hosts are exempt: they are resolved without the guard, so a proxy on the loopback +/// interface stays usable. Nothing else is, because everything else is a destination a client picked. +#[derive(Debug, Clone, Default)] +pub struct ResourceDestinationResolver { + allowed_hosts: Arc<[Arc]>, +} + +impl ResourceDestinationResolver { + /// Resolver that resolves the proxy hosts of this configuration and guards every other name. + pub fn allowing_proxy_hosts(app_config: &AppConfig) -> Self { + Self { allowed_hosts: proxy_hosts(app_config).into() } + } +} + +impl reqwest::dns::Resolve for ResourceDestinationResolver { fn resolve(&self, name: reqwest::dns::Name) -> reqwest::dns::Resolving { - let host = name.as_str().to_string(); - let policy = Arc::clone(&self.policy); + let host = name.as_str().to_owned(); + let allowed = self.allowed_hosts.iter().any(|allowed| allowed.as_ref().eq_ignore_ascii_case(&host)); Box::pin(async move { - let addresses = resolve_policy_socket_addrs(&host, 0, &policy) - .await - .map_err(|err| Box::new(err) as Box)?; + let addresses = if allowed { + lookup_socket_addrs(&host, 0).await + } else { + resolve_resource_socket_addrs(&host, 0).await + } + .map_err(|err| Box::new(err) as Box)?; Ok(Box::new(addresses.into_iter()) as reqwest::dns::Addrs) }) } } -pub async fn resolve_policy_socket_addrs( - host: &str, - port: u16, - policy: &ResourcePolicy, -) -> std::io::Result> { - let addresses: Vec = if let Ok(address) = host.parse::() { - vec![SocketAddr::new(address, port)] - } else { - tokio::net::lookup_host((host, port)).await?.collect() - }; - - if addresses.is_empty() { - return Err(std::io::Error::new(std::io::ErrorKind::NotFound, "destination did not resolve to an address")); - } - - // Mixed answers keep only the approved addresses: a blocked entry must never be left in the - // list as a usable fallback, and an approved one must not be dropped because a sibling entry - // was rejected. - let literal = host.parse::().ok(); - let mut approved = Vec::with_capacity(addresses.len()); - let mut rejection = None; - for address in addresses { - // A literal has no host name to match, so it is judged by the address policy alone. IP - // literals normally never reach a resolver; this keeps the decision consistent if one does. - let decision = match literal { - Some(literal) => policy.authorize_literal(literal), - None => policy.authorize_resolved(host, address.ip()), - }; - match decision { - Ok(()) => approved.push(address), - Err(err) => rejection = Some(err), - } - } - - if approved.is_empty() { - return Err(std::io::Error::new( - std::io::ErrorKind::PermissionDenied, - rejection.map_or_else(|| "destination is not authorized".to_string(), |err| err.to_string()), - )); - } - Ok(approved) -} - -pub async fn resolve_public_socket_addrs(host: &str, port: u16) -> std::io::Result> { - let addresses = if let Ok(address) = host.parse::() { - vec![SocketAddr::new(address, port)] - } else { - tokio::net::lookup_host((host, port)).await?.collect() - }; - if addresses.is_empty() || addresses.iter().any(|address| !is_public_ip(address.ip())) { - return Err(std::io::Error::new( - std::io::ErrorKind::PermissionDenied, - "destination resolves to a non-public address", - )); - } - Ok(addresses) -} - -/// How a destination address is treated when a resource policy decides on it. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum AddressClass { - /// Routable on the public internet. +/// Reachability class of a resource destination. +/// +/// Resource URLs are supplied by providers and third-party EPG data, so they are untrusted input. +/// The class answers two separate questions: may a client be handed the URL as-is (any non-public +/// destination would expose an internal address), and may the proxy fetch it (a destination local +/// to the proxy host would turn the proxy into a reader for the host itself). +#[derive(Debug, Clone, Copy, Eq, PartialEq)] +pub enum ResourceDestination { + /// Publicly routable: safe to expose to clients and to fetch. Public, - /// RFC 1918 or IPv6 ULA: reachable only through an explicit policy entry. + /// Non-public but routable, which is normal for self-hosted services on the local network + /// (RFC1918, unique local addresses, carrier NAT, reserved ranges). Never exposed to clients. Private, - /// Never reachable, with or without a policy. + /// Local to the proxy host or its segment: loopback, link-local (including the cloud metadata + /// endpoints), unspecified, multicast or broadcast. Never exposed and never fetched. Blocked, } -/// Classifies an address, canonicalizing IPv4-mapped IPv6 first so a mapped private address is -/// judged as the IPv4 address it represents. -pub fn classify_ip(address: IpAddr) -> AddressClass { - match canonicalize_ip(address) { +/// Reachability class of an address. Single place that decides which range belongs to which class. +pub fn classify_ip(address: IpAddr) -> ResourceDestination { + match address { IpAddr::V4(address) => classify_ipv4(address), IpAddr::V6(address) => classify_ipv6(address), } } -/// Collapses an IPv4-mapped IPv6 address to the IPv4 address it represents. -/// -/// Policy entries are stored as either family, so both the classification and the network -/// containment check have to see the same family the operator wrote down. -pub fn canonicalize_ip(address: IpAddr) -> IpAddr { - match address { - IpAddr::V6(address) => address.to_ipv4_mapped().map_or(IpAddr::V6(address), IpAddr::V4), - IpAddr::V4(address) => IpAddr::V4(address), +fn classify_ipv4(address: Ipv4Addr) -> ResourceDestination { + let [first, second, _, _] = address.octets(); + if address.is_loopback() + || address.is_link_local() + || address.is_unspecified() + || address.is_multicast() + || address.is_broadcast() + || first == 0 + { + ResourceDestination::Blocked + } else if address.is_private() + || address.is_documentation() + || (first == 100 && (64..=127).contains(&second)) + || (first == 198 && matches!(second, 18 | 19)) + || first >= 240 + { + ResourceDestination::Private + } else { + ResourceDestination::Public } } +/// IPv4 address carried inside an IPv6 address: IPv4-compatible, NAT64 (`64:ff9b::/96`) or 6to4. fn embedded_ipv4(address: Ipv6Addr) -> Option { let segments = address.segments(); let (high, low) = if segments[..6] == [0, 0, 0, 0, 0, 0] || segments[..6] == [0x0064, 0xff9b, 0, 0, 0, 0] { @@ -202,49 +236,167 @@ fn embedded_ipv4(address: Ipv6Addr) -> Option { Some(Ipv4Addr::new(a, b, c, d)) } -fn classify_ipv4(address: Ipv4Addr) -> AddressClass { - let [a, b, _, _] = address.octets(); - if address.is_private() { - return AddressClass::Private; - } - if address.is_loopback() - || address.is_link_local() - || address.is_broadcast() - || address.is_documentation() - || address.is_multicast() - || address.is_unspecified() - || a == 0 - || (a == 100 && (64..=127).contains(&b)) - || (a == 198 && matches!(b, 18 | 19)) - || a >= 240 - { - return AddressClass::Blocked; - } - AddressClass::Public -} - -fn classify_ipv6(address: Ipv6Addr) -> AddressClass { - if let Some(address) = embedded_ipv4(address) { - return classify_ipv4(address); - } +fn classify_ipv6(address: Ipv6Addr) -> ResourceDestination { let segments = address.segments(); - if segments[0] & 0xfe00 == 0xfc00 { - // fc00::/7 unique local addresses. - return AddressClass::Private; - } + let embedded = address.to_ipv4_mapped().or_else(|| embedded_ipv4(address)); if address.is_loopback() || address.is_unspecified() || address.is_multicast() || segments[0] & 0xffc0 == 0xfe80 + || embedded.is_some_and(|mapped| classify_ipv4(mapped) == ResourceDestination::Blocked) + { + ResourceDestination::Blocked + } else if segments[0] & 0xfe00 == 0xfc00 || segments[0] & 0xffc0 == 0xfec0 || (segments[0] == 0x2001 && segments[1] == 0x0db8) + || embedded.is_some_and(|mapped| classify_ipv4(mapped) == ResourceDestination::Private) { - return AddressClass::Blocked; + ResourceDestination::Private + } else { + ResourceDestination::Public } - AddressClass::Public } -pub fn is_public_ip(address: IpAddr) -> bool { matches!(classify_ip(address), AddressClass::Public) } +/// Whether an address is publicly routable. +/// +/// Derived from [`classify_ip`] so classification and this answer cannot drift apart. +pub fn is_public_ip(address: IpAddr) -> bool { classify_ip(address) == ResourceDestination::Public } + +/// Strips the brackets of an IPv6 literal so that it parses as an address. +fn host_literal(host: &str) -> &str { host.strip_prefix('[').and_then(|value| value.strip_suffix(']')).unwrap_or(host) } + +/// How long a verdict that allows a URL to be handed to a client is remembered. +const RESOURCE_DESTINATION_PUBLIC_TTL: Duration = Duration::from_secs(60); +/// How long a verdict that keeps a destination away from clients is remembered. Both classes fail +/// closed, so they can be cached longer than the one that decides exposure. +const RESOURCE_DESTINATION_PRIVATE_TTL: Duration = Duration::from_secs(300); +/// How long a name that produced no answer within the resolution budget is remembered. +/// +/// The cause - a resolver that is slow, overloaded, or briefly unreachable - is transient, so such a +/// name must not be pinned to a non-public verdict for minutes. It stays non-public while it is +/// remembered, because nothing was proven. +const RESOURCE_DESTINATION_UNANSWERED_TTL: Duration = Duration::from_secs(10); +/// Upper bound for remembered verdicts, evicted least-recently-used. +const RESOURCE_DESTINATION_CACHE_CAPACITY: usize = 4096; +/// Resolution budget when classifying a destination given as a name. +const RESOURCE_DESTINATION_LOOKUP_TIMEOUT: Duration = Duration::from_millis(250); + +struct RememberedDestination { + verdict: ResourceDestination, + /// Whether the verdict came from a resolved answer. A name whose lookup produced none stays + /// non-public, but nothing is known about it either. + answered: bool, + expires_at: Instant, +} + +/// Memoizes destination verdicts so that rendering a playlist or EPG does not resolve the same host +/// per icon. +/// +/// The verdict is a property of the host, so one memo serves the whole process: every route and +/// every rendering path has to reach the same answer, and a per-request memo would put a DNS lookup +/// on the playlist path for each request instead of once per host. +/// +/// This is a hot-path memo, not policy state: the verdict decides whether a URL may be exposed, and +/// the resource client resolves and validates again while the connection is built, so a stale entry +/// can never open a connection to a local address. +struct DestinationCache { + destinations: Mutex>, +} + +impl DestinationCache { + fn new() -> Self { + Self { + destinations: Mutex::new(LruCache::new( + NonZeroUsize::new(RESOURCE_DESTINATION_CACHE_CAPACITY).unwrap_or(NonZeroUsize::MIN), + )), + } + } + + /// The process-wide memo. + fn shared() -> &'static Self { + static SHARED: OnceLock = OnceLock::new(); + SHARED.get_or_init(DestinationCache::new) + } + + fn verdict_and_answer(&self, host: &str) -> Option<(ResourceDestination, bool)> { + let mut destinations = self.destinations.lock().unwrap_or_else(PoisonError::into_inner); + match destinations.get(host) { + Some(entry) if entry.expires_at > Instant::now() => Some((entry.verdict, entry.answered)), + Some(_) => { + destinations.pop(host); + None + } + None => None, + } + } + + fn remember(&self, host: &str, verdict: ResourceDestination, answered: bool, ttl: Duration) { + let mut destinations = self.destinations.lock().unwrap_or_else(PoisonError::into_inner); + let entry = RememberedDestination { verdict, answered, expires_at: Instant::now() + ttl }; + destinations.put(host.to_owned(), entry); + } +} + +/// Classifies a resource destination given as host name or IP literal, bracketed IPv6 included. +/// +/// An IP literal is decided by its address alone, without resolving anything. Names are resolved once +/// per verdict lifetime and remembered in the process-wide memo. A name that cannot be resolved is +/// reported as [`ResourceDestination::Private`]: callers must not hand a possibly internal name to a +/// client, and the fetch attempt itself decides whether the destination is reachable. +pub async fn classify_resource_destination(host: &str) -> ResourceDestination { classify_host(host).await.0 } + +/// Classifies a destination and reports whether the verdict came from a resolved answer. +/// +/// The second value separates "proven to be local" from "nothing was proven": both keep a client away +/// from the destination, but only the first says the destination cannot be reached from outside. +pub(crate) async fn classify_host(host: &str) -> (ResourceDestination, bool) { + if let Ok(address) = host_literal(host).parse::() { + // An IP literal is decided by its address alone, so the answer is factual either way. + return (classify_ip(address), true); + } + let cache = DestinationCache::shared(); + if let Some(verdict) = cache.verdict_and_answer(host) { + return verdict; + } + + let mut resolved = Vec::new(); + if let Ok(Ok(addresses)) = timeout(RESOURCE_DESTINATION_LOOKUP_TIMEOUT, tokio::net::lookup_host((host, 0))).await { + resolved.extend(addresses.map(|address| address.ip())); + } + let (verdict, ttl) = verdict_for_addresses(&resolved); + let answered = !resolved.is_empty(); + cache.remember(host, verdict, answered, ttl); + (verdict, answered) +} + +/// Aggregates the addresses a name resolved to into a single verdict, and the time that verdict may be +/// remembered. +/// +/// One local-only address blocks the whole destination, and one private address keeps it non-public: +/// a client must not be able to reach the destination by any of its addresses. An empty answer is +/// non-public, because nothing was proven, and is remembered only briefly: unlike a resolved address, +/// it does not prove anything about the destination, so the name may well be reachable once the +/// resolver answers again. +fn verdict_for_addresses(addresses: &[IpAddr]) -> (ResourceDestination, Duration) { + if addresses.is_empty() { + return (ResourceDestination::Private, RESOURCE_DESTINATION_UNANSWERED_TTL); + } + let mut verdict = ResourceDestination::Public; + for address in addresses { + match classify_ip(*address) { + ResourceDestination::Blocked => { + return (ResourceDestination::Blocked, RESOURCE_DESTINATION_PRIVATE_TTL); + } + ResourceDestination::Private => verdict = ResourceDestination::Private, + ResourceDestination::Public => {} + } + } + let ttl = match verdict { + ResourceDestination::Public => RESOURCE_DESTINATION_PUBLIC_TTL, + ResourceDestination::Private | ResourceDestination::Blocked => RESOURCE_DESTINATION_PRIVATE_TTL, + }; + (verdict, ttl) +} /// Options applied at the final boundary of every physical request attempt. #[derive(Debug, Clone, Copy, Default)] @@ -3107,84 +3259,6 @@ pub fn create_tmdb_client(cfg: &AppConfig) -> Result, mode: ResourceRedirectMode) -> Policy { - match mode { - ResourceRedirectMode::NoRedirect => Policy::none(), - ResourceRedirectMode::Bounded => Policy::custom(move |attempt| { - if attempt.previous().len() >= RESOURCE_REDIRECT_LIMIT { - return attempt.error(ResourcePolicyError::TooManyRedirects); - } - match attempt.url().host() { - None => attempt.error(ResourcePolicyError::MissingHost), - Some(Host::Ipv4(address)) => match policy.authorize_literal(IpAddr::V4(address)) { - Ok(()) => attempt.follow(), - Err(err) => attempt.error(err), - }, - Some(Host::Ipv6(address)) => match policy.authorize_literal(IpAddr::V6(address)) { - Ok(()) => attempt.follow(), - Err(err) => attempt.error(err), - }, - Some(Host::Domain(_)) => attempt.follow(), - } - }), - } -} - -/// Warns once per client build that resource-proxy fetches ignore the configured proxy. -/// -/// A proxy resolves names on its own side, which would bypass connection-time destination -/// validation, so every policy-checked resource fetch connects directly. This applies to -/// public-only resources as well, which is why it is not tied to a `resource_policy` being set. -pub fn warn_resource_proxy_bypass(config: &Config) { - let proxy_configured = config.proxy.is_some(); - let env_proxy_configured = ["HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "http_proxy", "https_proxy", "all_proxy"] - .iter() - .any(|key| std::env::var_os(key).is_some_and(|value| !value.is_empty())); - - if proxy_configured { - warn!( - "A proxy is configured, but server-side resource-proxy fetches connect directly so \ - destination policy can be enforced at connection time" - ); - } - if env_proxy_configured { - warn!( - "HTTP_PROXY/HTTPS_PROXY/ALL_PROXY are set, but server-side resource-proxy fetches \ - ignore environment proxies so destination policy can be enforced at connection time" - ); - } -} - -/// Builds a client for one normalized policy and redirect mode. -/// -/// Resource clients always connect directly: the policy resolver must see the real destination -/// address, and a proxy would resolve it elsewhere. -pub fn create_resource_http_client( - cfg: &AppConfig, - policy: Arc, - mode: ResourceRedirectMode, -) -> Result { - let config = cfg.config.load(); - let redirect_policy = resource_redirect_policy(Arc::clone(&policy), mode); - let mut builder = create_client_with_redirect(cfg, redirect_policy) - .no_proxy() - .dns_resolver(PolicyIpResolver::new(policy)) - .http1_only(); - if config.connect_timeout_secs > 0 { - builder = builder.connect_timeout(Duration::from_secs(u64::from(config.connect_timeout_secs))); - } - builder.build().map_err(|err| TuliproxError::Config(format!("Failed to create resource HTTP client: {err}"))) -} - pub fn parse_range(range: &str) -> Option<(u64, Option)> { // expect: "bytes=START-END" if !range.starts_with("bytes=") { @@ -3245,16 +3319,16 @@ mod tests { mod tmdb_profile; use super::{ - classify_ip, download_text_content, download_text_content_with_headers_and_options, + classify_resource_destination, download_text_content, download_text_content_with_headers_and_options, get_input_epg_content_as_file, get_remote_content_as_stream, is_safe_cross_origin_redirect_header, next_provider_url_index, preview_request_diagnostics_for_logging, preview_request_target_for_logging, - resolve_attempt_target, same_origin, + resolve_attempt_target, resolve_resource_socket_addrs, same_origin, send_input_with_retry_and_provider_policy_with_manual_redirects_and_options_result, send_input_with_retry_and_provider_policy_with_options_result, send_with_retry_and_provider, send_with_retry_and_provider_policy, should_retry_text_body_error, should_try_next_ip_on_connect_error, - strip_sensitive_headers_for_cross_origin_redirect, text_response_error_log_label, AddressClass, - InputEpgFileRequest, PublicIpResolver, RequestFetchOptions, TextContentBodyOptions, TextContentFetchOptions, - STREAM_IDLE_TIMEOUT, + strip_sensitive_headers_for_cross_origin_redirect, text_response_error_log_label, InputEpgFileRequest, + PublicIpResolver, RequestFetchOptions, ResourceDestination, ResourceDestinationResolver, + TextContentBodyOptions, TextContentFetchOptions, STREAM_IDLE_TIMEOUT, }; use crate::{ model::{ @@ -3299,31 +3373,145 @@ mod tests { use url::Url; #[test] - fn embedded_ipv4_destinations_use_ipv4_classification() { - for (address, expected) in [ - ("64:ff9b::a00:1", AddressClass::Private), - ("64:ff9b::808:808", AddressClass::Public), - ("2002:a00:1::", AddressClass::Private), - ("2002:808:808::", AddressClass::Public), - ("::a00:1", AddressClass::Private), - ("::808:808", AddressClass::Public), - ("64:ff9b::7f00:1", AddressClass::Blocked), - ] { - assert_eq!(classify_ip(address.parse::().expect("valid IP address")), expected, "{address}"); + fn ipv6_addresses_embedding_private_ipv4_are_not_public() { + for address in ["::10.0.0.1", "64:ff9b::a00:1", "2002:0a00:0001::", "::ffff:10.0.0.1"] { + assert!(address.parse::().is_ok_and(|ip| !super::is_public_ip(ip)), "{address}"); } } #[test] - fn native_ipv6_destinations_keep_ipv6_classification() { + fn resource_destinations_are_classified_by_reachability() { for (address, expected) in [ - ("2001:4860:4860::8888", AddressClass::Public), - ("fc00::1", AddressClass::Private), - ("2001:db8::1", AddressClass::Blocked), + ("1.1.1.1", ResourceDestination::Public), + ("8.8.8.8", ResourceDestination::Public), + ("64:ff9b::808:808", ResourceDestination::Public), + ("2002:808:808::", ResourceDestination::Public), + ("::808:808", ResourceDestination::Public), + ("10.0.0.1", ResourceDestination::Private), + ("172.16.5.5", ResourceDestination::Private), + ("192.168.1.20", ResourceDestination::Private), + ("100.64.0.1", ResourceDestination::Private), + ("fc00::1", ResourceDestination::Private), + ("64:ff9b::a00:1", ResourceDestination::Private), + ("2002:0a00:0001::", ResourceDestination::Private), + ("::a00:1", ResourceDestination::Private), + ("::ffff:10.0.0.1", ResourceDestination::Private), + ("0.0.0.0", ResourceDestination::Blocked), + ("127.0.0.1", ResourceDestination::Blocked), + ("169.254.1.1", ResourceDestination::Blocked), + ("169.254.169.254", ResourceDestination::Blocked), + ("255.255.255.255", ResourceDestination::Blocked), + ("224.0.0.1", ResourceDestination::Blocked), + ("::", ResourceDestination::Blocked), + ("::1", ResourceDestination::Blocked), + ("fe80::1", ResourceDestination::Blocked), + ("ff02::1", ResourceDestination::Blocked), + ("::ffff:127.0.0.1", ResourceDestination::Blocked), + ("64:ff9b::7f00:1", ResourceDestination::Blocked), + ("2002:7f00:1::", ResourceDestination::Blocked), ] { - assert_eq!(classify_ip(address.parse::().expect("valid IP address")), expected, "{address}"); + let address = address.parse::().expect("valid IP address"); + assert_eq!(super::classify_ip(address), expected, "{address}"); } } + #[tokio::test] + async fn resource_destination_names_are_classified_without_resolving_ip_literals() { + assert_eq!(classify_resource_destination("192.168.1.20").await, ResourceDestination::Private); + assert_eq!(classify_resource_destination("[::1]").await, ResourceDestination::Blocked); + assert_eq!(classify_resource_destination("[64:ff9b::7f00:1]").await, ResourceDestination::Blocked); + assert_eq!(classify_resource_destination("8.8.8.8").await, ResourceDestination::Public); + } + + #[tokio::test] + async fn resource_destination_resolution_refuses_addresses_local_to_this_host() { + let private = resolve_resource_socket_addrs("10.0.0.1", 80).await.expect("private destination is allowed"); + assert_eq!(private.len(), 1); + + for local_only in ["127.0.0.1", "169.254.169.254", "[::1]", "fe80::1", "0.0.0.0"] { + let error = resolve_resource_socket_addrs(local_only, 80) + .await + .expect_err("local-only destination must be refused"); + assert_eq!(error.kind(), ErrorKind::PermissionDenied, "{local_only}"); + } + } + + #[test] + fn proxy_hosts_are_exempt_from_the_connect_time_local_address_guard() { + use crate::{model::Config, utils::request::proxy_hosts}; + + // `localhost` is what a proxy on this host is usually configured as, and reqwest asks the + // resolver for the proxy host as well: guarding it would break every fetch through such a proxy. + let app_config = make_test_app_config(Config { + proxy: Some(crate::model::ProxyConfig { + url: "http://localhost:8118".to_string(), + username: None, + password: None, + }), + ..Config::default() + }); + + assert_eq!(proxy_hosts(&app_config), vec![Arc::from("localhost")]); + } + + #[test] + fn resolved_address_sets_are_aggregated_for_exposure() { + use super::{ + verdict_for_addresses, RESOURCE_DESTINATION_PRIVATE_TTL, RESOURCE_DESTINATION_PUBLIC_TTL, + RESOURCE_DESTINATION_UNANSWERED_TTL, + }; + + let public: IpAddr = "8.8.8.8".parse().expect("valid public address"); + let private: IpAddr = "10.0.0.1".parse().expect("valid private address"); + let blocked: IpAddr = "127.0.0.1".parse().expect("valid loopback address"); + + assert_eq!(verdict_for_addresses(&[public]), (ResourceDestination::Public, RESOURCE_DESTINATION_PUBLIC_TTL)); + assert_eq!( + verdict_for_addresses(&[public, public]), + (ResourceDestination::Public, RESOURCE_DESTINATION_PUBLIC_TTL) + ); + assert_eq!(verdict_for_addresses(&[private]), (ResourceDestination::Private, RESOURCE_DESTINATION_PRIVATE_TTL)); + assert_eq!( + verdict_for_addresses(&[public, private]), + (ResourceDestination::Private, RESOURCE_DESTINATION_PRIVATE_TTL) + ); + assert_eq!( + verdict_for_addresses(&[public, blocked]), + (ResourceDestination::Blocked, RESOURCE_DESTINATION_PRIVATE_TTL) + ); + assert_eq!( + verdict_for_addresses(&[private, blocked]), + (ResourceDestination::Blocked, RESOURCE_DESTINATION_PRIVATE_TTL) + ); + // An unanswered name stays non-public, but only briefly: a resolver hiccup must not pin the + // destination to the proxy for minutes. + assert_eq!(verdict_for_addresses(&[]), (ResourceDestination::Private, RESOURCE_DESTINATION_UNANSWERED_TTL)); + assert!(RESOURCE_DESTINATION_UNANSWERED_TTL < RESOURCE_DESTINATION_PRIVATE_TTL); + } + + #[tokio::test] + async fn resource_destination_resolver_refuses_local_only_names() { + use reqwest::dns::Resolve; + use std::str::FromStr; + + let resolver = ResourceDestinationResolver::default(); + let name = reqwest::dns::Name::from_str("127.0.0.1").expect("valid destination name"); + let result = resolver.resolve(name).await; + + match result { + Ok(_) => panic!("local-only destination must be refused"), + Err(error) => assert!(error.to_string().contains("local to this host"), "{error}"), + } + + // Only the proxy hosts are exempt, so a proxy on the loopback interface stays usable while a + // destination that resolves to the loopback interface is still refused. + let resolver = ResourceDestinationResolver { allowed_hosts: vec![Arc::from("localhost")].into() }; + let name = reqwest::dns::Name::from_str("localhost").expect("valid proxy name"); + assert!(resolver.resolve(name).await.is_ok(), "the configured proxy host must resolve"); + let name = reqwest::dns::Name::from_str("127.0.0.1").expect("valid destination name"); + assert!(resolver.resolve(name).await.is_err(), "a destination may not use the exemption"); + } + fn make_test_app_config(config: Config) -> Arc { Arc::new(AppConfig { config: Arc::new(ArcSwap::from_pointee(config)), diff --git a/backend/core/src/utils/resource_output.rs b/backend/core/src/utils/resource_output.rs new file mode 100644 index 000000000..2e4ab8578 --- /dev/null +++ b/backend/core/src/utils/resource_output.rs @@ -0,0 +1,295 @@ +use crate::utils::request::{classify_host, ResourceDestination}; +use futures::{stream, StreamExt}; +use shared::model::{ResourceOutputPolicy, XtreamMappingFlags, XtreamMappingOptions, XtreamPlaylistItem}; +use std::borrow::Cow; +use url::Url; + +/// How many destinations of one item are classified at the same time. +/// +/// An item can carry a whole season list of icons, and a lookup has its own budget, so classifying one +/// host after the other would multiply that budget by the field count. +const RESOURCE_HOST_CLASSIFY_CONCURRENCY: usize = 8; + +/// Host of a resource URL without parsing it, for the shape provider data actually uses. +/// +/// Returns `None` for anything a plain scan could read differently than a full parse would - user +/// info, an uppercase label, a percent escape, a backslash - so those go through [`Url::parse`]. +/// Callers must treat `None` as "no host found", never as "no destination": the fallback needs to run +/// for them to get an answer. A host that the fast path reads but the parser would read differently can +/// only cause a cache miss, and a miss fails closed to a proxy link. +fn plain_resource_host(resource_url: &str) -> Option<&str> { + let rest = resource_url.strip_prefix("http://").or_else(|| resource_url.strip_prefix("https://"))?; + let authority = rest.split(['/', '?', '#']).next()?; + // An IPv6 literal goes through the parser: it is written with the brackets a consumer of the host + // expects, and the parser normalizes its spelling. + if authority.is_empty() || authority.starts_with('[') { + return None; + } + // A colon separates the port, and an authority with two of them is not a plain host. + let (host, port) = authority.split_once(':').map_or((authority, ""), |(host, port)| (host, port)); + let is_label_byte = |byte: u8| byte.is_ascii_lowercase() || byte.is_ascii_digit() || matches!(byte, b'.' | b'-'); + if host.is_empty() + || host.contains(':') + || !host.bytes().all(is_label_byte) + || !port.bytes().all(|byte| byte.is_ascii_digit()) + { + return None; + } + Some(host) +} + +/// Host of a resource URL that a client could fetch, if it has one. +fn resource_host(resource_url: &str) -> Option> { + if let Some(host) = plain_resource_host(resource_url) { + return Some(Cow::Borrowed(host)); + } + let url = Url::parse(resource_url).ok()?; + matches!(url.scheme(), "http" | "https").then(|| url.host_str().map(|host| Cow::Owned(host.to_owned())))? +} + +/// Where a hop of a resource fetch must be routed. +#[derive(Debug, Clone, Copy, Eq, PartialEq)] +pub enum ResourceHop { + /// The destination can leave the local network: fetched through the client that honours the + /// configured proxy, so the request cannot disclose the address of this host. + ViaProxy, + /// The destination is only reachable from here: fetched by connecting directly, because a proxy has + /// no route to it and its address is validated while the connection is built. + Direct, + /// The destination is local to this host: refused, because the fetch would turn this instance into + /// a reader for itself. + Blocked, +} + +/// Routes a hop of a resource fetch. +/// +/// [`classify_output_resource_url`] answers what a client may be handed; this answers who fetches it. +/// The class of the destination decides, with one case decided by what is *not* known: a name whose +/// lookup produced no answer is neither reachable directly nor provably public. The configured proxy is +/// then the only egress that can still resolve it, so it is used when there is one - which also keeps +/// this host's address out of the request. Without a proxy nothing changes: the direct client tries and +/// fails, as it would have anyway. +pub async fn classify_resource_hop(resource_url: &str, proxy_configured: bool) -> ResourceHop { + let Some(host) = resource_host(resource_url) else { + return ResourceHop::Blocked; + }; + match classify_host(&host).await { + (ResourceDestination::Public, _) => ResourceHop::ViaProxy, + (ResourceDestination::Blocked, _) => ResourceHop::Blocked, + // A destination that is not provably public connects directly: that is what keeps a + // self-hosted server on the local network reachable, and a name that resolved to a private + // address is exactly that. Only a name that produced no answer at all is fetched through the + // configured proxy, because the direct client cannot resolve it either. + (ResourceDestination::Private, answered) => { + if answered || !proxy_configured { + ResourceHop::Direct + } else { + ResourceHop::ViaProxy + } + } + } +} + +/// Classifies a resource URL that is about to be written into player output. +/// +/// This decides what a client may reach, so the URL is read with the same parser a client uses rather +/// than with the cheaper scan [`resource_host`] offers for the hint path. +pub async fn classify_output_resource_url(resource_url: &str) -> ResourceOutputPolicy { + if resource_url.starts_with("/api/v1/library/thumbnail/") { + return ResourceOutputPolicy::Direct; + } + if resource_url.starts_with("media-server://image/") { + return ResourceOutputPolicy::Proxy; + } + let Ok(url) = Url::parse(resource_url) else { + return ResourceOutputPolicy::Blocked; + }; + if !matches!(url.scheme(), "http" | "https") { + return ResourceOutputPolicy::Blocked; + } + let Some(host) = url.host_str() else { + return ResourceOutputPolicy::Blocked; + }; + classify_resource_host(host).await +} + +/// Output policy of a single destination host. +async fn classify_resource_host(host: &str) -> ResourceOutputPolicy { + match crate::utils::request::classify_resource_destination(host).await { + ResourceDestination::Public => ResourceOutputPolicy::Direct, + ResourceDestination::Private => ResourceOutputPolicy::Proxy, + ResourceDestination::Blocked => ResourceOutputPolicy::Blocked, + } +} + +/// Records the output policy of every destination this item carries, so that rendering it needs no +/// name resolution. +/// +/// Xtream output is written field by field (`to_document`), which has no place to await a lookup. +/// The item's destinations are collected and classified once here; a field for a destination that is +/// not in the map fails closed to a proxy link. +pub async fn prepare_xtream_resource_hosts(item: &XtreamPlaylistItem, options: &XtreamMappingOptions) { + if options.web_ui_request || options.flags.contains(XtreamMappingFlags::RewriteResourceUrl) { + return; + } + // The map is the dedupe: a host classified by an earlier item is skipped, so a warm map costs no + // allocation per item. Only hosts that are new to this item are collected, and the host is read + // without parsing the URL, because this runs for every resource field of every item. + let mut unresolved: Vec = Vec::new(); + item.visit_resource_urls(|resource_url| { + let Some(host) = resource_host(resource_url) else { + return; + }; + if options.resource_host_policies.contains_key(host.as_ref()) + || unresolved.iter().any(|known| known == host.as_ref()) + { + return; + } + unresolved.push(host.into_owned()); + }); + let classified: Vec<_> = stream::iter(unresolved) + .map(|host| async move { + let policy = classify_resource_host(&host).await; + (host, policy) + }) + .buffered(RESOURCE_HOST_CLASSIFY_CONCURRENCY) + .collect() + .await; + for (host, policy) in classified { + options.resource_host_policies.insert(host, policy); + } +} + +#[cfg(test)] +mod tests { + use super::{classify_output_resource_url, prepare_xtream_resource_hosts}; + use shared::{ + model::{ + PlaylistItemType, PlaylistItemTypeSet, ResourceOutputPolicy, VirtualId, XtreamCluster, + XtreamMappingFlagsSet, XtreamMappingOptions, XtreamPlaylistItem, + }, + utils::Internable, + }; + use std::sync::Arc; + + #[tokio::test] + async fn an_unanswered_name_is_routed_through_a_configured_proxy() { + use super::{classify_resource_hop, ResourceHop}; + + // `.invalid` is reserved and never resolves: the destination is neither reachable directly nor + // provably local, so a configured proxy is the only egress that can still resolve it. + assert_eq!(classify_resource_hop("http://unresolved.invalid/logo.png", false).await, ResourceHop::Direct); + assert_eq!(classify_resource_hop("http://unresolved.invalid/logo.png", true).await, ResourceHop::ViaProxy); + // A destination on a private network is reachable directly and not through a proxy, so it stays + // direct however a proxy is configured. + assert_eq!(classify_resource_hop("http://192.168.1.20/logo.png", true).await, ResourceHop::Direct); + // A local destination is refused, and a public one always uses the client that honours the proxy. + assert_eq!(classify_resource_hop("http://127.0.0.1/logo.png", true).await, ResourceHop::Blocked); + assert_eq!(classify_resource_hop("http://8.8.8.8/logo.png", true).await, ResourceHop::ViaProxy); + assert_eq!(classify_resource_hop("media-server://image/plex/server/item", true).await, ResourceHop::Blocked); + } + + #[test] + fn resource_host_reads_the_same_host_a_parser_reads() { + use super::{plain_resource_host, resource_host}; + use url::Url; + + // Every accepted URL either agrees with the parser, or the fast path declines and the parser + // answers. Both directions are checked, so the shortcut can never decide a different host. + let urls = [ + "http://cdn.example.com/logo.png", + "https://cdn.example.com:8443/logo.png", + "http://192.168.1.20/logo.png", + "http://[2001:db8::1]:8080/logo.png", + "http://example.com", + "http://example.com?x=1", + "http://example.com#fragment", + "http://user:pass@example.com/logo.png", + "http://Example.COM/logo.png", + "http://evil.example\\@trusted.example/logo.png", + "http://ex%41mple.com/logo.png", + "http://example.com:abc/logo.png", + "http://a:b:80/logo.png", + "http:///logo.png", + "http://.", + "/relative/path/logo.png", + "media-server://image/plex/server/item", + "ftp://example.com/logo.png", + "data:image/png;base64,AAAA", + ]; + for url in urls { + let parsed = Url::parse(url).ok().filter(|parsed| matches!(parsed.scheme(), "http" | "https")); + let expected = parsed.as_ref().and_then(Url::host_str); + match plain_resource_host(url) { + Some(host) => assert_eq!(Some(host), expected, "fast path must agree for {url}"), + None => assert_eq!(resource_host(url).as_deref(), expected, "fallback must answer for {url}"), + } + } + } + + #[tokio::test] + async fn private_resources_require_a_proxy_even_when_public_resources_do_not() { + assert_eq!(classify_output_resource_url("http://192.168.1.20/logo.png").await, ResourceOutputPolicy::Proxy); + assert_eq!(classify_output_resource_url("http://8.8.8.8/logo.png").await, ResourceOutputPolicy::Direct); + assert_eq!(classify_output_resource_url("http://127.0.0.1/logo.png").await, ResourceOutputPolicy::Blocked); + assert_eq!( + classify_output_resource_url("media-server://image/plex/server/item").await, + ResourceOutputPolicy::Proxy + ); + } + + #[tokio::test] + async fn xtream_output_prepares_public_and_private_resource_hosts() { + let item = XtreamPlaylistItem { + virtual_id: VirtualId::new(41), + provider_id: 41, + name: "channel".intern(), + logo: "http://192.168.1.20/logo.png".intern(), + logo_small: "http://8.8.8.8/logo.png".intern(), + group: "group".intern(), + title: "channel".intern(), + parent_code: "".intern(), + rec: "".intern(), + url: "http://provider.example/live/41.ts".intern(), + epg_channel_id: None, + xtream_cluster: XtreamCluster::Live, + additional_properties: None, + item_type: PlaylistItemType::Live, + category_id: 1, + input_name: "input".intern(), + channel_no: 0, + source_ordinal: 0, + input_stream_id: "41".intern(), + upstream_user_agent: None, + }; + let options = XtreamMappingOptions { + flags: XtreamMappingFlagsSet::new(), + force_redirect: None, + reverse_item_types: PlaylistItemTypeSet::empty(), + resource_proxy_item_types: PlaylistItemTypeSet::from_item(PlaylistItemType::Live), + username: "user".to_string(), + password: "pass".to_string(), + base_url: "https://proxy.example".to_string(), + web_ui_request: false, + encrypt_secret: [0; 16], + resource_host_policies: Arc::default(), + }; + + prepare_xtream_resource_hosts(&item, &options).await; + + assert_eq!( + options.get_resource_url(XtreamCluster::Live, PlaylistItemType::Live, item.virtual_id, &item.logo, "logo",), + "https://proxy.example/resource/live/user/pass/41/logo" + ); + assert_eq!( + options.get_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + item.virtual_id, + &item.logo_small, + "logo_small", + ), + "http://8.8.8.8/logo.png" + ); + } +} diff --git a/backend/core/src/utils/resource_token.rs b/backend/core/src/utils/resource_token.rs deleted file mode 100644 index 8d5ef72d8..000000000 --- a/backend/core/src/utils/resource_token.rs +++ /dev/null @@ -1,135 +0,0 @@ -//! Encoding and decoding of authenticated resource links. -//! -//! Two legacy encodings predate this module and are neither confidential nor authenticated: -//! `/resource/epg/...` uses `obscure_text`, the Web UI resource route uses `obfuscate_text`. -//! Both carry a bare URL, so a token cannot be traced back to the input that supplied the URL and -//! every legacy link is treated as public-only. New links use the payload defined here, which -//! carries the origin and is protected by the existing authenticated helper. - -use crate::utils::crypto_utils::{deobscure_authenticated_bytes, obscure_authenticated_bytes}; -use shared::model::ResourceToken; -use std::fmt; - -/// Prefix that discriminates a new token from a legacy encoding. It marks the token type, not a -/// payload version: the payload version stays in the authenticated envelope. -pub const RESOURCE_TOKEN_PREFIX: &str = "a1_"; - -const RESOURCE_TOKEN_DOMAIN: &[u8] = b"tuliprox.resource-token.v1"; - -/// Largest accepted token. Checked against the raw string before base64 decoding, so an oversized -/// token is rejected before it allocates or deserializes anything. -pub const MAX_RESOURCE_TOKEN_BYTES: usize = 16384; - -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum ResourceTokenError { - TooLarge, - MissingPrefix, - Invalid, -} - -impl fmt::Display for ResourceTokenError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::TooLarge => f.write_str("resource token exceeds the size limit"), - Self::MissingPrefix => f.write_str("resource token has no token prefix"), - Self::Invalid => f.write_str("resource token is malformed or not authenticated"), - } - } -} - -impl std::error::Error for ResourceTokenError {} - -/// True when the encoded value is a new authenticated token rather than a legacy encoding. -pub fn has_resource_token_prefix(encoded: &str) -> bool { encoded.starts_with(RESOURCE_TOKEN_PREFIX) } - -/// Encodes a token, or reports that the payload does not fit the accepted size. -pub fn encode_resource_token(secret: &[u8; 16], token: &ResourceToken) -> Result { - let payload = rmp_serde::to_vec(token).map_err(|_| ResourceTokenError::Invalid)?; - let encoded = obscure_authenticated_bytes(secret, RESOURCE_TOKEN_DOMAIN, &payload) - .map_err(|_| ResourceTokenError::Invalid)?; - if encoded.len() + RESOURCE_TOKEN_PREFIX.len() > MAX_RESOURCE_TOKEN_BYTES { - return Err(ResourceTokenError::TooLarge); - } - Ok(format!("{RESOURCE_TOKEN_PREFIX}{encoded}")) -} - -pub fn decode_resource_token(secret: &[u8; 16], encoded: &str) -> Result { - if encoded.len() > MAX_RESOURCE_TOKEN_BYTES { - return Err(ResourceTokenError::TooLarge); - } - let payload = encoded.strip_prefix(RESOURCE_TOKEN_PREFIX).ok_or(ResourceTokenError::MissingPrefix)?; - if payload.is_empty() { - return Err(ResourceTokenError::Invalid); - } - let bytes = deobscure_authenticated_bytes(secret, RESOURCE_TOKEN_DOMAIN, payload) - .map_err(|_| ResourceTokenError::Invalid)?; - rmp_serde::from_slice(&bytes).map_err(|_| ResourceTokenError::Invalid) -} - -pub fn resource_token(resource: &str) -> ResourceToken { ResourceToken { resource: resource.to_string() } } - -#[cfg(test)] -mod tests { - use super::{ - decode_resource_token, encode_resource_token, has_resource_token_prefix, resource_token, ResourceTokenError, - MAX_RESOURCE_TOKEN_BYTES, RESOURCE_TOKEN_PREFIX, - }; - use rand::Rng; - - fn random_secret() -> [u8; 16] { rand::rng().random() } - - #[test] - fn token_round_trip_keeps_url_and_origin() { - let secret = random_secret(); - let token = resource_token("resource://v1/example"); - let encoded = encode_resource_token(&secret, &token).expect("encode"); - - assert!(has_resource_token_prefix(&encoded)); - assert_eq!(decode_resource_token(&secret, &encoded).expect("decode"), token); - } - - #[test] - fn raw_public_resource_round_trips() { - let secret = random_secret(); - let token = resource_token("https://cdn.example.com/logo.png"); - let encoded = encode_resource_token(&secret, &token).expect("encode"); - let decoded = decode_resource_token(&secret, &encoded).expect("decode"); - - assert_eq!(decoded.resource, "https://cdn.example.com/logo.png"); - } - - #[test] - fn tampered_or_foreign_tokens_are_rejected() { - let secret = random_secret(); - let token = resource_token("https://cdn.example.com/logo.png"); - let encoded = encode_resource_token(&secret, &token).expect("encode"); - - let mut tampered = encoded.clone(); - let last = tampered.pop().expect("token char"); - tampered.push(if last == 'A' { 'B' } else { 'A' }); - assert_eq!(decode_resource_token(&secret, &tampered), Err(ResourceTokenError::Invalid)); - - // A different secret must not authenticate the same payload. - assert_eq!(decode_resource_token(&random_secret(), &encoded), Err(ResourceTokenError::Invalid)); - - // A legacy encoding has no prefix and is never decoded as a token. - let legacy = shared::utils::obfuscate_text(&secret, "https://cdn.example.com/logo.png"); - assert_eq!(decode_resource_token(&secret, &legacy), Err(ResourceTokenError::MissingPrefix)); - } - - #[test] - fn oversized_tokens_are_rejected_before_decoding() { - let secret = random_secret(); - let oversized = format!("{RESOURCE_TOKEN_PREFIX}{}", "A".repeat(MAX_RESOURCE_TOKEN_BYTES)); - assert_eq!(decode_resource_token(&secret, &oversized), Err(ResourceTokenError::TooLarge)); - - let long_url = format!("https://cdn.example.com/{}", "a".repeat(MAX_RESOURCE_TOKEN_BYTES)); - assert_eq!(encode_resource_token(&secret, &resource_token(&long_url)), Err(ResourceTokenError::TooLarge)); - } - - #[test] - fn empty_payload_is_rejected() { - let secret = random_secret(); - assert_eq!(decode_resource_token(&secret, RESOURCE_TOKEN_PREFIX), Err(ResourceTokenError::Invalid)); - } -} diff --git a/backend/iptv/src/xtream/mod.rs b/backend/iptv/src/xtream/mod.rs index 2347f3c6e..09ddc1958 100644 --- a/backend/iptv/src/xtream/mod.rs +++ b/backend/iptv/src/xtream/mod.rs @@ -22,7 +22,7 @@ use tuliprox_core::{ ClusterForceUpdate, ConfigInput, ConfigInputFlags, ConfigTarget, InputSource, ProxyUserCredentials, XtreamTargetOutput, }, - utils::request, + utils::{prepare_xtream_resource_hosts, request}, }; use tuliprox_parser::{xtream, xtream::parse_xtream_series_info}; use tuliprox_repository::{ @@ -87,6 +87,7 @@ pub async fn get_xtream_stream_info( let app_config = &app_config; let encrypt_secret = app_config.get_encrypt_secret(); let options = xtream_mapping_option_from_target_options(target, xtream_output, app_config, user, encrypt_secret)?; + prepare_xtream_resource_hosts(pli, &options).await; if let Some(content) = pli.get_resolved_info_document(&options) { return serde_json::to_string(&content).map_err(|err| TuliproxError::ApiXtream(format!("{err}"))); @@ -148,7 +149,9 @@ pub async fn get_xtream_stream_info( target, xtream_output, &vod_pli, - ) { + ) + .await + { return value; } } @@ -293,7 +296,9 @@ pub async fn get_xtream_stream_info( target, xtream_output, &series_pli, - ) { + ) + .await + { return value; } } @@ -318,7 +323,7 @@ pub async fn get_xtream_stream_info( ))) } -fn xtream_resolve_stream_info( +async fn xtream_resolve_stream_info( app_config: &Arc, _playlists: &PlaylistStorageState, user: &ProxyUserCredentials, @@ -333,6 +338,7 @@ fn xtream_resolve_stream_info( Ok(options) => options, Err(err) => return Some(Err(err)), }; + prepare_xtream_resource_hosts(pli, &options).await; if let Some(content) = pli.get_resolved_info_document(&options) { return Some( serde_json::to_string(&content) diff --git a/backend/metadata/src/manager.rs b/backend/metadata/src/manager.rs index 978e19a08..23e2e9bc8 100644 --- a/backend/metadata/src/manager.rs +++ b/backend/metadata/src/manager.rs @@ -2589,36 +2589,6 @@ impl InputWorker { false } - fn ingest_vod_resources(updates: &mut [(ProviderIdType, VideoStreamProperties)], input_name: &Arc) { - for (_, props) in updates { - let mut value = StreamProperties::Video(Box::new(std::mem::take(props))); - value.normalize_internal_resource_values(input_name); - if let StreamProperties::Video(normalized) = value { - *props = *normalized; - } - } - } - - fn ingest_series_resources(updates: &mut [(ProviderIdType, SeriesStreamProperties)], input_name: &Arc) { - for (_, props) in updates { - let mut value = StreamProperties::Series(Box::new(std::mem::take(props))); - value.normalize_internal_resource_values(input_name); - if let StreamProperties::Series(normalized) = value { - *props = *normalized; - } - } - } - - fn ingest_live_resources(updates: &mut [(ProviderIdType, LiveStreamProperties)], input_name: &Arc) { - for (_, props) in updates { - let mut value = StreamProperties::Live(Box::new(std::mem::take(props))); - value.normalize_internal_resource_values(input_name); - if let StreamProperties::Live(normalized) = value { - *props = *normalized; - } - } - } - async fn flush_batch_static( input_name: &str, bound_ctx: Option<&MetadataUpdateCtx>, @@ -2631,13 +2601,9 @@ impl InputWorker { let Some(ctx) = bound_ctx else { return }; let app_config = &ctx.app_config; let cfg = app_config.config.load(); - let mut vod_updates = batch_buffer.take_vod_updates(); - let mut series_updates = batch_buffer.take_series_updates(); - let mut live_updates = batch_buffer.take_live_updates(); - let input_name_arc: Arc = Arc::from(input_name); - Self::ingest_vod_resources(&mut vod_updates, &input_name_arc); - Self::ingest_series_resources(&mut series_updates, &input_name_arc); - Self::ingest_live_resources(&mut live_updates, &input_name_arc); + let vod_updates = batch_buffer.take_vod_updates(); + let series_updates = batch_buffer.take_series_updates(); + let live_updates = batch_buffer.take_live_updates(); if vod_updates.is_empty() && series_updates.is_empty() && live_updates.is_empty() { return; diff --git a/backend/parser/src/hls/mod.rs b/backend/parser/src/hls/mod.rs index a96de68af..8d060fe28 100644 --- a/backend/parser/src/hls/mod.rs +++ b/backend/parser/src/hls/mod.rs @@ -1,7 +1,7 @@ use shared::{ concat_string, defaults::{HLS_EXT, HLS_PREFIX}, - utils::{deobfuscate_text, extract_extension_from_url, obfuscate_text, CONSTANTS}, + utils::{extract_extension_from_url, open_hls_resource_url, seal_hls_resource_url, CONSTANTS}, }; use std::{borrow::Cow, str}; use tuliprox_core::model::ProxyUserCredentials; @@ -12,16 +12,16 @@ pub mod origin_manifest; const TOKEN_SEPARATOR: char = '\x1F'; const TOKEN_SEPARATOR_STR: &str = "\x1F"; -fn create_hls_session_token_and_url(secret: &[u8], session_token: &str, stream_url: &str) -> String { - let cookie_value = obfuscate_text(secret, &concat_string!(session_token, TOKEN_SEPARATOR_STR, stream_url)); +fn create_hls_session_token_and_url(secret: &[u8; 16], session_token: &str, stream_url: &str) -> String { + let cookie_value = seal_hls_resource_url(secret, &concat_string!(session_token, TOKEN_SEPARATOR_STR, stream_url)); if let Some(ext) = extract_extension_from_url(stream_url) { return concat_string!(&cookie_value, ext); } cookie_value } -fn create_hls_url_without_session_token(secret: &[u8], stream_url: &str) -> String { - let token = obfuscate_text(secret, stream_url); +fn create_hls_url_without_session_token(secret: &[u8; 16], stream_url: &str) -> String { + let token = seal_hls_resource_url(secret, stream_url); if let Some(ext) = extract_extension_from_url(stream_url) { return concat_string!(&token, ext); } @@ -34,8 +34,8 @@ fn remove_any_ext(s: &str) -> &str { None => s, } } -pub fn get_hls_session_token_and_url_from_token(secret: &[u8], token: &str) -> Option<(Option, String)> { - if let Ok(decrypted) = deobfuscate_text(secret, remove_any_ext(token)) { +pub fn get_hls_session_token_and_url_from_token(secret: &[u8; 16], token: &str) -> Option<(Option, String)> { + if let Ok(decrypted) = open_hls_resource_url(secret, remove_any_ext(token)) { let parts: Vec<&str> = decrypted.split(TOKEN_SEPARATOR).collect(); if parts.len() == 2 { let session_token: String = parts[0].to_string(); diff --git a/backend/parser/src/m3u.rs b/backend/parser/src/m3u.rs index 98542be0d..b26203075 100644 --- a/backend/parser/src/m3u.rs +++ b/backend/parser/src/m3u.rs @@ -471,7 +471,6 @@ fn process_header_internal( } } - plih.ingest_resource_values(input_name); plih.freeze_input_stream_id(); plih } @@ -794,7 +793,7 @@ mod test { use crate::m3u::{classify_token, parse_m3u, process_header, M3uToken}; use shared::{ defaults::default_episode_pattern, - model::{PlaylistItemType, ResourceLocator, StreamProperties, XtreamCluster, REGEX_CACHE}, + model::{PlaylistItemType, StreamProperties, XtreamCluster, REGEX_CACHE}, utils::{fnv1a_32, parse_season_episode, Internable, CONSTANTS}, }; use tokio::io::AsyncWriteExt; @@ -823,12 +822,6 @@ mod test { } } - fn assert_resource(locator: &str, input_name: &str, url: &str) { - let locator = ResourceLocator::decode(locator).expect("resource locator"); - assert_eq!(locator.input_name.as_ref(), input_name); - assert_eq!(locator.url.as_ref(), url); - } - #[tokio::test] async fn m3u_update_quality_parse_failure_does_not_yield_a_partial_candidate() { for text in [ @@ -1096,7 +1089,7 @@ https://example.test/series/user/pass/episode-2 // tvg-id is preserved as epg_channel_id, id falls back to numeric url segment assert_eq!(pli.epg_channel_id, Some("abc-seven".intern())); assert_eq!(pli.id, "70001".intern()); - assert_resource(&pli.logo, "hello", "https://abc.nz/.images/seven.png"); + assert_eq!(pli.logo, "https://abc.nz/.images/seven.png".intern()); assert_eq!(pli.chno, 7); assert_eq!(&*pli.group, "Sydney"); } @@ -1112,7 +1105,7 @@ https://example.test/series/user/pass/episode-2 assert_eq!(pli.title, "Seven".intern()); assert_eq!(pli.epg_channel_id, Some("abc-seven".intern())); assert_eq!(pli.id, "70002".intern()); - assert_resource(&pli.logo, "hello", "https://abc.nz/.images/seven.png"); + assert_eq!(pli.logo, "https://abc.nz/.images/seven.png".intern()); assert_eq!(pli.chno, 7); assert_eq!(&*pli.group, "Sydney"); } @@ -1128,7 +1121,7 @@ https://example.test/series/user/pass/episode-2 assert_eq!(pli.name, "UK-NOWTV| SKY CRIME FHD".intern()); assert_eq!(pli.title, "UK-NOWTV| SKY CRIME FHD".intern()); assert_eq!(pli.id, "1905905".intern()); // URL id is master; CUID is only fallback - assert_resource(&pli.logo, "test", "https://logo.m3uassets.com/skycrime.png"); + assert_eq!(pli.logo, "https://logo.m3uassets.com/skycrime.png".intern()); assert_eq!(&*pli.group, "๐Ÿ”ชMurder Mystery"); assert_eq!(pli.epg_channel_id, Some("skycrime.uk".intern())); } diff --git a/backend/parser/src/xtream.rs b/backend/parser/src/xtream.rs index 65a053e2c..dcc480674 100644 --- a/backend/parser/src/xtream.rs +++ b/backend/parser/src/xtream.rs @@ -153,7 +153,6 @@ pub fn parse_xtream_series_info( ..Default::default() }, }; - item.header.ingest_resource_values(&input.name); item.header.freeze_input_stream_id(); item }) @@ -281,7 +280,6 @@ pub async fn parse_xtream( ..Default::default() }, }; - item.header.ingest_resource_values(&input_name); item.header.freeze_input_stream_id(); group.add(item); } @@ -512,7 +510,6 @@ where ..Default::default() }, }; - item.header.ingest_resource_values(input_name); item.header.freeze_input_stream_id(); // if let Some(StreamProperties::Series(props)) = item.header.additional_properties.as_mut() { diff --git a/backend/processing/src/epg.rs b/backend/processing/src/epg.rs index d4cf06606..f2493082b 100644 --- a/backend/processing/src/epg.rs +++ b/backend/processing/src/epg.rs @@ -9,7 +9,6 @@ use shared::{ use std::{ collections::HashSet, path::{Path, PathBuf}, - sync::Arc, }; use tuliprox_core::{ model::{ConfigInput, EpgSource, EpgSourceType, PersistedEpgSource, PersistedEpgSourceKind}, @@ -201,7 +200,7 @@ pub async fn get_xmltv( match download_epg_file(epg_source, ctx, input, headers, storage_dir).await { Ok(file_path) => { stored_file_paths.push(file_path.clone()); - match persisted_source_from_config(epg_source, file_path, input) { + match persisted_source_from_config(epg_source, file_path) { Ok(persisted) => file_paths.push(persisted), Err(err) => errors.push(err), } @@ -230,7 +229,6 @@ pub async fn get_xmltv( fn persisted_source_from_config( epg_source: &EpgSource, file_path: PathBuf, - input: &ConfigInput, ) -> Result { let kind = match epg_source.source_type { EpgSourceType::Xmltv => PersistedEpgSourceKind::Xmltv, @@ -248,13 +246,7 @@ fn persisted_source_from_config( } }; - Ok(PersistedEpgSource { - file_path, - priority: epg_source.priority, - logo_override: epg_source.logo_override, - kind, - input_name: Some(Arc::clone(&input.name)), - }) + Ok(PersistedEpgSource { file_path, priority: epg_source.priority, logo_override: epg_source.logo_override, kind }) } #[cfg(test)] diff --git a/backend/processing/src/parser/xmltv.rs b/backend/processing/src/parser/xmltv.rs index 1d98b638d..eb0abe10e 100644 --- a/backend/processing/src/parser/xmltv.rs +++ b/backend/processing/src/parser/xmltv.rs @@ -4,7 +4,7 @@ use quick_xml::events::{BytesStart, BytesText, Event}; use serde::{Deserialize, Serialize}; use shared::{ concat_string, - model::{has_resource_scheme, ingest_resource_value, EpgCategory, EpgChannel, EpgNamePrefix, EpgProgramme}, + model::{EpgCategory, EpgChannel, EpgNamePrefix, EpgProgramme}, utils::{deunicode_string, Internable, CONSTANTS}, }; use std::{ @@ -358,7 +358,6 @@ impl TVGuide { if add_channel { with_folded_epg_id(&tag_epg_id, |folded| source_processed.insert(folded.intern())); id_cache.insert_processed_epg_id(&tag_epg_id); - let icon_source = ingest_epg_icon(epg_source, Self::channel_icon(&tag)); accumulator.upsert_channel( epg_source.priority, source_order, @@ -366,7 +365,7 @@ impl TVGuide { EpgChannel { id: Arc::clone(&tag_epg_id), title: Self::channel_display_name(&tag), - icon: icon_source, + icon: Self::channel_icon(&tag), programmes: vec![], }, ); @@ -376,14 +375,13 @@ impl TVGuide { EPG_TAG_PROGRAMME => { if let Some(epg_id) = tag.get_attribute_value(&epg_attrib_channel) { if with_folded_epg_id(epg_id, |folded| source_processed.contains(folded)) { - if let Some(mut programme) = Self::extract_programme( + if let Some(programme) = Self::extract_programme( &tag, epg_id, &start_attrib, &stop_attrib, &catchup_id_attrib, ) { - programme.icon = ingest_epg_icon(epg_source, programme.icon); accumulator.push_programme(epg_source.priority, source_order, programme); } } @@ -447,9 +445,8 @@ impl TVGuide { ) .await { - Ok(mut channel) => { + Ok(channel) => { id_cache.insert_processed_epg_id(channel_id); - channel.icon = ingest_epg_icon(epg_source, channel.icon); accumulator.add_channel_with_programmes( epg_source.priority, source_order, @@ -1134,22 +1131,6 @@ fn apply_dummy_policies(channels: &mut [ChannelMergeAcc], dummy_policies: &HashM } } -/// Origin of an icon belonging to `source`. -/// -/// An icon without a value has no origin, and a source without an origin keeps `None`, which the -/// request path treats as public-only. -fn ingest_epg_icon(source: &PersistedEpgSource, mut icon: Option>) -> Option> { - let value = icon.as_mut()?; - if let Some(input_name) = &source.input_name { - if ingest_resource_value(value, input_name).is_err() { - return None; - } - } else if has_resource_scheme(value) { - return None; - } - icon -} - fn backfill_programme_metadata(existing: &mut EpgProgramme, incoming: EpgProgramme) { if existing.title.is_none() { existing.title = incoming.title; @@ -1327,7 +1308,7 @@ mod tests { } fn xmltv_source(file_path: PathBuf, priority: i16, logo_override: bool) -> PersistedEpgSource { - PersistedEpgSource { file_path, priority, logo_override, kind: PersistedEpgSourceKind::Xmltv, input_name: None } + PersistedEpgSource { file_path, priority, logo_override, kind: PersistedEpgSourceKind::Xmltv } } fn dummy_policy_source(priority: i16, source_order: usize, title: &str) -> EpgDummyPolicySource { @@ -1934,7 +1915,6 @@ mod tests { ..IcsEpgSourceConfig::default() }), }, - input_name: None, }, ]); let mut id_cache = EpgIdCache::new(None); diff --git a/backend/processing/src/processor/epg.rs b/backend/processing/src/processor/epg.rs index ad74f5a22..a4dc7b658 100644 --- a/backend/processing/src/processor/epg.rs +++ b/backend/processing/src/processor/epg.rs @@ -952,7 +952,6 @@ mod tests { match_names, config: Box::new(IcsEpgSourceConfig::default()), }, - input_name: None, }]) } @@ -1041,7 +1040,6 @@ mod tests { priority: 0, logo_override: false, kind: PersistedEpgSourceKind::Xmltv, - input_name: None, }]); let mut playlist = FetchedPlaylist { input: &input, @@ -1256,7 +1254,6 @@ mod tests { priority: 0, logo_override: true, kind: PersistedEpgSourceKind::Xmltv, - input_name: Some("epg-input".into()), }]); let mut playlist = FetchedPlaylist { input: &input, @@ -1269,12 +1266,8 @@ mod tests { let updated = playlist.items_mut().next().unwrap(); assert_eq!(updated.header.epg_channel_id.as_deref(), Some("demo.channel")); - let logo = shared::model::ResourceLocator::decode(&updated.header.logo).expect("EPG logo locator"); - let logo_small = - shared::model::ResourceLocator::decode(&updated.header.logo_small).expect("EPG small logo locator"); - assert_eq!(logo.input_name.as_ref(), "epg-input"); - assert_eq!(logo.url.as_ref(), "http://guide/icon.png"); - assert_eq!(logo_small, logo); + assert_eq!(updated.header.logo.as_ref(), "http://guide/icon.png"); + assert_eq!(updated.header.logo_small.as_ref(), "http://guide/icon.png"); assert_eq!(epg[0].children[0].id.as_ref(), "Demo.Channel"); }); } @@ -1334,7 +1327,6 @@ mod tests { priority: 0, logo_override: false, kind: PersistedEpgSourceKind::Xmltv, - input_name: None, }]); let mut playlist = FetchedPlaylist { input: &input, @@ -1442,7 +1434,6 @@ mod tests { priority: 0, logo_override: false, kind: PersistedEpgSourceKind::Xmltv, - input_name: None, }]); let mut playlist = FetchedPlaylist { input: &input, @@ -1477,7 +1468,6 @@ mod tests { priority: 0, logo_override: false, kind: PersistedEpgSourceKind::Xmltv, - input_name: None, }]); let mut playlist = FetchedPlaylist { input: &input, diff --git a/backend/processing/src/processor/library.rs b/backend/processing/src/processor/library.rs index c757b40dd..bae2c44ef 100644 --- a/backend/processing/src/processor/library.rs +++ b/backend/processing/src/processor/library.rs @@ -139,21 +139,14 @@ fn to_playlist_item( ) { match &entry.metadata { MediaMetadata::Movie(_) => { - let mut item = build_movie_playlist_item(entry, input_name, group_name, api_base_path); - item.header.ingest_resource_values(input_name); - channels.push(item); + channels.push(build_movie_playlist_item(entry, input_name, group_name, api_base_path)); } MediaMetadata::Series(_) => { if let Some((series_info, episodes)) = build_series_playlist_items(entry, input_name, group_name, api_base_path) { - let mut series_info = series_info; - series_info.header.ingest_resource_values(input_name); channels.push(series_info); - channels.extend(episodes.into_iter().map(|mut episode| { - episode.header.ingest_resource_values(input_name); - episode - })); + channels.extend(episodes); } } } diff --git a/backend/processing/src/processor/playlist/tests.rs b/backend/processing/src/processor/playlist/tests.rs index ea1f9bb0e..80f856584 100644 --- a/backend/processing/src/processor/playlist/tests.rs +++ b/backend/processing/src/processor/playlist/tests.rs @@ -2425,7 +2425,6 @@ mod mapping_stage { match_names: vec!["BBC One".intern()], config: Box::new(IcsEpgSourceConfig::default()), }, - input_name: None, }]); let mut playlist = FetchedPlaylist { @@ -2521,7 +2520,6 @@ match { match_names: vec![], config: Box::new(IcsEpgSourceConfig::default()), }, - input_name: None, }]); let mut playlist = FetchedPlaylist { input: &input, diff --git a/backend/processing/src/processor/playlist/transform.rs b/backend/processing/src/processor/playlist/transform.rs index b2232b8b1..a0f6b5f5a 100644 --- a/backend/processing/src/processor/playlist/transform.rs +++ b/backend/processing/src/processor/playlist/transform.rs @@ -164,12 +164,6 @@ impl MappingStageOutcome { } pub(crate) fn map_channel(mut channel: PlaylistItem, mapping: &CompiledMapping) -> ChannelMappingOutcome { - let mut trusted_locators = HashSet::new(); - channel.header.visit_resource_values(&mut |value| { - if shared::model::resolve_resource_value(value).is_ok_and(|locator| locator.is_some()) { - trusted_locators.insert(Arc::clone(value)); - } - }); let mut matched_rules = 0; let mut virtual_items = vec![]; let mut changed_fields = HashSet::new(); @@ -204,16 +198,6 @@ pub(crate) fn map_channel(mut channel: PlaylistItem, mapping: &CompiledMapping) } } } - let normalize_mapped_item = |item: &mut PlaylistItem| { - item.header.visit_resource_values_mut(&mut |value| { - if shared::model::has_resource_scheme(value) && !trusted_locators.contains(value) { - *value = Arc::from(""); - } - }); - item.header.normalize_internal_resource_values(); - }; - normalize_mapped_item(&mut channel); - virtual_items.iter_mut().for_each(normalize_mapped_item); ChannelMappingOutcome { channel, virtual_items, matched_rules, changed_fields, diagnostics } } diff --git a/backend/processing/src/processor/stalker.rs b/backend/processing/src/processor/stalker.rs index c84a829f9..c61e95439 100644 --- a/backend/processing/src/processor/stalker.rs +++ b/backend/processing/src/processor/stalker.rs @@ -202,7 +202,7 @@ pub async fn download_stalker_playlist( app_config, api_client.as_ref(), &handshake, - refresh_plan.clone(), + refresh_plan, &storage_path, identity_fingerprint, refresh_mode.budget(), diff --git a/backend/processing/src/processor/stalker_refresh.rs b/backend/processing/src/processor/stalker_refresh.rs index e54515f88..5009cc3f3 100644 --- a/backend/processing/src/processor/stalker_refresh.rs +++ b/backend/processing/src/processor/stalker_refresh.rs @@ -128,12 +128,11 @@ impl StalkerClusterSelection { } /// Requested clusters and the publication policy applied once their generation is complete. -#[derive(Clone)] +#[derive(Clone, Copy)] pub struct StalkerRefreshPlan { selection: StalkerClusterSelection, update_quality: ConfigInputUpdateQuality, quality_bypass_mask: u8, - input_name: Arc, } impl StalkerRefreshPlan { @@ -144,7 +143,7 @@ impl StalkerRefreshPlan { let quality_bypass_mask = matches!(quality_policy, UpdateQualityPolicy::Bypass) .then_some(selection.mask() & MEDIA_SELECTION) .unwrap_or(0); - Self { selection, update_quality, quality_bypass_mask, input_name: Arc::clone(&input.name) } + Self { selection, update_quality, quality_bypass_mask } } } @@ -560,16 +559,13 @@ fn map_items( categories: &HashMap, kind: StalkerStreamKind, added_at: i64, - input_name: &Arc, ) -> Vec { raw_items .iter() .map(|raw| { let category = raw.category_id().and_then(|value| value.parse::().ok()).and_then(|id| categories.get(&id)); - let mut item = parser::map_stalker_to_playlist_item(raw, category, kind, added_at); - item.ingest_resource_values(input_name); - item + parser::map_stalker_to_playlist_item(raw, category, kind, added_at) }) .collect() } @@ -640,7 +636,7 @@ pub async fn advance_stalker_refresh( identity_fingerprint: u64, mut budget: StalkerRefreshBudget, ) -> Result { - let StalkerRefreshPlan { selection, update_quality, quality_bypass_mask, input_name } = refresh_plan; + let StalkerRefreshPlan { selection, update_quality, quality_bypass_mask } = refresh_plan; let mut checkpoint = load_or_start_checkpoint(storage_path, identity_fingerprint, selection, quality_bypass_mask).await?; if checkpoint.phase == StalkerRefreshPhase::Terminal { @@ -677,7 +673,7 @@ pub async fn advance_stalker_refresh( checkpoint.retry_count = 0; } Ok(raw) => { - let items = map_items(&raw, categories, StalkerStreamKind::Live, added_at, &input_name); + let items = map_items(&raw, categories, StalkerStreamKind::Live, added_at); let path = generation_data_path(storage_path, checkpoint.generation, StalkerGenerationData::Live); snapshot_stalker_items_at(app_config, path.clone(), &items).await?; @@ -711,7 +707,7 @@ pub async fn advance_stalker_refresh( { return yield_after_error(storage_path, checkpoint, err).await; } - let items = map_items(&response.items, categories, StalkerStreamKind::Live, added_at, &input_name); + let items = map_items(&response.items, categories, StalkerStreamKind::Live, added_at); let path = generation_data_path(storage_path, checkpoint.generation, StalkerGenerationData::Live); upsert_stalker_items_at(app_config, &path, &items).await?; checkpoint.processed = checkpoint.processed.saturating_add(items.len() as u64); @@ -743,7 +739,7 @@ pub async fn advance_stalker_refresh( { return yield_after_error(storage_path, checkpoint, err).await; } - let items = map_items(&response.items, categories, StalkerStreamKind::Movie, added_at, &input_name); + let items = map_items(&response.items, categories, StalkerStreamKind::Movie, added_at); let path = generation_data_path(storage_path, checkpoint.generation, StalkerGenerationData::Vod); upsert_stalker_items_at(app_config, &path, &items).await?; checkpoint.processed = checkpoint.processed.saturating_add(items.len() as u64); @@ -790,7 +786,6 @@ pub async fn advance_stalker_refresh( root.flussonic_tmp_link = capabilities.flussonic_temporary_link; root.wowza_tmp_link = capabilities.wowza_temporary_link; root.use_http_tmp_link = capabilities.use_http_temporary_link; - root.ingest_resource_values(&input_name); root }) .collect(); @@ -835,10 +830,7 @@ pub async fn advance_stalker_refresh( used_episode_ids .insert(prepare_stalker_episode_series_at(app_config, &path, series_id).await?) }; - let mut episodes = parser::map_stalker_series_details(&details, &root, added_at, used); - for episode in &mut episodes { - episode.ingest_resource_values(&input_name); - } + let episodes = parser::map_stalker_series_details(&details, &root, added_at, used); upsert_stalker_items_at(app_config, &path, &episodes).await?; checkpoint.processed = checkpoint.processed.saturating_add(episodes.len() as u64); checkpoint.phase = StalkerRefreshPhase::SeriesDetails { provider_id: Some(root.stream_id) }; @@ -1529,7 +1521,7 @@ mod tests { StalkerCategory { id: "10".to_string(), title: "News".to_string(), alias: None, number: 1 }, )]); - let items = map_items(&[raw], &categories, StalkerStreamKind::Live, 0, &Arc::from("stalker-input")); + let items = map_items(&[raw], &categories, StalkerStreamKind::Live, 0); assert_eq!(items.len(), 1); assert_eq!(items[0].category_id, 10); diff --git a/backend/repository/src/m3u_playlist_iterator.rs b/backend/repository/src/m3u_playlist_iterator.rs index 04de72f7f..a1fda3412 100644 --- a/backend/repository/src/m3u_playlist_iterator.rs +++ b/backend/repository/src/m3u_playlist_iterator.rs @@ -2,15 +2,16 @@ use crate::{ ensure_target_storage_path, get_file_path_for_db_index, m3u_get_file_path_for_db, open_playlist_reader, storage_const, user_get_bouquet_filter, LockedReceiverStream, }; -use futures::Stream; +use futures::{Stream, StreamExt}; use log::error; use shared::{ create_bitset, error::TuliproxError, model::{ - ConfigTargetOptions, M3uPlaylistItem, PlaylistItemType, ProxyType, StreamProperties, TargetType, XtreamCluster, + ConfigTargetOptions, M3uPlaylistItem, PlaylistItemType, ProxyType, ResourceOutputPolicy, StreamProperties, + TargetType, XtreamCluster, }, - utils::{extract_extension_from_url, sanitize_sensitive_info, Internable, PROVIDER_SCHEME_PREFIX}, + utils::{concat_path, extract_extension_from_url, sanitize_sensitive_info, Internable, PROVIDER_SCHEME_PREFIX}, }; use std::{ borrow::Cow, @@ -20,7 +21,10 @@ use std::{ task::{Context, Poll}, }; use tokio::{sync::mpsc, task}; -use tuliprox_core::model::{AppConfig, ConfigTarget, ProxyUserCredentials}; +use tuliprox_core::{ + model::{AppConfig, ConfigTarget, ProxyUserCredentials}, + utils::classify_output_resource_url, +}; use tuliprox_parser::m3u_format::{build_m3u_catchup_rewrite, flussonic_proxy_live_file}; create_bitset!(u8, M3uPlaylistIteratorFlags, MaskRedirectUrl, IncludeTypeInUrl, RewriteResource); @@ -135,6 +139,8 @@ fn apply_rewrite( flags: M3uPlaylistIteratorFlagsSet, proxy_type: ProxyType, ) -> M3uPlaylistItem { + m3u_pli.logo = shared::model::persisted_resource_arc(&m3u_pli.logo); + m3u_pli.logo_small = shared::model::persisted_resource_arc(&m3u_pli.logo_small); let is_redirect = proxy_type.is_redirect(m3u_pli.item_type) || target_options.and_then(|o| o.force_redirect.as_ref()).is_some_and(|f| f.has_cluster(m3u_pli.item_type)); let should_rewrite_urls = @@ -210,22 +216,19 @@ fn apply_rewrite( shared::concat_string!(&base, "/", live_file) }, ); - let resource_url = if flags.contains(M3uPlaylistIteratorFlags::RewriteResource) { - let source_url = if m3u_pli.logo.is_empty() { m3u_pli.logo_small.as_ref() } else { m3u_pli.logo.as_ref() }; - Some(build_rewritten_url(ctx, source_url, &m3u_pli, false, storage_const::M3U_RESOURCE_PATH, false)) - } else { - None - }; m3u_pli.t_stream_url = stream_url.intern(); - m3u_pli.t_resource_url = resource_url; } else { m3u_pli.t_stream_url = match effective_source_url { Cow::Borrowed(_) => Arc::clone(&m3u_pli.url), Cow::Owned(url) => url.intern(), }; - m3u_pli.t_resource_url = None; } + m3u_pli.t_resource_url = flags.contains(M3uPlaylistIteratorFlags::RewriteResource).then(|| { + let source_url = if m3u_pli.logo.is_empty() { m3u_pli.logo_small.as_ref() } else { m3u_pli.logo.as_ref() }; + build_rewritten_url(ctx, source_url, &m3u_pli, false, storage_const::M3U_RESOURCE_PATH, false) + }); + if let Some(rewrite) = catchup_rewrite { m3u_pli.t_catchup_mode = Some(rewrite.mode); m3u_pli.t_catchup_source = Some(rewrite.source); @@ -237,6 +240,49 @@ fn apply_rewrite( m3u_pli } +fn player_resource_url( + item: &M3uPlaylistItem, + source_url: &Arc, + field: &str, + ctx: &UrlRewriteContext<'_>, + policy: ResourceOutputPolicy, +) -> Arc { + match policy { + ResourceOutputPolicy::Direct if source_url.starts_with("/api/v1/library/thumbnail/") => { + concat_path(ctx.base_url, source_url).intern() + } + ResourceOutputPolicy::Direct => Arc::clone(source_url), + ResourceOutputPolicy::Proxy => { + // Built in one step: the resource link is the rewritten item URL plus the requested field, + // and an intermediate string would only be allocated to be copied. + let virtual_id = item.virtual_id.to_string(); + shared::concat_string!( + cap = ctx.base_url.len() + storage_const::M3U_RESOURCE_PATH.len() + + ctx.username.len() + ctx.password.len() + virtual_id.len() + field.len() + 8; + ctx.base_url, "/", storage_const::M3U_RESOURCE_PATH, "/", + ctx.username, "/", ctx.password, "/", &virtual_id, "/", field + ) + .intern() + } + ResourceOutputPolicy::Blocked => "".intern(), + } +} + +/// Replaces the icons the player must not fetch itself. +/// +/// Only a provably public destination may be handed to the player; everything else is served through +/// this instance, so a private address never reaches the client and a local media server still works. +async fn rewrite_player_resources_for_output(item: &mut M3uPlaylistItem, ctx: &UrlRewriteContext<'_>) { + if !item.logo.is_empty() { + let policy = classify_output_resource_url(&item.logo).await; + item.logo = player_resource_url(item, &item.logo, "logo", ctx, policy); + } + if !item.logo_small.is_empty() { + let policy = classify_output_resource_url(&item.logo_small).await; + item.logo_small = player_resource_url(item, &item.logo_small, "logo_small", ctx, policy); + } +} + #[allow(clippy::too_many_lines)] impl M3uPlaylistIterator { pub async fn new( @@ -397,10 +443,47 @@ fn build_proxy_xmltv_url_tvg(base_url: &str, username: &str, password: &str) -> Some(format!("{base}/xmltv.php?{query}")) } +type M3uTextStream = Pin> + Send>>; + +/// Owned rewrite inputs of the text output, so that resource rewriting can run per item without +/// re-deriving them. +struct ResourceRewrite { + base_url: String, + username: String, + password: String, +} + +/// Serializes playlist items into M3U lines, rewriting the icons the player must not fetch itself. +/// +/// Resource rewriting and serialization run here rather than in the reading task: classifying a +/// destination resolves a name, which must not block the blocking task that reads the playlist. +fn m3u_text_stream( + items: impl Stream> + Send + 'static, + target_options: Option>, + rewrite: Arc, + rewrite_resources: bool, +) -> M3uTextStream { + Box::pin(items.then(move |entry| { + let rewrite = Arc::clone(&rewrite); + let target_options = target_options.clone(); + async move { + let (mut item, _has_next) = entry.map_err(|error| error.to_string())?; + if !rewrite_resources { + let ctx = UrlRewriteContext { + base_url: &rewrite.base_url, + username: &rewrite.username, + password: &rewrite.password, + }; + rewrite_player_resources_for_output(&mut item, &ctx).await; + } + Ok(item.to_m3u(target_options.as_deref(), true)) + } + })) +} + pub struct M3uPlaylistM3uTextIterator { - inner: M3uPlaylistIterator, + inner: M3uTextStream, started: bool, - target_options: Option, url_tvg: Option, } @@ -411,12 +494,14 @@ impl M3uPlaylistM3uTextIterator { user: &ProxyUserCredentials, ) -> Result { let base_url = cfg.get_user_server_info(user).map(|server| server.get_base_url()).unwrap_or_default(); - Ok(Self { - inner: M3uPlaylistIterator::new(cfg, target, user).await?, - started: false, - target_options: target.options.clone(), - url_tvg: build_proxy_xmltv_url_tvg(&base_url, &user.username, &user.password), - }) + let url_tvg = build_proxy_xmltv_url_tvg(&base_url, &user.username, &user.password); + let target_options = target.options.clone().map(Arc::new); + let rewrite = + Arc::new(ResourceRewrite { base_url, username: user.username.clone(), password: user.password.clone() }); + let rewrite_resources = cfg.is_reverse_proxy_resource_rewrite_enabled(); + let items = M3uPlaylistIterator::new(cfg, target, user).await?; + let inner = m3u_text_stream(items, target_options, rewrite, rewrite_resources); + Ok(Self { inner, started: false, url_tvg }) } } @@ -433,25 +518,19 @@ impl Stream for M3uPlaylistM3uTextIterator { return Poll::Ready(Some(Ok(header))); } - match Pin::new(&mut self.inner).poll_next(cx) { - Poll::Ready(Some(Ok((m3u_pli, _has_next)))) => { - let target_options = self.target_options.as_ref(); - Poll::Ready(Some(Ok(m3u_pli.to_m3u(target_options, true)))) - } - Poll::Ready(Some(Err(error))) => Poll::Ready(Some(Err(error.to_string()))), - Poll::Ready(None) => Poll::Ready(None), - Poll::Pending => Poll::Pending, - } + self.inner.as_mut().poll_next(cx) } } #[cfg(test)] mod tests { use super::{ - apply_rewrite, build_proxy_xmltv_url_tvg, M3uPlaylistIterator, M3uPlaylistIteratorFlags, - M3uPlaylistIteratorFlagsSet, M3uPlaylistM3uTextIterator, UrlRewriteContext, + apply_rewrite, build_proxy_xmltv_url_tvg, m3u_text_stream, rewrite_player_resources_for_output, + M3uPlaylistIterator, M3uPlaylistIteratorFlags, M3uPlaylistIteratorFlagsSet, M3uPlaylistM3uTextIterator, + ResourceRewrite, UrlRewriteContext, }; use crate::LockedReceiverStream; + use base64::Engine; use futures::StreamExt; use shared::{ model::{ @@ -548,6 +627,51 @@ mod tests { assert_eq!(rewritten.t_stream_url.as_ref(), "http://example.com/live/user/pass/813294.ts"); } + #[test] + fn redirect_without_mask_still_rewrites_private_logo() { + let input_by_name = HashMap::new(); + let ctx = UrlRewriteContext { base_url: "https://example.com", username: "user", password: "pass" }; + let mut item = m3u_item("http://example.com/live/813294.ts"); + item.logo = "http://192.168.1.20/logo.png".intern(); + let mut flags = M3uPlaylistIteratorFlagsSet::new(); + flags.set(M3uPlaylistIteratorFlags::RewriteResource); + + let rewritten = apply_rewrite(item, &ctx, 1, &[7u8; 16], &input_by_name, None, flags, ProxyType::Redirect); + + assert_eq!(rewritten.t_stream_url.as_ref(), "http://example.com/live/813294.ts"); + assert_eq!(rewritten.t_resource_url.as_deref(), Some("https://example.com/resource/m3u/user/pass/813294")); + assert!(!rewritten.to_m3u(None, true).contains("192.168.1.20")); + } + + #[tokio::test] + async fn disabled_resource_rewrite_keeps_public_logo_and_proxies_private_logo() { + let input_by_name = HashMap::new(); + let ctx = UrlRewriteContext { base_url: "https://proxy.example", username: "user", password: "pass" }; + let mut item = m3u_item("http://provider.example/live/813294.ts"); + let old_locator = + rmp_serde::to_vec(&("example1", "http://192.168.1.20/logo.png")).expect("persisted logo locator"); + item.logo = + format!("resource://v1/{}", base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(old_locator)).intern(); + item.logo_small = "http://8.8.8.8/public.png".intern(); + let mut item = apply_rewrite( + item, + &ctx, + 1, + &[7u8; 16], + &input_by_name, + None, + M3uPlaylistIteratorFlagsSet::new(), + ProxyType::Redirect, + ); + + rewrite_player_resources_for_output(&mut item, &ctx).await; + + let playlist = item.to_m3u(None, true); + assert!(playlist.contains("tvg-logo=\"https://proxy.example/resource/m3u/user/pass/813294/logo\"")); + assert!(playlist.contains("tvg-logo-small=\"http://8.8.8.8/public.png\"")); + assert!(!playlist.contains("192.168.1.20")); + } + #[test] fn upstream_user_agent_is_only_exposed_for_direct_provider_urls() { let input_by_name = HashMap::new(); @@ -713,15 +837,65 @@ mod tests { assert_eq!(build_proxy_xmltv_url_tvg("", "user", "pass"), None); } + #[tokio::test] + async fn m3u_text_output_proxies_private_logo_when_resource_rewrite_is_disabled() { + let mut item = m3u_item("http://provider.example/live/813294.ts"); + item.logo = "http://192.168.1.20/logo.png".intern(); + item.logo_small = "http://8.8.8.8/public.png".intern(); + let rewrite = Arc::new(ResourceRewrite { + base_url: "https://proxy.example".to_string(), + username: "user".to_string(), + password: "pass".to_string(), + }); + + let lines: Vec<_> = + m3u_text_stream(futures::stream::iter(vec![Ok((item, false))]), None, rewrite, false).collect().await; + + let line = lines[0].as_ref().expect("item line"); + assert!(line.contains("tvg-logo=\"https://proxy.example/resource/m3u/user/pass/813294/logo\""), "{line}"); + assert!(line.contains("tvg-logo-small=\"http://8.8.8.8/public.png\""), "{line}"); + assert!(!line.contains("192.168.1.20"), "{line}"); + } + + #[tokio::test] + async fn m3u_text_output_uses_the_stream_link_when_resource_rewrite_is_enabled() { + let mut item = m3u_item("http://provider.example/live/813294.ts"); + item.logo = "http://192.168.1.20/logo.png".intern(); + let rewrite = Arc::new(ResourceRewrite { + base_url: "https://proxy.example".to_string(), + username: "user".to_string(), + password: "pass".to_string(), + }); + let mut flags = M3uPlaylistIteratorFlagsSet::new(); + flags.set(M3uPlaylistIteratorFlags::RewriteResource); + let item = apply_rewrite( + item, + &UrlRewriteContext { base_url: "https://proxy.example", username: "user", password: "pass" }, + 1, + &[7u8; 16], + &HashMap::new(), + None, + flags, + ProxyType::Redirect, + ); + + let lines: Vec<_> = + m3u_text_stream(futures::stream::iter(vec![Ok((item, false))]), None, rewrite, true).collect().await; + + let line = lines[0].as_ref().expect("item line"); + assert!(line.contains("tvg-logo=\"https://proxy.example/resource/m3u/user/pass/813294/logo\""), "{line}"); + assert!(!line.contains("192.168.1.20"), "{line}"); + } + // Regression lock: never forward the provider's source `url-tvg`. Without // configured Tuliprox server information, the header stays bare. #[tokio::test] async fn m3u_text_iterator_emits_bare_extm3u_header_without_proxying_source_url_tvg() { - let (tx, rx) = mpsc::channel::>(1); - drop(tx); - let inner_iter = M3uPlaylistIterator { inner: LockedReceiverStream::new_empty(rx) }; - let mut text_iter = - M3uPlaylistM3uTextIterator { inner: inner_iter, started: false, target_options: None, url_tvg: None }; + let mut text_iter = M3uPlaylistM3uTextIterator { + inner: Box::pin(futures::stream::empty::>()), + started: false, + url_tvg: None, + }; let first = text_iter.next().await; assert_eq!( @@ -736,13 +910,9 @@ mod tests { #[tokio::test] async fn m3u_text_iterator_emits_proxy_xmltv_header_when_configured() { - let (tx, rx) = mpsc::channel::>(1); - drop(tx); - let inner_iter = M3uPlaylistIterator { inner: LockedReceiverStream::new_empty(rx) }; let mut text_iter = M3uPlaylistM3uTextIterator { - inner: inner_iter, + inner: Box::pin(futures::stream::empty::>()), started: false, - target_options: None, url_tvg: Some("https://proxy.example/xmltv.php?username=user&password=pass".to_string()), }; diff --git a/backend/repository/src/xtream_playlist_iterator.rs b/backend/repository/src/xtream_playlist_iterator.rs index 8bacf477c..4e3841f91 100644 --- a/backend/repository/src/xtream_playlist_iterator.rs +++ b/backend/repository/src/xtream_playlist_iterator.rs @@ -2,11 +2,11 @@ use crate::{ get_file_path_for_db_index, open_playlist_reader, user_get_bouquet_filter, xtream_get_file_path, xtream_get_storage_path, LockedReceiverStream, }; -use futures::Stream; +use futures::{Stream, StreamExt}; use log::error; use shared::{ error::TuliproxError, - model::{PlaylistItemType, TargetType, XtreamCluster, XtreamMappingOptions, XtreamPlaylistItem}, + model::{PlaylistItemType, TargetType, XtreamCluster, XtreamPlaylistItem}, }; use std::{ collections::HashSet, @@ -15,7 +15,10 @@ use std::{ task::{Context, Poll}, }; use tokio::{sync::mpsc, task}; -use tuliprox_core::model::{xtream_mapping_option_from_target_options, AppConfig, ConfigTarget, ProxyUserCredentials}; +use tuliprox_core::{ + model::{xtream_mapping_option_from_target_options, AppConfig, ConfigTarget, ProxyUserCredentials}, + utils::prepare_xtream_resource_hosts, +}; pub struct XtreamPlaylistIterator { inner: LockedReceiverStream>, @@ -183,9 +186,10 @@ impl Stream for XtreamPlaylistIterator { } } +type XtreamJsonStream = Pin> + Send>>; + pub struct XtreamPlaylistJsonIterator { - inner: XtreamPlaylistIterator, - options: Option, + inner: XtreamJsonStream, } impl XtreamPlaylistJsonIterator { @@ -200,34 +204,35 @@ impl XtreamPlaylistJsonIterator { TuliproxError::Config(format!("Unexpected: xtream output required for target {}", target.name)) })?; if !is_cluster_allowed_for_user(user, cluster) { - return Ok(Self { inner: XtreamPlaylistIterator::empty(), options: None }); + return Ok(Self { inner: Box::pin(futures::stream::empty()) }); } let encrypt_secret = app_config.get_encrypt_secret(); - let options = - xtream_mapping_option_from_target_options(target, xtream_output, app_config, user, encrypt_secret)?; - Ok(Self { - inner: XtreamPlaylistIterator::new(cluster, app_config, target, category_id, user).await?, - options: Some(options), - }) + let options = Arc::new(xtream_mapping_option_from_target_options( + target, + xtream_output, + app_config, + user, + encrypt_secret, + )?); + let raw = XtreamPlaylistIterator::new(cluster, app_config, target, category_id, user).await?; + let inner = raw.then(move |entry| { + let options = Arc::clone(&options); + async move { + let (item, has_next) = entry?; + prepare_xtream_resource_hosts(&item, &options).await; + let json = serde_json::to_string(&item.to_document(&options)) + .map_err(|error| TuliproxError::RepositoryXtream(error.to_string()))?; + Ok((json, has_next)) + } + }); + Ok(Self { inner: Box::pin(inner) }) } } impl Stream for XtreamPlaylistJsonIterator { type Item = Result<(String, bool), TuliproxError>; fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { - match Pin::new(&mut self.inner).poll_next(cx) { - Poll::Ready(Some(Ok((pli, has_next)))) => { - let Some(options) = self.options.as_ref() else { - return Poll::Ready(None); - }; - let json = serde_json::to_string(&pli.to_document(options)) - .map_err(|error| TuliproxError::RepositoryXtream(error.to_string())); - Poll::Ready(Some(json.map(|json| (json, has_next)))) - } - Poll::Ready(Some(Err(error))) => Poll::Ready(Some(Err(error))), - Poll::Ready(None) => Poll::Ready(None), - Poll::Pending => Poll::Pending, - } + self.inner.as_mut().poll_next(cx) } } diff --git a/backend/repository/src/xtream_repository.rs b/backend/repository/src/xtream_repository.rs index 4b5a9d765..e6e5ccd62 100644 --- a/backend/repository/src/xtream_repository.rs +++ b/backend/repository/src/xtream_repository.rs @@ -2612,9 +2612,8 @@ async fn persist_input_info( cluster: XtreamCluster, input_name: &str, provider_id: u32, - mut props: StreamProperties, + props: StreamProperties, ) -> Result<(), Error> { - props.normalize_internal_resource_values(&Arc::from(input_name)); let xtream_path = xtream_get_file_path(storage_path, cluster); if xtream_path.exists() { let file_lock = app_config.file_locks.write_lock(&xtream_path).await; @@ -2657,15 +2656,11 @@ pub async fn persist_input_info_batch( storage_path: &Path, cluster: XtreamCluster, input_name: &str, - mut updates: Vec<(u32, StreamProperties)>, + updates: Vec<(u32, StreamProperties)>, ) -> Result<(), Error> { if updates.is_empty() { return Ok(()); } - let input_name_arc: Arc = Arc::from(input_name); - for (_, props) in &mut updates { - props.normalize_internal_resource_values(&input_name_arc); - } let xtream_path = xtream_get_file_path(storage_path, cluster); if xtream_path.exists() { let file_lock = app_config.file_locks.write_lock(&xtream_path).await; diff --git a/backend/session/src/provider_lineup_manager.rs b/backend/session/src/provider_lineup_manager.rs index 81f91a7e6..d67faf863 100644 --- a/backend/session/src/provider_lineup_manager.rs +++ b/backend/session/src/provider_lineup_manager.rs @@ -1065,7 +1065,6 @@ mod tests { provider_configs: None, cache_duration_seconds: 0, stalker: None, - resource_policy: None, } } diff --git a/docs/src/configuration/reverse-proxy.md b/docs/src/configuration/reverse-proxy.md index 8e0a18f26..0b0df03ae 100644 --- a/docs/src/configuration/reverse-proxy.md +++ b/docs/src/configuration/reverse-proxy.md @@ -56,11 +56,25 @@ reverse_proxy: | Parameter | Type | Default | Technical Impact & Background | | :--- | :--- | :--- | :--- | -| `resource_rewrite_disabled` | Bool | `false` | Normally, Tuliprox rewrites all image URLs in playlists to point to itself (e.g., `http://tuliprox:8901/resource/...`). If set to `true`, original URLs are kept (clients load images directly from the provider). **Warning:** Local caching will stop working if this is enabled! | +| `resource_rewrite_disabled` | Bool | `false` | Normally, Tuliprox rewrites image URLs in playlists to point to itself (e.g., `http://tuliprox:8901/resource/...`). If set to `true`, only provably public image URLs are kept; private destinations still use Tuliprox resource links so external clients can load them. Public images bypass local caching in this mode. | | `rewrite_secret` | String | `""` | A 32-character Hex string (16 bytes). Tuliprox encrypts/signs the original image URLs during the rewrite process. To prevent image URLs from becoming invalid after a server restart, you MUST enter a static secret here. | | `stream_history` | Block | `null` | Optional stream telemetry block that persists raw connect/disconnect/startup-failure events to daily history files. | | `qos_aggregation` | Block | `null` | Optional background worker that aggregates stream history into compact per-stream QoS snapshots. | +**Internal destinations are never exposed through the resource route.** When resource rewriting is disabled, channel +logos, Xtream covers and backdrops, and EPG image links are classified before they are written into player output. +Only a provably public destination is handed to the client while `resource_rewrite_disabled: true`; a +destination on a private network or a name that cannot be resolved is replaced by an +authenticated `/resource/...` link, so the internal address never reaches the client. This holds in both modes, and it is +what makes images hosted on a local media server work without publishing their address. Destinations local to the Tuliprox +host โ€” loopback, link-local (including cloud metadata endpoints) โ€” are dropped from the output and refused by the resource +route, because they can only reach Tuliprox itself. The resource route checks that while the connection is built as well, +so a name cannot resolve to a public address during the check and to a local one afterwards. + +**Player metadata follows the resource setting independently of stream redirects.** The `tvg-logo`, cover and backdrop +fields of M3U and Xtream output use Tuliprox links for private destinations even when stream URLs are redirected. +`mask_redirect_url` controls stream URLs; keep it enabled when clients should not see the original stream destination. + > **Note:** You can generate a random secret using: ```bash diff --git a/docs/src/configuration/source.md b/docs/src/configuration/source.md index 6f1458d66..06a4800c2 100644 --- a/docs/src/configuration/source.md +++ b/docs/src/configuration/source.md @@ -144,7 +144,6 @@ inputs: | `aliases` | List | No | | Connection pooling / Sub-accounts (see [below](#input-subsections-object-keys)). | | `staged` | Object | No | | Staged overlay settings. Only valid when `type: staged` (see [below](#input-subsections-object-keys)). | | `panel_api` | Object | No | | Automated reseller account generation (see [below](#input-subsections-object-keys)). | -| `resource_policy` | Object | No | | Trusted private destinations for resource URLs supplied by this input (see [below](#27-resource-policy-resource_policy)). | #### Minimal Stalker Input Example @@ -208,15 +207,14 @@ logic. ### Input Subsections (Object Keys) -| Block | Description | Link | -| :---------------- | :------------------------------------------------------------------------- | :--------------------------------------------------------- | -| `headers` | Custom HTTP request headers for playlist and EPG downloads. | [See Headers](#21-headers-headers) | -| `options` | Behavior controls for metadata resolution, stream probing, and skip logic. | [See Options](#22-input-options-options) | -| `epg` | XMLTV source management and Smart Match fuzzy logic settings. | [See EPG](#23-epg-assignment--smart-match-epg) | -| `aliases` | Connection pooling for multiple subscriptions from the same provider. | [See Aliases](#24-provider-aliases-aliases--batch) | -| `staged` | Overlay settings for first-class staged inputs. | [See Staged](#25-staged-sources-staged) | -| `panel_api` | Automated reseller panel integration (provisioning/renewal). | [See Panel API](#26-provider-panel-api-panel_api) | -| `resource_policy` | Trusted private destinations for resource URLs supplied by this input. | [See Resource Policy](#27-resource-policy-resource_policy) | +| Block | Description | Link | +|:------------|:---------------------------------------------------------------------------|:---------------------------------------------------| +| `headers` | Custom HTTP request headers for playlist and EPG downloads. | [See Headers](#21-headers-headers) | +| `options` | Behavior controls for metadata resolution, stream probing, and skip logic. | [See Options](#22-input-options-options) | +| `epg` | XMLTV source management and Smart Match fuzzy logic settings. | [See EPG](#23-epg-assignment--smart-match-epg) | +| `aliases` | Connection pooling for multiple subscriptions from the same provider. | [See Aliases](#24-provider-aliases-aliases--batch) | +| `staged` | Overlay settings for first-class staged inputs. | [See Staged](#25-staged-sources-staged) | +| `panel_api` | Automated reseller panel integration (provisioning/renewal). | [See Panel API](#26-provider-panel-api-panel_api) | --- @@ -1181,104 +1179,6 @@ Tuliprox processes all Panel API responses as JSON and strictly requires `status --- -### 2.7 Resource Policy (`resource_policy`) - -Restricts which destinations Tuliprox may reach when it proxies a resource URL that came from this input, including -channel logos, EPG channel and programme icons, covers, posters, episode images, backdrops, and nested Xtream -metadata resources. - -Without a policy, every resource URL is treated as **public-only**: only publicly routable destinations are -fetched. An internal logo host therefore stops working until it is listed here. - -```yaml -inputs: - - name: local-playlist - type: m3u - url: /data/local-playlist.m3u - resource_policy: - allowed_hosts: - - media.home.arpa - allowed_networks: - - 192.168.50.20/32 -``` - -#### Configure it in the Web UI - -1. Open the **Source Editor** and select the input that supplies the resource. -2. Open the **Resource Policy** page using the shield icon. -3. Add the exact DNS names under **Allowed Resource Hosts**. Enter host names only, without a scheme, path, port, - wildcard, or IP address. -4. Add the smallest required private CIDR ranges under **Allowed Resource Networks**. -5. Apply the input changes and save the source configuration. - -For a private DNS destination, configure both a matching host and network. A private IP literal needs only a -matching network. Removing every host and network removes the policy from the input and restores the public-only -default. - -#### Parameters - -| Parameter | Type | Required | Default | Technical Impact & Background | -| :--------------------- | :--- | :------: | :------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| **`allowed_hosts`** | List | No | `[]` | Exact DNS names that may resolve to a private address. Matched case-insensitively and without a trailing dot. Wildcards, ports, schemes, paths, and IP literals are rejected while the configuration is loaded. | -| **`allowed_networks`** | List | No | `[]` | Private CIDR ranges a destination address may fall into (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `fc00::/7` and subnets of them). Anything broader, and every public or special-use range, is rejected while the configuration is loaded. Use `/32`, `/128`, or the smallest practical subnet. | - -#### What a policy authorizes - -* A **publicly routable** destination is always allowed, with or without a policy. -* A **private DNS destination** requires both: the exact host name in `allowed_hosts` **and** the resolved address - inside `allowed_networks`. Listing one without the other authorizes nothing. -* A **private IP literal** in the URL (for example `http://192.168.1.1/logo.png`) is authorized by - `allowed_networks` alone, because a literal has no host name to match. -* Loopback, link-local, cloud-metadata, unspecified, multicast, CGNAT, documentation, benchmarking, broadcast, and - reserved addresses are always rejected, even when a matching CIDR is configured. -* Redirects are re-checked on every hop with the same policy, and redirect hops are bounded. -* `allowed_hosts` and `allowed_networks` contain no port, so an authorized host is reachable on **every** port that - serves `http` or `https`. This is deliberate: providers serve images on non-standard ports, and the address - policy is the actual restriction, not the port. - -#### Where the policy comes from - -The policy is looked up on the input that supplied the concrete resource URL, not on the target, the item, or the -record that contains it. Two consequences matter in practice: - -* Items delivered through an alias are authorized with the policy of their main input; aliases inherit it. -* `logo_override: true` copies an EPG icon into a playlist logo. That logo keeps the EPG input as its origin and is - authorized with the **EPG input's** policy, not the playlist input's. - -The canonical input name is the authorization identity of a resource origin. Configured input and alias names are -non-empty, globally unique strings: no two inputs โ€” and no input and alias โ€” may share a name, and the loader rejects -a configuration that does. Internal numeric IDs and generated playlist UUIDs are managed separately from these names. -Renaming an input invalidates the links that were issued for it, and reusing a name for a different input would hand -it the authority of the old input, so treat a rename as a new identity. - -#### Legacy data and links - -Legacy raw resources stored on playlist and Xtream items use the containing item's input. Legacy EPG resources and -external links without an authoritative input remain public-only until the data is regenerated. An origin that -names an input which no longer exists or is disabled is rejected with `400`; it is never silently downgraded to -public-only. - -Tuliprox stores ownership internally in a value beginning with `resource://`. This scheme is reserved and must not -appear in provider data, playlists, EPG documents, metadata, or mapping configuration. Provider-supplied and mapped -values using the reserved scheme are discarded so they cannot claim another input's network policy. External M3U, -XMLTV, Xtream, and Web UI responses always contain either the original HTTP(S) URL or a Tuliprox proxy URL, never the -internal representation. - -#### Proxy interaction - -Resource proxying always connects directly, so the destination policy can be enforced while the connection is -established. A configured proxy, and the `HTTP_PROXY` / `HTTPS_PROXY` / `ALL_PROXY` environment variables, are -ignored for these requests; Tuliprox logs a warning at startup and on reload when one is configured. Provider -fetches, playlist and EPG downloads, and streams keep using the proxy. - -#### Cache - -Proxied resources are cached per policy scope. An entry fetched under one policy is never served to a request -authorized by another, and the resource cache starts cold once on upgrade because the cache key layout changes. -The cache size limits are unchanged. - ---- - ## 3. Routing & Targets (`sources`) This block links your inputs to one or more output targets and defines how Tuliprox transforms, filters, sorts, and diff --git a/frontend/public/assets/i18n/ar.json b/frontend/public/assets/i18n/ar.json index aa68d028e..69071fd46 100644 --- a/frontend/public/assets/i18n/ar.json +++ b/frontend/public/assets/i18n/ar.json @@ -493,8 +493,6 @@ }, "INPUT_FORM": { "ALIASES": "ุฃุณู…ุงุก ู…ุณุชุนุงุฑุฉ ุงุฎุชูŠุงุฑูŠุฉ ุชูุณุชุฎุฏู… ู„ุฅุฏุงุฑุฉ ุฎุทูˆุท ู…ุชุนุฏุฏุฉ ู…ู† ู†ูุณ ุงู„ู…ุฒูˆุฏ.", - "RESOURCE_POLICY_ALLOWED_HOSTS": "ุฃุณู…ุงุก DNS ุงู„ุฏู‚ูŠู‚ุฉ ุงู„ู…ุณู…ูˆุญ ู„ู‡ุง ุจุงู„ุชุญูˆูŠู„ ุฅู„ู‰ ุนู†ูˆุงู† ุฎุงุต. ุฃุฏุฎู„ ุงุณู… ุงู„ู…ุถูŠู ูู‚ุท ุฏูˆู† ู…ุฎุทุท ุฃูˆ ู…ุณุงุฑ ุฃูˆ ู…ู†ูุฐ ุฃูˆ ุญุฑู ุจุฏู„ ุฃูˆ ุนู†ูˆุงู† IP. ูŠุชุทู„ุจ ู…ูˆุฑุฏ DNS ุงู„ุฎุงุต ุชุทุงุจู‚ ุงู„ู…ุถูŠู ูˆุงู„ุดุจูƒุฉ ู…ุนู‹ุง.", - "RESOURCE_POLICY_ALLOWED_NETWORKS": "ู†ุทุงู‚ุงุช CIDR ุงู„ุฎุงุตุฉ ุงู„ู…ุณู…ูˆุญ ุจู‡ุง ู„ู…ูˆุงุฑุฏ ู‡ุฐุง ุงู„ุฅุฏุฎุงู„. ุชูู‚ุจู„ ูู‚ุท ุงู„ุดุจูƒุงุช ุงู„ูุฑุนูŠุฉ ู…ู† 10.0.0.0/8 ูˆ172.16.0.0/12 ูˆ192.168.0.0/16 ูˆfc00::/7. ุงุณุชุฎุฏู… ุฃุตุบุฑ ุดุจูƒุฉ ูุฑุนูŠุฉ ุนู…ู„ูŠุฉ.", "EPG_SOURCES": "ู…ุตุงุฏุฑ EPG (ุฏู„ูŠู„ ุงู„ุจุฑุงู…ุฌ ุงู„ุฅู„ูƒุชุฑูˆู†ูŠ) ุงู„ู…ุฑุชุจุทุฉ ุจู‡ุฐุง ุงู„ุฅุฏุฎุงู„.\n ูŠู…ูƒู†ูƒ ุงุณุชุฎุฏุงู… `auto` ู„ุฅู†ุดุงุก ุนู†ูˆุงู† URL ู„ู€ EPG ุจู†ุงุกู‹ ุนู„ู‰ ุฅุนุฏุงุฏุงุช ุงู„ุฅุฏุฎุงู„ ุงู„ุฎุงุตุฉ ุจูƒุŒ\nุฃูˆ ูŠู…ูƒู†ูƒ ูƒุชุงุจุฉ ุนู†ูˆุงู† URL ุงู„ูƒุงู…ู„ ู„ู€ EPG.", "FETCH_METHOD": "ุทุฑูŠู‚ุฉ HTTP ุงู„ู…ุณุชุฎุฏู…ุฉ ู„ุงุณุชุฑุฏุงุฏ ุงู„ุฅุฏุฎุงู„ (ู…ุซู„ GETุŒ POST).", "HEADERS": "ุฑุคูˆุณ HTTP ุงู„ู…ุฎุตุตุฉ ุงู„ู…ูุฑุณู„ุฉ ุนู†ุฏ ุงุณุชุฑุฏุงุฏ ู‚ุงุฆู…ุฉ ุชุดุบูŠู„ ุงู„ุฅุฏุฎุงู„ ู‡ุฐู‡.", @@ -916,7 +914,6 @@ "ADD_FORMAT": "ุฅุถุงูุฉ ุชู†ุณูŠู‚", "ADD_HEADER": "ุฅุถุงูุฉ ุฑุฃุณ", "ADD_COUNTRY": "ุฅุถุงูุฉ ุจู„ุฏ", - "ADD_HOST": "ุฅุถุงูุฉ ู…ุถูŠู", "ADD_NETWORK": "ุฅุถุงูุฉ ุดุจูƒุฉ", "ADD_MAPPING": "ุฅุถุงูุฉ ุชุนูŠูŠู†", "ADD_PATTERN": "ุฅุถุงูุฉ ู†ู…ุท", @@ -949,7 +946,6 @@ "ADD_VALUE": "ุฅุถุงูุฉ ู‚ูŠู…ุฉ", "AUTOMATIC_CODEC_TRIGGER_POLICY": "ุณูŠุงุณุฉ ุงู„ุชุดุบูŠู„ ุงู„ุชู„ู‚ุงุฆูŠ ู„ู„ุชุฑู…ูŠุฒ", "ALLOWED_COUNTRIES": "ุงู„ุจู„ุฏุงู† ุงู„ู…ุณู…ูˆุญ ุจู‡ุง", - "ALLOWED_HOSTS": "ุงู„ู…ุถูŠููˆู† ุงู„ู…ุณู…ูˆุญ ุจู‡ู…", "ALLOWED_NETWORKS": "ุงู„ุดุจูƒุงุช ุงู„ู…ุณู…ูˆุญ ุจู‡ุง", "ACCOUNT_TOKEN": "ุฑู…ุฒ ุงู„ุญุณุงุจ", "ALLOW_RELAY": "ุงู„ุณู…ุงุญ ุจุงู„ุชุฑุญูŠู„", @@ -1448,9 +1444,6 @@ "RESOLVE_FILTER": "ู…ุฑุดุญ ุงู„ุญู„", "RESOLVE_TMDB": "ุญู„ TMDB", "RESOURCE_IMAGE_CACHE": "ุฐุงูƒุฑุฉ ุงู„ุชุฎุฒูŠู† ุงู„ู…ุคู‚ุช ู„ู„ู…ูˆุงุฑุฏ/ุงู„ุตูˆุฑ", - "RESOURCE_ALLOWED_HOSTS": "ู…ุถูŠููˆ ุงู„ู…ูˆุงุฑุฏ ุงู„ู…ุณู…ูˆุญ ุจู‡ู…", - "RESOURCE_ALLOWED_NETWORKS": "ุดุจูƒุงุช ุงู„ู…ูˆุงุฑุฏ ุงู„ู…ุณู…ูˆุญ ุจู‡ุง", - "RESOURCE_POLICY": "ุณูŠุงุณุฉ ุงู„ู…ูˆุงุฑุฏ", "RESOURCE_RETRY": "ุฅุนุงุฏุฉ ู…ุญุงูˆู„ุฉ ุงู„ู…ูˆุฑุฏ", "RESOURCE_REWRITE_DISABLE": "ุชุนุทูŠู„ ุฅุนุงุฏุฉ ูƒุชุงุจุฉ ุงู„ู…ูˆุฑุฏ", "RESOURCE_REWRITE_DISABLED": "ุฅุนุงุฏุฉ ูƒุชุงุจุฉ ุงู„ู…ูˆุฑุฏ ู…ุนุทู„ุฉ", diff --git a/frontend/public/assets/i18n/en.json b/frontend/public/assets/i18n/en.json index 1cd4a15be..4374f8c4f 100644 --- a/frontend/public/assets/i18n/en.json +++ b/frontend/public/assets/i18n/en.json @@ -497,8 +497,6 @@ }, "INPUT_FORM": { "ALIASES": "Optional aliases used to manage multiple lines from the same provider.", - "RESOURCE_POLICY_ALLOWED_HOSTS": "Exact DNS names allowed to resolve to a private address. Enter host names only, without scheme, path, port, wildcard, or IP address. A private DNS resource requires both a matching host and network.", - "RESOURCE_POLICY_ALLOWED_NETWORKS": "Private CIDR ranges allowed for resources from this input. Only subnets of 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, and fc00::/7 are accepted. Use the smallest practical subnet.", "EPG_SOURCES": "Associated EPG (Electronic Program Guide) sources for this input.\n You can use `auto` to create the epg url based on your input settings,\nor you can write the full epg url.", "FETCH_METHOD": "HTTP method used to retrieve the input (e.g., GET, POST).", "HEADERS": "Custom HTTP headers sent when retrieving the input playlist.", @@ -918,7 +916,6 @@ "ADD_FORMAT": "Add Format", "ADD_HEADER": "Add header", "ADD_COUNTRY": "Add country", - "ADD_HOST": "Add host", "ADD_NETWORK": "Add network", "ADD_MAPPING": "Add Mapping", "ADD_PATTERN": "Add Pattern", @@ -952,7 +949,6 @@ "ADD_VALUE": "Add value", "AUTOMATIC_CODEC_TRIGGER_POLICY": "automatic codec trigger policy", "ALLOWED_COUNTRIES": "Allowed Countries", - "ALLOWED_HOSTS": "Allowed Hosts", "ALLOWED_NETWORKS": "Allowed Networks", "ACCOUNT_TOKEN": "Account token", "ALLOW_RELAY": "Allow relay", @@ -1533,9 +1529,6 @@ "RESOLVE_FILTER": "Resolve Filter", "RESOLVE_TMDB": "Resolve TMDB", "RESOURCE_IMAGE_CACHE": "Resource/Image Cache", - "RESOURCE_ALLOWED_HOSTS": "Allowed Resource Hosts", - "RESOURCE_ALLOWED_NETWORKS": "Allowed Resource Networks", - "RESOURCE_POLICY": "Resource Policy", "RESOURCE_RETRY": "Resource Retry", "RESOURCE_REWRITE_DISABLE": "Resource Rewrite disable", "RESOURCE_REWRITE_DISABLED": "Resource Rewrite Disabled", diff --git a/frontend/public/assets/i18n/ru.json b/frontend/public/assets/i18n/ru.json index 28775700a..5969d0af4 100644 --- a/frontend/public/assets/i18n/ru.json +++ b/frontend/public/assets/i18n/ru.json @@ -469,8 +469,6 @@ }, "INPUT_FORM": { "ALIASES": "ะะตะพะฑัะทะฐั‚ะตะปัŒะฝั‹ะต ะฟัะตะฒะดะพะฝะธะผั‹, ะธัะฟะพะปัŒะทัƒะตะผั‹ะต ะดะปั ัƒะฟั€ะฐะฒะปะตะฝะธั ะฝะตัะบะพะปัŒะบะธะผะธ ะปะธะฝะธัะผะธ ะพั‚ ะพะดะฝะพะณะพ ะฟั€ะพะฒะฐะนะดะตั€ะฐ.", - "RESOURCE_POLICY_ALLOWED_HOSTS": "ะขะพั‡ะฝั‹ะต DNS-ะธะผะตะฝะฐ, ะบะพั‚ะพั€ั‹ะผ ั€ะฐะทั€ะตัˆะตะฝะพ ั€ะฐะทั€ะตัˆะฐั‚ัŒัั ะฒ ั‡ะฐัั‚ะฝั‹ะน ะฐะดั€ะตั. ะฃะบะฐะทั‹ะฒะฐะนั‚ะต ั‚ะพะปัŒะบะพ ะธะผั ัƒะทะปะฐ ะฑะตะท ัั…ะตะผั‹, ะฟัƒั‚ะธ, ะฟะพั€ั‚ะฐ, ัˆะฐะฑะปะพะฝะฐ ะธะปะธ IP-ะฐะดั€ะตัะฐ. ะ”ะปั ั‡ะฐัั‚ะฝะพะณะพ DNS-ั€ะตััƒั€ัะฐ ะดะพะปะถะฝั‹ ัะพะฒะฟะฐะดะฐั‚ัŒ ะธ ัƒะทะตะป, ะธ ัะตั‚ัŒ.", - "RESOURCE_POLICY_ALLOWED_NETWORKS": "ะงะฐัั‚ะฝั‹ะต CIDR-ัะตั‚ะธ, ั€ะฐะทั€ะตัˆั‘ะฝะฝั‹ะต ะดะปั ั€ะตััƒั€ัะพะฒ ัั‚ะพะณะพ ะฒั…ะพะดะฐ. ะ”ะพะฟัƒัั‚ะธะผั‹ ั‚ะพะปัŒะบะพ ะฟะพะดัะตั‚ะธ 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 ะธ fc00::/7. ะ˜ัะฟะพะปัŒะทัƒะนั‚ะต ะผะธะฝะธะผะฐะปัŒะฝะพ ะฝะตะพะฑั…ะพะดะธะผัƒัŽ ะฟะพะดัะตั‚ัŒ.", "EPG_SOURCES": "ะกะฒัะทะฐะฝะฝั‹ะต ะธัั‚ะพั‡ะฝะธะบะธ EPG (Electronic Program Guide) ะดะปั ัั‚ะพะณะพ ะฒะฒะพะดะฐ.\n ะ’ั‹ ะผะพะถะตั‚ะต ะธัะฟะพะปัŒะทะพะฒะฐั‚ัŒ `auto` ะดะปั ัะพะทะดะฐะฝะธั URL epg ะฝะฐ ะพัะฝะพะฒะต ะฒะฐัˆะธั… ะฝะฐัั‚ั€ะพะตะบ ะฒะฒะพะดะฐ,\nะธะปะธ ะฒั‹ ะผะพะถะตั‚ะต ะฝะฐะฟะธัะฐั‚ัŒ ะฟะพะปะฝั‹ะน URL epg.", "FETCH_METHOD": "HTTP-ะผะตั‚ะพะด, ะธัะฟะพะปัŒะทัƒะตะผั‹ะน ะดะปั ะฟะพะปัƒั‡ะตะฝะธั ะฒะฒะพะดะฐ (ะฝะฐะฟั€ะธะผะตั€, GET, POST).", "HEADERS": "ะŸะพะปัŒะทะพะฒะฐั‚ะตะปัŒัะบะธะต HTTP-ะทะฐะณะพะปะพะฒะบะธ, ะพั‚ะฟั€ะฐะฒะปัะตะผั‹ะต ะฟั€ะธ ะฟะพะปัƒั‡ะตะฝะธะธ ะฟะปะตะนะปะธัั‚ะฐ ะฒะฒะพะดะฐ.", @@ -879,7 +877,6 @@ "ADD_FORMAT": "ะ”ะพะฑะฐะฒะธั‚ัŒ ั„ะพั€ะผะฐั‚", "ADD_HEADER": "ะ”ะพะฑะฐะฒะธั‚ัŒ ะทะฐะณะพะปะพะฒะพะบ", "ADD_COUNTRY": "ะ”ะพะฑะฐะฒะธั‚ัŒ ัั‚ั€ะฐะฝัƒ", - "ADD_HOST": "ะ”ะพะฑะฐะฒะธั‚ัŒ ัƒะทะตะป", "ADD_NETWORK": "ะ”ะพะฑะฐะฒะธั‚ัŒ ัะตั‚ัŒ", "ADD_MAPPING": "ะ”ะพะฑะฐะฒะธั‚ัŒ ัะพะฟะพัั‚ะฐะฒะปะตะฝะธะต", "ADD_PATTERN": "ะ”ะพะฑะฐะฒะธั‚ัŒ ัˆะฐะฑะปะพะฝ", @@ -911,7 +908,6 @@ "ADD_VALUE": "ะ”ะพะฑะฐะฒะธั‚ัŒ ะทะฝะฐั‡ะตะฝะธะต", "AUTOMATIC_CODEC_TRIGGER_POLICY": "ะฟะพะปะธั‚ะธะบะฐ ะฐะฒั‚ะพะผะฐั‚ะธั‡ะตัะบะพะณะพ ะทะฐะฟัƒัะบะฐ ะบะพะดะตะบะฐ", "ALLOWED_COUNTRIES": "ะ ะฐะทั€ะตัˆะตะฝะฝั‹ะต ัั‚ั€ะฐะฝั‹", - "ALLOWED_HOSTS": "ะ ะฐะทั€ะตัˆั‘ะฝะฝั‹ะต ัƒะทะปั‹", "ALLOWED_NETWORKS": "ะ ะฐะทั€ะตัˆะตะฝะฝั‹ะต ัะตั‚ะธ", "API": "API", "API_CONFIG": "API", @@ -1388,9 +1384,6 @@ "RESOLVE_DELAY_SEC": "ะ—ะฐะดะตั€ะถะบะฐ ั€ะฐะทั€ะตัˆะตะฝะธั (ัะตะบ)", "RESOLVE_FILTER": "ะคะธะปัŒั‚ั€ ั€ะฐะทั€ะตัˆะตะฝะธั", "RESOLVE_TMDB": "ะ ะฐะทั€ะตัˆะฐั‚ัŒ TMDB", - "RESOURCE_ALLOWED_HOSTS": "ะ ะฐะทั€ะตัˆั‘ะฝะฝั‹ะต ัƒะทะปั‹ ั€ะตััƒั€ัะพะฒ", - "RESOURCE_ALLOWED_NETWORKS": "ะ ะฐะทั€ะตัˆั‘ะฝะฝั‹ะต ัะตั‚ะธ ั€ะตััƒั€ัะพะฒ", - "RESOURCE_POLICY": "ะŸะพะปะธั‚ะธะบะฐ ั€ะตััƒั€ัะพะฒ", "RESOURCE_RETRY": "ะŸะพะฒั‚ะพั€ ั€ะตััƒั€ัะฐ", "RESOURCE_REWRITE_DISABLE": "ะžั‚ะบะปัŽั‡ะธั‚ัŒ ะฟะตั€ะตะฟะธัั‹ะฒะฐะฝะธะต ั€ะตััƒั€ัะพะฒ", "RESOURCE_REWRITE_DISABLED": "ะŸะตั€ะตะฟะธัั‹ะฒะฐะฝะธะต ั€ะตััƒั€ัะพะฒ ะพั‚ะบะปัŽั‡ะตะฝะพ", diff --git a/frontend/src/app/components/source_editor/input_form.rs b/frontend/src/app/components/source_editor/input_form.rs index 05fc316d6..9ae4e3e8c 100644 --- a/frontend/src/app/components/source_editor/input_form.rs +++ b/frontend/src/app/components/source_editor/input_form.rs @@ -22,8 +22,7 @@ use crate::{ app::{ components::{ config::HasFormData, AliasItemForm, BlockId, BlockInstance, Card, EditMode, EpgSmartMatchForm, - EpgSourceItemForm, IconButton, Panel, ProviderItemForm, SourceEditorContext, Tag, TagList, TextButton, - TitledCard, + EpgSourceItemForm, IconButton, Panel, ProviderItemForm, SourceEditorContext, TextButton, }, ConfigContext, }, @@ -37,7 +36,7 @@ use shared::{ ConfigInputAliasDto, ConfigInputDto, ConfigInputOptionsDto, ConfigInputStagedDto, ConfigInputUpdateQualityDto, ConfigProviderDto, EpgSmartMatchConfigDto, EpgSourceDto, InputFetchMethod, InputType, MediaServerInputConfigDto, MediaServerLibrarySelector, OnConnectErrorPolicy, ProviderUrlSelectionPolicy, - ResourcePolicyDto, StagedInputType, StalkerDeviceProfileDto, StalkerInputConfigDto, + StagedInputType, StalkerDeviceProfileDto, StalkerInputConfigDto, }, utils::{Internable, BATCH_SCHEME_PREFIX}, }; @@ -121,11 +120,6 @@ const LABEL_SERVER_ID: &str = "LABEL.SERVER_ID"; const LABEL_SERVER_NAME: &str = "LABEL.SERVER_NAME"; const LABEL_PREFER_HTTPS: &str = "LABEL.PREFER_HTTPS"; const LABEL_ALLOW_RELAY: &str = "LABEL.ALLOW_RELAY"; -const LABEL_RESOURCE_POLICY: &str = "LABEL.RESOURCE_POLICY"; -const LABEL_RESOURCE_ALLOWED_HOSTS: &str = "LABEL.RESOURCE_ALLOWED_HOSTS"; -const LABEL_RESOURCE_ALLOWED_NETWORKS: &str = "LABEL.RESOURCE_ALLOWED_NETWORKS"; -const LABEL_ADD_HOST: &str = "LABEL.ADD_HOST"; -const LABEL_ADD_NETWORK: &str = "LABEL.ADD_NETWORK"; fn input_persist_hint_key(staged_input: bool) -> &'static str { if staged_input { @@ -247,7 +241,6 @@ const LABEL_EDIT_EPG_SMART_MATCH: &str = "LABEL.EDIT_EPG_SMART_MATCH"; #[derive(Debug, Copy, Clone, PartialEq, Eq)] enum InputFormPage { Main, - ResourcePolicy, Device, Options, Libraries, @@ -258,7 +251,6 @@ enum InputFormPage { impl InputFormPage { const MAIN: &str = "Main"; - const RESOURCE_POLICY: &str = "ResourcePolicy"; const DEVICE: &str = "Device"; const OPTIONS: &str = "Options"; const LIBRARIES: &str = "Libraries"; @@ -273,7 +265,6 @@ impl FromStr for InputFormPage { fn from_str(s: &str) -> Result { match s { Self::MAIN => Ok(InputFormPage::Main), - Self::RESOURCE_POLICY => Ok(InputFormPage::ResourcePolicy), Self::DEVICE => Ok(InputFormPage::Device), Self::OPTIONS => Ok(InputFormPage::Options), Self::LIBRARIES => Ok(InputFormPage::Libraries), @@ -292,7 +283,6 @@ impl Display for InputFormPage { "{}", match *self { InputFormPage::Main => Self::MAIN, - InputFormPage::ResourcePolicy => Self::RESOURCE_POLICY, InputFormPage::Device => Self::DEVICE, InputFormPage::Options => Self::OPTIONS, InputFormPage::Libraries => Self::LIBRARIES, @@ -308,7 +298,6 @@ impl Internable for InputFormPage { fn intern(self) -> Arc { match self { Self::Main => Self::MAIN, - Self::ResourcePolicy => Self::RESOURCE_POLICY, Self::Device => Self::DEVICE, Self::Options => Self::OPTIONS, Self::Libraries => Self::LIBRARIES, @@ -337,28 +326,9 @@ fn input_form_pages(input_type: shared::model::InputType) -> Vec if !input_type.is_library() && !input_type.is_staged() && !input_type.is_media_server() { pages.extend([InputFormPage::Epg, InputFormPage::Provider]); } - pages.push(InputFormPage::ResourcePolicy); pages } -fn normalized_resource_policy(mut policy: ResourcePolicyDto) -> Option { - fn normalize(values: &mut Vec) { - values.iter_mut().for_each(|value| *value = value.trim().to_string()); - values.retain(|value| !value.is_empty()); - values.dedup(); - } - - normalize(&mut policy.allowed_hosts); - normalize(&mut policy.allowed_networks); - (!policy.is_empty()).then_some(policy) -} - -fn resource_policy_tags(values: &[String]) -> Vec> { - values.iter().map(|value| Rc::new(Tag { label: value.clone(), class: None })).collect() -} - -fn tag_values(tags: Vec>) -> Vec { tags.into_iter().map(|tag| tag.label.clone()).collect() } - fn normalize_optional_device_field(value: &mut Option) { *value = value.take().and_then(|value| { let value = value.trim(); @@ -432,7 +402,6 @@ generate_form_reducer!( MediaServer => media_server: Option, ExpDate => exp_date: Option, CacheDuration => cache_duration: Option, - ResourcePolicy => resource_policy: Option, } ); @@ -927,55 +896,6 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html { } }; - let render_resource_policy = || { - let policy = input_form_state.form.resource_policy.clone().unwrap_or_default(); - let allowed_hosts = resource_policy_tags(&policy.allowed_hosts); - let allowed_networks = resource_policy_tags(&policy.allowed_networks); - let handle_hosts_change = { - let state = input_form_state.clone(); - Callback::from(move |tags: Vec>| { - let mut policy = state.form.resource_policy.clone().unwrap_or_default(); - policy.allowed_hosts = tag_values(tags); - state.dispatch(ConfigInputFormAction::ResourcePolicy(normalized_resource_policy(policy))); - }) - }; - let handle_networks_change = { - let state = input_form_state.clone(); - Callback::from(move |tags: Vec>| { - let mut policy = state.form.resource_policy.clone().unwrap_or_default(); - policy.allowed_networks = tag_values(tags); - state.dispatch(ConfigInputFormAction::ResourcePolicy(normalized_resource_policy(policy))); - }) - }; - - html! { - - - { config_field_child!(translate.t(LABEL_RESOURCE_ALLOWED_HOSTS), "INPUT_FORM.RESOURCE_POLICY_ALLOWED_HOSTS", { - html! { - - } - })} - { config_field_child!(translate.t(LABEL_RESOURCE_ALLOWED_NETWORKS), "INPUT_FORM.RESOURCE_POLICY_ALLOWED_NETWORKS", { - html! { - - } - })} - - - } - }; - let render_alias = || { let aliases = aliases_state.clone(); let show_alias_form = show_alias_form_state.clone(); @@ -1331,7 +1251,6 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html { Callback::from(move |_| { let mut input = input_form_state.data().clone(); - input.resource_policy = input.resource_policy.take().and_then(normalized_resource_policy); let options = input_options_state.data(); input.options = if options.is_empty() { None } else { Some(options.clone()) }; @@ -1413,9 +1332,6 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html { {render_main()} - - {render_resource_policy()} - { html_if!(stalker_input, { @@ -1480,7 +1396,6 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html { })} - } }; @@ -1519,32 +1434,6 @@ mod tests { assert_eq!(InputFormPage::Libraries.to_string(), InputFormPage::LIBRARIES); } - #[test] - fn resource_policy_page_is_available_for_every_input_family() { - assert_eq!(InputFormPage::from_str(InputFormPage::RESOURCE_POLICY).ok(), Some(InputFormPage::ResourcePolicy)); - assert_eq!(InputFormPage::ResourcePolicy.to_string(), InputFormPage::RESOURCE_POLICY); - for input_type in [InputType::M3u, InputType::Xtream, InputType::Stalker, InputType::Library, InputType::Plex] { - assert!(input_form_pages(input_type).contains(&InputFormPage::ResourcePolicy)); - } - } - - #[test] - fn resource_policy_materialization_trims_values_and_omits_empty_policy() { - let policy = ResourcePolicyDto { - allowed_hosts: vec![" media.home.arpa ".to_string(), String::new()], - allowed_networks: vec![" 192.168.50.0/24 ".to_string()], - }; - - assert_eq!( - normalized_resource_policy(policy), - Some(ResourcePolicyDto { - allowed_hosts: vec!["media.home.arpa".to_string()], - allowed_networks: vec!["192.168.50.0/24".to_string()], - }) - ); - assert_eq!(normalized_resource_policy(ResourcePolicyDto::default()), None); - } - #[test] fn stalker_device_page_is_directly_after_main() { assert_eq!(InputFormPage::from_str(InputFormPage::DEVICE).ok(), Some(InputFormPage::Device)); @@ -1659,37 +1548,6 @@ mod tests { Ok(()) } - #[test] - fn resource_policy_translations_exist_in_every_frontend_locale() -> Result<(), serde_json::Error> { - let locales = [ - ("en", include_str!("../../../../public/assets/i18n/en.json")), - ("ru", include_str!("../../../../public/assets/i18n/ru.json")), - ("ar", include_str!("../../../../public/assets/i18n/ar.json")), - ]; - - for (locale, source) in locales { - let translations: serde_json::Value = serde_json::from_str(source)?; - for pointer in [ - "/LABEL/RESOURCE_POLICY", - "/LABEL/RESOURCE_ALLOWED_HOSTS", - "/LABEL/RESOURCE_ALLOWED_NETWORKS", - "/LABEL/ADD_HOST", - "/LABEL/ADD_NETWORK", - "/EXPLANATION/INPUT_FORM/RESOURCE_POLICY_ALLOWED_HOSTS", - "/EXPLANATION/INPUT_FORM/RESOURCE_POLICY_ALLOWED_NETWORKS", - ] { - assert!( - translations - .pointer(pointer) - .and_then(serde_json::Value::as_str) - .is_some_and(|value| !value.is_empty()), - "missing {pointer} translation for {locale}" - ); - } - } - Ok(()) - } - #[test] fn libraries_from_text_preserves_existing_detailed_selector() { let detailed = MediaServerLibrarySelector::Detailed(MediaServerLibrarySelectorDetailsDto { diff --git a/shared/Cargo.toml b/shared/Cargo.toml index 422c2aaab..6330daa64 100644 --- a/shared/Cargo.toml +++ b/shared/Cargo.toml @@ -8,12 +8,13 @@ rust-version.workspace = true workspace = true [dependencies] +aes = "0.8.4" bitflags = "2.13.1" ciborium = "0.2.2" hex = "0.4.3" serde_tuple = "1.1.3" -rmp-serde = "1.3.1" serde.workspace = true +rmp-serde = "1.3.1" regex.workspace = true log.workspace = true url.workspace = true @@ -27,6 +28,7 @@ fastrand = { workspace = true, features = ["js"] } zeroize.workspace = true chrono.workspace = true chrono-tz = "0.10.4" +ctr = "0.9.2" bytes.workspace = true lz4_flex.workspace = true brotli.workspace = true @@ -43,5 +45,3 @@ js-sys = "0.3.104" [target.'cfg(not(target_arch = "wasm32"))'.dependencies] getrandom = "0.4.3" - -[dev-dependencies] diff --git a/shared/src/model/config/input.rs b/shared/src/model/config/input.rs index 9cc6a3eac..1eb3ae0c9 100644 --- a/shared/src/model/config/input.rs +++ b/shared/src/model/config/input.rs @@ -1,4 +1,4 @@ -use super::{PanelApiConfigDto, ResourcePolicyDto}; +use super::PanelApiConfigDto; use crate::{ check_input_connections, check_input_credentials, defaults::{ @@ -524,10 +524,6 @@ pub struct ConfigInputDto { /// is `Stalker` or `StalkerBatch`. #[serde(default, skip_serializing_if = "Option::is_none")] pub stalker: Option, - /// Trusted private destinations for resource URLs supplied by this input. Absent means - /// public destinations only. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub resource_policy: Option, } impl Default for ConfigInputDto { @@ -558,7 +554,6 @@ impl Default for ConfigInputDto { panel_api: None, provider: None, stalker: None, - resource_policy: None, } } } diff --git a/shared/src/model/config/mod.rs b/shared/src/model/config/mod.rs index 050968347..ce403dbc6 100644 --- a/shared/src/model/config/mod.rs +++ b/shared/src/model/config/mod.rs @@ -32,7 +32,6 @@ mod proxy; mod proxy_type; mod rate_limit; mod rename; -mod resource_policy; mod reverse_proxy; mod schedule; mod sort; @@ -97,7 +96,6 @@ pub use proxy_user_status::*; pub use qos_aggregation::*; pub use rate_limit::*; pub use rename::*; -pub use resource_policy::*; pub use reverse_proxy::*; pub use schedule::*; pub use sort::*; diff --git a/shared/src/model/config/resource_policy.rs b/shared/src/model/config/resource_policy.rs deleted file mode 100644 index 8fd2c25fd..000000000 --- a/shared/src/model/config/resource_policy.rs +++ /dev/null @@ -1,19 +0,0 @@ -use serde::{Deserialize, Serialize}; - -/// Trusted private destinations for the resource URLs supplied by one input. -/// -/// `allowed_hosts` holds exact DNS names, `allowed_networks` holds private CIDR ranges. Both -/// empty (or an absent policy) means public destinations only, which is the default for every -/// input. A private target is authorized only when the host name *and* the resolved address are -/// covered, so enabling one private logo host cannot open a whole subnet. -#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct ResourcePolicyDto { - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub allowed_hosts: Vec, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub allowed_networks: Vec, -} - -impl ResourcePolicyDto { - pub fn is_empty(&self) -> bool { self.allowed_hosts.is_empty() && self.allowed_networks.is_empty() } -} diff --git a/shared/src/model/epg.rs b/shared/src/model/epg.rs index 8f1ef7c5b..f5f79f4d2 100644 --- a/shared/src/model/epg.rs +++ b/shared/src/model/epg.rs @@ -275,75 +275,3 @@ pub struct EpgProgrammeDto { pub stop: String, pub title: String, } - -/// Append-only compatibility of the persisted EPG records. -/// -/// The B+Tree encodes these structs positionally, so the origin fields are appended at the end of -/// the field list and an EPG database written by an earlier version must still load. -#[cfg(test)] -mod persistence_compatibility { - use super::{EpgCategory, EpgChannel, EpgProgramme}; - use crate::utils::Internable; - use serde::Serialize; - use std::sync::Arc; - - /// The `EpgChannel` field list without the appended origin field. - #[derive(Serialize)] - struct LegacyChannel { - id: Arc, - title: Option>, - icon: Option>, - programmes: Vec, - } - - /// The `EpgProgramme` field list without the appended origin field. - #[derive(Serialize)] - struct LegacyProgramme { - start: i64, - stop: i64, - title: Option>, - desc: Option>, - catchup_id: Option>, - categories: Vec, - is_live: bool, - is_new: bool, - previously_shown: bool, - icon: Option>, - } - - #[test] - fn channel_written_before_origin_tracking_still_decodes() { - let legacy = LegacyChannel { - id: "epg.channel".intern(), - title: Some("Channel".intern()), - icon: Some("https://cdn.example.com/icon.png".intern()), - programmes: Vec::new(), - }; - let bytes = rmp_serde::to_vec(&legacy).expect("encode legacy channel"); - - let decoded: EpgChannel = rmp_serde::from_slice(&bytes).expect("legacy channel still decodes"); - - assert_eq!(decoded.icon.as_deref(), Some("https://cdn.example.com/icon.png")); - } - - #[test] - fn programme_written_before_origin_tracking_still_decodes() { - let legacy = LegacyProgramme { - start: 100, - stop: 200, - title: Some("Title".intern()), - desc: None, - catchup_id: None, - categories: Vec::new(), - is_live: false, - is_new: false, - previously_shown: false, - icon: Some("https://cdn.example.com/programme.png".intern()), - }; - let bytes = rmp_serde::to_vec(&legacy).expect("encode legacy programme"); - - let decoded: EpgProgramme = rmp_serde::from_slice(&bytes).expect("legacy programme still decodes"); - - assert_eq!(decoded.icon.as_deref(), Some("https://cdn.example.com/programme.png")); - } -} diff --git a/shared/src/model/playlist.rs b/shared/src/model/playlist.rs index d094f6f2b..52b69f1fd 100644 --- a/shared/src/model/playlist.rs +++ b/shared/src/model/playlist.rs @@ -8,9 +8,10 @@ use crate::{ }, utils::{ arc_str_option_serde, arc_str_serde, concat_path, extract_extension_from_url, generate_provider_playlist_uuid, - generate_runtime_playlist_uuid, get_provider_id, obfuscate_text, Internable, + generate_runtime_playlist_uuid, get_provider_id, seal_web_ui_resource_url, Internable, }, }; +use dashmap::DashMap; use serde::{Deserialize, Serialize}; use std::{ fmt::{Display, Formatter, Write}, @@ -435,51 +436,6 @@ impl Default for PlaylistItemHeader { } impl PlaylistItemHeader { - pub fn visit_resource_values(&self, visitor: &mut impl FnMut(&Arc)) { - visitor(&self.logo); - visitor(&self.logo_small); - if let Some(properties) = &self.additional_properties { - properties.visit_resource_values(visitor); - } - } - - pub fn visit_resource_values_mut(&mut self, visitor: &mut impl FnMut(&mut Arc)) { - visitor(&mut self.logo); - visitor(&mut self.logo_small); - if let Some(properties) = &mut self.additional_properties { - properties.visit_resource_values_mut(visitor); - } - } - - /// Validates and wraps every resource carried by this item at provider ingress. - pub fn ingest_resource_values(&mut self, input_name: &Arc) -> usize { - let mut rejected = 0; - for value in [&mut self.logo, &mut self.logo_small] { - if crate::model::ingest_resource_value(value, input_name).is_err() { - rejected += 1; - } - } - if let Some(properties) = &mut self.additional_properties { - rejected += properties.ingest_resource_values(input_name); - } - rejected - } - - pub fn normalize_internal_resource_values(&mut self) -> usize { - let input_name = Arc::clone(&self.input_name); - let mut rejected = 0; - for value in [&mut self.logo, &mut self.logo_small] { - if crate::model::normalize_internal_resource_value(value, &input_name).is_err() { - *value = Arc::from(""); - rejected += 1; - } - } - if let Some(properties) = &mut self.additional_properties { - rejected += properties.normalize_internal_resource_values(&input_name); - } - rejected - } - /// Captures the input playlist item ID at the input-processing boundary. /// /// This must run before target transformations and must not be used to recover an identity @@ -537,10 +493,7 @@ macro_rules! to_m3u_non_empty_fields { ($header:expr, $line:expr, $(($prop:ident, $field:expr)),*;) => { $( if !$header.$prop.is_empty() { - let value = crate::model::external_resource_value(&$header.$prop); - if !value.is_empty() { - let _ = write!($line," {}=\"{}\"", $field, value); - } + let _ = write!($line," {}=\"{}\"", $field, $header.$prop ); } )* }; @@ -622,12 +575,7 @@ impl crate::model::FieldGetAccessor for crate::model::PlaylistItemHeader { #[inline] fn get_field(&self, field: &str) -> Option> { use crate::model::FieldGet; - let field = HeaderField::parse(field)?; - match field { - HeaderField::Logo => Some(crate::model::external_resource_value(&self.logo)), - HeaderField::LogoSmall => Some(crate::model::external_resource_value(&self.logo_small)), - _ => self.get(field).map(|value| value.to_arc()), - } + self.get(HeaderField::parse(field)?).map(|value| value.to_arc()) } } @@ -939,7 +887,13 @@ impl crate::model::FieldGetAccessor for M3uPlaylistItem { #[inline] fn get_field(&self, field: &str) -> Option> { use crate::model::FieldGet; - self.get(HeaderField::parse(field)?).map(|value| value.to_arc()) + let field = HeaderField::parse(field)?; + let value = self.get(field)?.to_arc(); + Some(if matches!(field, HeaderField::Logo | HeaderField::LogoSmall) { + super::persisted_resource_arc(&value) + } else { + value + }) } } @@ -966,6 +920,14 @@ pub struct XtreamMappingOptions { pub base_url: String, pub web_ui_request: bool, pub encrypt_secret: [u8; 16], + pub resource_host_policies: Arc>, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ResourceOutputPolicy { + Direct, + Proxy, + Blocked, } impl XtreamMappingOptions { @@ -982,9 +944,10 @@ impl XtreamMappingOptions { xtream_cluster: XtreamCluster, item_type: PlaylistItemType, virtual_id: VirtualId, + force_proxy: bool, ) -> Option { let proxy_resource = self.resource_proxy_item_types.is_set(item_type); - if proxy_resource && self.flags.contains(XtreamMappingFlags::RewriteResourceUrl) { + if force_proxy || (proxy_resource && self.flags.contains(XtreamMappingFlags::RewriteResourceUrl)) { Some(format!( "{}/resource/{}/{}/{}/{}", self.base_url, @@ -998,6 +961,34 @@ impl XtreamMappingOptions { } } + fn resource_output_policy(&self, resource_url: &str) -> ResourceOutputPolicy { + if self.flags.contains(XtreamMappingFlags::RewriteResourceUrl) { + return ResourceOutputPolicy::Proxy; + } + if resource_url.starts_with("media-server://image/") { + return ResourceOutputPolicy::Proxy; + } + let Ok(url) = url::Url::parse(resource_url) else { + return if resource_url.starts_with("//") + || resource_url.starts_with("http:") + || resource_url.starts_with("https:") + { + ResourceOutputPolicy::Blocked + } else { + ResourceOutputPolicy::Direct + }; + }; + if !matches!(url.scheme(), "http" | "https") { + return ResourceOutputPolicy::Blocked; + } + let Some(host) = url.host_str() else { + return ResourceOutputPolicy::Blocked; + }; + // A host missing from the prepared set still needs a proxy link; a new resource + // field must not expose its origin before the output classifier knows the host. + self.resource_host_policies.get(host).map_or(ResourceOutputPolicy::Proxy, |entry| *entry) + } + pub fn get_resource_url( &self, xtream_cluster: XtreamCluster, @@ -1006,31 +997,11 @@ impl XtreamMappingOptions { resource_url: &str, resource_field: &str, ) -> String { - if !self.web_ui_request && Self::is_trusted_web_ui_resource_path(resource_url) { - return concat_path(&self.base_url, resource_url); - } - - if self.web_ui_request { - if resource_url.is_empty() { - return resource_url.to_string(); - } - if Self::is_trusted_web_ui_resource_path(resource_url) { - return resource_url.to_string(); - } - let rewrite_url = concat_path(&self.base_url, &obfuscate_text(&self.encrypt_secret, resource_url)); - return rewrite_url; - } - - let external_resource = crate::model::external_resource_value(resource_url); - let rewrite_url = self.build_reverse_proxy_base_url(xtream_cluster, item_type, virtual_id); - - if let Some(url) = rewrite_url { - if external_resource.starts_with("http://") || external_resource.starts_with("https://") { - return format!("{url}/{resource_field}"); - } - } - external_resource.to_string() + self.rewrite_resource_url_inner(xtream_cluster, item_type, virtual_id, resource_url, |base| { + format!("{base}/{resource_field}") + }) } + pub fn get_bd_path_resource_url( &self, xtream_cluster: XtreamCluster, @@ -1040,6 +1011,23 @@ impl XtreamMappingOptions { resource_field: &str, index: usize, ) -> String { + self.rewrite_resource_url_inner(xtream_cluster, item_type, virtual_id, resource_url, |base| { + format!("{base}/{resource_field}{}_{index}", xtream_const::XC_PROP_BACKDROP_PATH) + }) + } + + fn rewrite_resource_url_inner( + &self, + xtream_cluster: XtreamCluster, + item_type: PlaylistItemType, + virtual_id: VirtualId, + resource_url: &str, + proxy_suffix: impl FnOnce(&str) -> String, + ) -> String { + let Some(resource_url) = super::persisted_resource_url(resource_url) else { + return String::new(); + }; + let resource_url = resource_url.as_ref(); if !self.web_ui_request && Self::is_trusted_web_ui_resource_path(resource_url) { return concat_path(&self.base_url, resource_url); } @@ -1051,19 +1039,29 @@ impl XtreamMappingOptions { if Self::is_trusted_web_ui_resource_path(resource_url) { return resource_url.to_string(); } - let rewrite_url = concat_path(&self.base_url, &obfuscate_text(&self.encrypt_secret, resource_url)); - return rewrite_url; + return concat_path(&self.base_url, &seal_web_ui_resource_url(&self.encrypt_secret, resource_url)); } - let external_resource = crate::model::external_resource_value(resource_url); - let rewrite_url = self.build_reverse_proxy_base_url(xtream_cluster, item_type, virtual_id); + let policy = self.resource_output_policy(resource_url); + if policy == ResourceOutputPolicy::Blocked { + return String::new(); + } + let rewrite_url = self.build_reverse_proxy_base_url( + xtream_cluster, + item_type, + virtual_id, + policy == ResourceOutputPolicy::Proxy, + ); if let Some(url) = rewrite_url { - if external_resource.starts_with("http://") || external_resource.starts_with("https://") { - return format!("{url}/{resource_field}{}_{index}", xtream_const::XC_PROP_BACKDROP_PATH); + if resource_url.starts_with("http://") + || resource_url.starts_with("https://") + || resource_url.starts_with("media-server://image/") + { + return proxy_suffix(&url); } } - external_resource.to_string() + resource_url.to_string() } } @@ -1148,14 +1146,72 @@ impl XtreamPlaylistItem { self.additional_properties.as_ref().is_some_and(super::stream_properties::StreamProperties::has_details) } + pub fn visit_resource_urls(&self, mut visit: impl FnMut(&str)) { + let mut visit_persisted = |value: &str| { + if let Some(url) = super::persisted_resource_url(value) { + visit(url.as_ref()); + } + }; + visit_persisted(&self.logo); + visit_persisted(&self.logo_small); + match self.additional_properties.as_ref() { + Some(StreamProperties::Live(live)) => visit_persisted(&live.stream_icon), + Some(StreamProperties::Video(video)) => { + visit_persisted(&video.stream_icon); + if let Some(details) = video.details.as_ref() { + if let Some(url) = details.cover_big.as_deref() { + visit_persisted(url); + } + if let Some(url) = details.movie_image.as_deref() { + visit_persisted(url); + } + if let Some(backdrops) = details.backdrop_path.as_ref() { + for url in backdrops { + visit_persisted(url); + } + } + } + } + Some(StreamProperties::Series(series)) => { + visit_persisted(&series.cover); + if let Some(backdrops) = series.backdrop_path.as_ref() { + for url in backdrops { + visit_persisted(url); + } + } + if let Some(details) = series.details.as_ref() { + if let Some(seasons) = details.seasons.as_ref() { + for season in seasons { + for url in [&season.cover, &season.cover_tmdb, &season.cover_big].into_iter().flatten() { + visit_persisted(url); + } + } + } + if let Some(episodes) = details.episodes.as_ref() { + for episode in episodes { + visit_persisted(&episode.movie_image); + } + } + } + } + Some(StreamProperties::Episode(episode)) => visit_persisted(&episode.movie_image), + None => {} + } + } + pub fn resolve_resource_url(&self, field: &str) -> Option> { let bytes = field.as_bytes(); - if bytes.eq_ignore_ascii_case(b"logo") && !self.logo.is_empty() { - return Some(Arc::clone(&self.logo)); + let url = if bytes.eq_ignore_ascii_case(b"logo") && !self.logo.is_empty() { + Some(Arc::clone(&self.logo)) } else if bytes.eq_ignore_ascii_case(b"logo_small") && !self.logo_small.is_empty() { - return Some(Arc::clone(&self.logo_small)); - } - self.additional_properties.as_ref().and_then(|a| a.resolve_resource_url(field)) + Some(Arc::clone(&self.logo_small)) + } else { + self.additional_properties.as_ref().and_then(|a| a.resolve_resource_url(field)) + }?; + super::persisted_resource_url(&url).map(|decoded| match decoded { + std::borrow::Cow::Borrowed(_) => Arc::clone(&url), + std::borrow::Cow::Owned(value) => Arc::from(value), + }) } } @@ -1782,9 +1838,75 @@ mod tests { web_ui_request: true, flags: XtreamMappingFlags::RewriteResourceUrl.into(), encrypt_secret: [3u8; 16], + resource_host_policies: Arc::default(), } } + #[test] + fn disabled_resource_rewrite_proxies_private_xtream_images() { + let mut options = sample_options(); + options.web_ui_request = false; + options.base_url = "https://proxy.example".to_string(); + options.flags = XtreamMappingFlagsSet::new(); + options.resource_host_policies.insert("192.168.1.20".to_string(), ResourceOutputPolicy::Proxy); + options.resource_host_policies.insert("8.8.8.8".to_string(), ResourceOutputPolicy::Direct); + options.resource_host_policies.insert("127.0.0.1".to_string(), ResourceOutputPolicy::Blocked); + + assert_eq!( + options.get_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + VirtualId::new(1), + "http://192.168.1.20/logo.png", + "logo", + ), + "https://proxy.example/resource/live/user/pass/1/logo" + ); + assert_eq!( + options.get_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + VirtualId::new(1), + "http://8.8.8.8/logo.png", + "logo", + ), + "http://8.8.8.8/logo.png" + ); + assert_eq!( + options.get_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + VirtualId::new(1), + "http://127.0.0.1/logo.png", + "logo", + ), + "" + ); + assert_eq!( + options.get_bd_path_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + VirtualId::new(1), + "http://192.168.1.20/backdrop.png", + "", + 0, + ), + "https://proxy.example/resource/live/user/pass/1/backdrop_path_0" + ); + + options.resource_proxy_item_types = PlaylistItemTypeSet::empty(); + assert_eq!( + options.get_resource_url( + XtreamCluster::Live, + PlaylistItemType::Live, + VirtualId::new(1), + "http://192.168.1.20/logo.png", + "logo", + ), + "https://proxy.example/resource/live/user/pass/1/logo" + ); + } + #[test] fn get_resource_url_keeps_internal_paths_for_web_ui_requests() { let options = sample_options(); @@ -1872,7 +1994,7 @@ mod tests { resource_url, "logo", ), - concat_path(&options.base_url, &obfuscate_text(&options.encrypt_secret, resource_url)), + concat_path(&options.base_url, &seal_web_ui_resource_url(&options.encrypt_secret, resource_url)), ); } @@ -1896,29 +2018,6 @@ mod tests { ); } - #[test] - fn get_resource_url_never_exposes_internal_locator_to_xtream_clients() { - let mut options = sample_options(); - options.web_ui_request = false; - options.reverse_item_types = PlaylistItemTypeSet::empty(); - options.resource_proxy_item_types = PlaylistItemTypeSet::empty(); - let locator = - crate::model::ResourceLocator::new(Arc::from("input"), Arc::from("https://provider.example/logo.png")) - .and_then(|locator| locator.encode()) - .expect("locator"); - - assert_eq!( - options.get_resource_url( - XtreamCluster::Live, - PlaylistItemType::Live, - VirtualId::new(2017), - &locator, - "logo", - ), - "https://provider.example/logo.png", - ); - } - #[test] fn get_resource_url_does_not_bypass_untrusted_root_relative_paths_for_web_ui_requests() { let options = sample_options(); @@ -1932,7 +2031,7 @@ mod tests { resource_url, "logo", ), - concat_path(&options.base_url, &obfuscate_text(&options.encrypt_secret, resource_url)), + concat_path(&options.base_url, &seal_web_ui_resource_url(&options.encrypt_secret, resource_url)), ); } @@ -1949,7 +2048,7 @@ mod tests { resource_url, "logo", ), - concat_path(&options.base_url, &obfuscate_text(&options.encrypt_secret, resource_url)), + concat_path(&options.base_url, &seal_web_ui_resource_url(&options.encrypt_secret, resource_url)), ); } @@ -2373,192 +2472,3 @@ mod tests { assert!(!item.to_m3u(None, false).contains("#EXTVLCOPT:http-user-agent=")); } } - -/// Append-only compatibility of the persisted playlist records. -/// -/// The B+Tree stores these structs with `rmp_serde::to_vec` in its compact positional form, so a -/// field added anywhere but the end would shift the meaning of every following byte. These tests -/// encode the field list as it was before origin tracking and decode it with the current struct. -#[cfg(test)] -mod persistence_compatibility { - use super::{ - M3uPlaylistItem, PlaylistItemHeader, PlaylistItemType, StreamProperties, VirtualId, XtreamCluster, - XtreamPlaylistItem, - }; - use crate::utils::Internable; - use serde::Serialize; - use std::sync::Arc; - - /// The `M3uPlaylistItem` field list without the appended origin fields. - #[derive(Serialize)] - struct LegacyM3u { - virtual_id: VirtualId, - provider_id: Arc, - name: Arc, - chno: u32, - logo: Arc, - logo_small: Arc, - group: Arc, - title: Arc, - parent_code: Arc, - audio_track: Arc, - time_shift: Arc, - rec: Arc, - url: Arc, - epg_channel_id: Option>, - input_name: Arc, - item_type: PlaylistItemType, - source_ordinal: u32, - additional_properties: Option, - input_stream_id: Arc, - upstream_user_agent: Option>, - } - - /// The `XtreamPlaylistItem` field list without the appended origin fields. - #[derive(Serialize)] - struct LegacyXtream { - virtual_id: VirtualId, - provider_id: u32, - name: Arc, - logo: Arc, - logo_small: Arc, - group: Arc, - title: Arc, - parent_code: Arc, - rec: Arc, - url: Arc, - epg_channel_id: Option>, - xtream_cluster: XtreamCluster, - additional_properties: Option, - item_type: PlaylistItemType, - category_id: u32, - input_name: Arc, - channel_no: u32, - source_ordinal: u32, - input_stream_id: Arc, - upstream_user_agent: Option>, - } - - #[test] - fn m3u_record_written_before_origin_tracking_still_decodes() { - let legacy = LegacyM3u { - virtual_id: VirtualId::new(7), - provider_id: "1".intern(), - name: "Channel".intern(), - chno: 1, - logo: "https://cdn.example.com/logo.png".intern(), - logo_small: "https://cdn.example.com/small.png".intern(), - group: "Group".intern(), - title: "".intern(), - parent_code: "".intern(), - audio_track: "".intern(), - time_shift: "".intern(), - rec: "".intern(), - url: "http://provider.example/stream/1".intern(), - epg_channel_id: Some("channel".intern()), - input_name: "provider".intern(), - item_type: PlaylistItemType::Live, - source_ordinal: 0, - additional_properties: None, - input_stream_id: "1".intern(), - upstream_user_agent: None, - }; - let bytes = rmp_serde::to_vec(&legacy).expect("encode legacy record"); - - let decoded: M3uPlaylistItem = rmp_serde::from_slice(&bytes).expect("legacy record still decodes"); - - assert_eq!(decoded.name, "Channel".intern()); - assert_eq!(decoded.input_name, "provider".intern()); - } - - #[test] - fn xtream_record_written_before_origin_tracking_still_decodes() { - let legacy = LegacyXtream { - virtual_id: VirtualId::new(9), - provider_id: 2, - name: "Movie".intern(), - logo: "https://cdn.example.com/movie.png".intern(), - logo_small: "".intern(), - group: "Movies".intern(), - title: "".intern(), - parent_code: "".intern(), - rec: "".intern(), - url: "http://provider.example/movie/2".intern(), - epg_channel_id: None, - xtream_cluster: XtreamCluster::Video, - additional_properties: None, - item_type: PlaylistItemType::Video, - category_id: 4, - input_name: "provider".intern(), - channel_no: 0, - source_ordinal: 0, - input_stream_id: "2".intern(), - upstream_user_agent: Some("Agent/1".intern()), - }; - let bytes = rmp_serde::to_vec(&legacy).expect("encode legacy record"); - - let decoded: XtreamPlaylistItem = rmp_serde::from_slice(&bytes).expect("legacy record still decodes"); - - assert_eq!(decoded.provider_id, 2); - assert_eq!(decoded.upstream_user_agent.as_deref(), Some("Agent/1")); - } - - #[test] - fn header_written_before_origin_tracking_still_decodes() { - #[derive(Serialize)] - struct LegacyHeader { - id: Arc, - name: Arc, - logo: Arc, - logo_small: Arc, - group: Arc, - title: Arc, - parent_code: Arc, - audio_track: Arc, - time_shift: Arc, - rec: Arc, - url: Arc, - epg_channel_id: Option>, - input_name: Arc, - additional_properties: Option, - virtual_id: VirtualId, - chno: u32, - category_id: u32, - source_ordinal: u32, - xtream_cluster: XtreamCluster, - item_type: PlaylistItemType, - input_stream_id: Arc, - upstream_user_agent: Option>, - } - - let legacy = LegacyHeader { - id: "1".intern(), - name: "Channel".intern(), - logo: "https://cdn.example.com/logo.png".intern(), - logo_small: "".intern(), - group: "Group".intern(), - title: "".intern(), - parent_code: "".intern(), - audio_track: "".intern(), - time_shift: "".intern(), - rec: "".intern(), - url: "http://provider.example/stream/1".intern(), - epg_channel_id: None, - input_name: "provider".intern(), - additional_properties: None, - virtual_id: VirtualId::new(1), - chno: 0, - category_id: 0, - source_ordinal: 0, - xtream_cluster: XtreamCluster::Live, - item_type: PlaylistItemType::Live, - input_stream_id: "1".intern(), - upstream_user_agent: None, - }; - let bytes = rmp_serde::to_vec(&legacy).expect("encode legacy header"); - - let decoded: PlaylistItemHeader = rmp_serde::from_slice(&bytes).expect("legacy header still decodes"); - - assert_eq!(decoded.logo, "https://cdn.example.com/logo.png".intern()); - } -} diff --git a/shared/src/model/playlist_document.rs b/shared/src/model/playlist_document.rs index 962098756..5eda29766 100644 --- a/shared/src/model/playlist_document.rs +++ b/shared/src/model/playlist_document.rs @@ -461,6 +461,7 @@ mod tests { base_url: "http://proxy.example".to_string(), web_ui_request: false, encrypt_secret: [0u8; 16], + resource_host_policies: Arc::default(), } } diff --git a/shared/src/model/playlist_info_document.rs b/shared/src/model/playlist_info_document.rs index f6f505db1..93096ce00 100644 --- a/shared/src/model/playlist_info_document.rs +++ b/shared/src/model/playlist_info_document.rs @@ -619,6 +619,7 @@ mod tests { base_url: "http://proxy.example".to_string(), web_ui_request: false, encrypt_secret: [0u8; 16], + resource_host_policies: Arc::default(), } } diff --git a/shared/src/model/resource.rs b/shared/src/model/resource.rs index 73030cd58..05cd29f2c 100644 --- a/shared/src/model/resource.rs +++ b/shared/src/model/resource.rs @@ -1,350 +1,67 @@ use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine}; -use serde::{Deserialize, Deserializer, Serialize}; -use std::{fmt, sync::Arc}; +use serde::Deserialize; +use std::{borrow::Cow, sync::Arc}; use url::Url; -pub const RESOURCE_LOCATOR_PREFIX: &str = "resource://v1/"; -const RESOURCE_SCHEME_PREFIX: &str = "resource://"; -const MAX_RESOURCE_LOCATOR_ENCODED_BYTES: usize = 16 * 1024; -const MAX_RESOURCE_LOCATOR_PAYLOAD_BYTES: usize = 12 * 1024; +const LEGACY_PREFIX: &str = "resource://v1/"; +const MAX_ENCODED_BYTES: usize = 16 * 1024; +const MAX_PAYLOAD_BYTES: usize = 12 * 1024; -/// Internal representation of a HTTP(S) resource and the canonical input that supplied it. -#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] -pub struct ResourceLocator { - #[serde(with = "crate::utils::arc_str_serde")] - pub input_name: Arc, - #[serde(with = "crate::utils::arc_str_serde")] - pub url: Arc, +#[derive(Deserialize)] +struct LegacyResourceLocator { + input_name: String, + url: String, } -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum ResourceLocatorError { - MissingPrefix, - TooLarge, - InvalidEncoding, - InvalidPayload, - EmptyInput, - UnsupportedScheme, +/// Decodes stored resource locators without accepting an invalid locator as a URL. +pub fn persisted_resource_url(value: &str) -> Option> { + if !value.starts_with("resource://") { + return Some(Cow::Borrowed(value)); + } + let encoded = value.strip_prefix(LEGACY_PREFIX)?; + if encoded.is_empty() || encoded.len() > MAX_ENCODED_BYTES { + return None; + } + let payload = URL_SAFE_NO_PAD.decode(encoded).ok()?; + if payload.len() > MAX_PAYLOAD_BYTES { + return None; + } + let locator = rmp_serde::from_slice::(&payload).ok()?; + if locator.input_name.is_empty() + || !Url::parse(&locator.url).is_ok_and(|url| matches!(url.scheme(), "http" | "https") && url.host().is_some()) + { + return None; + } + Some(Cow::Owned(locator.url)) } -impl fmt::Display for ResourceLocatorError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::MissingPrefix => f.write_str("resource locator has no supported prefix"), - Self::TooLarge => f.write_str("resource locator exceeds the size limit"), - Self::InvalidEncoding => f.write_str("resource locator encoding is invalid"), - Self::InvalidPayload => f.write_str("resource locator payload is invalid"), - Self::EmptyInput => f.write_str("resource locator input is empty"), - Self::UnsupportedScheme => f.write_str("resource locator URL scheme is unsupported"), - } +pub fn persisted_resource_arc(value: &Arc) -> Arc { + match persisted_resource_url(value) { + Some(Cow::Borrowed(_)) => Arc::clone(value), + Some(Cow::Owned(decoded)) => Arc::from(decoded), + None => Arc::from(""), } } -impl std::error::Error for ResourceLocatorError {} - -impl ResourceLocator { - pub fn new(input_name: Arc, url: Arc) -> Result { - if input_name.is_empty() { - return Err(ResourceLocatorError::EmptyInput); - } - if !is_http_resource_url(&url) { - return Err(ResourceLocatorError::UnsupportedScheme); - } - Ok(Self { input_name, url }) - } - - pub fn encode(&self) -> Result, ResourceLocatorError> { - let payload = rmp_serde::to_vec(self).map_err(|_| ResourceLocatorError::InvalidPayload)?; - if payload.len() > MAX_RESOURCE_LOCATOR_PAYLOAD_BYTES { - return Err(ResourceLocatorError::TooLarge); - } - let encoded = URL_SAFE_NO_PAD.encode(payload); - if encoded.len() > MAX_RESOURCE_LOCATOR_ENCODED_BYTES { - return Err(ResourceLocatorError::TooLarge); - } - Ok(format!("{RESOURCE_LOCATOR_PREFIX}{encoded}").into()) - } - - pub fn decode(value: &str) -> Result { - let encoded = value.strip_prefix(RESOURCE_LOCATOR_PREFIX).ok_or(ResourceLocatorError::MissingPrefix)?; - if encoded.is_empty() { - return Err(ResourceLocatorError::InvalidEncoding); - } - if encoded.len() > MAX_RESOURCE_LOCATOR_ENCODED_BYTES { - return Err(ResourceLocatorError::TooLarge); - } - let payload = URL_SAFE_NO_PAD.decode(encoded).map_err(|_| ResourceLocatorError::InvalidEncoding)?; - if payload.len() > MAX_RESOURCE_LOCATOR_PAYLOAD_BYTES { - return Err(ResourceLocatorError::TooLarge); - } - let locator = rmp_serde::from_slice::(&payload).map_err(|_| ResourceLocatorError::InvalidPayload)?; - Self::new(locator.input_name, locator.url) - } -} - -pub fn has_resource_scheme(value: &str) -> bool { value.starts_with(RESOURCE_SCHEME_PREFIX) } - -/// Provider-facing serde formats are human-readable (JSON/XML adapters). Reserved locators are -/// accepted only from the binary repository representation and are also checked by the explicit -/// ingestion traversal used by manual parsers. -pub fn deserialize_untrusted_resource_arc<'de, D>(deserializer: D) -> Result, D::Error> -where - D: Deserializer<'de>, -{ - let human_readable = deserializer.is_human_readable(); - let mut value = crate::utils::arc_str_none_default_on_null(deserializer)?; - if human_readable && has_resource_scheme(&value) { - value = Arc::from(""); - } - Ok(value) -} - -pub fn deserialize_untrusted_resource_option<'de, D>(deserializer: D) -> Result>, D::Error> -where - D: Deserializer<'de>, -{ - let human_readable = deserializer.is_human_readable(); - let mut value = crate::utils::deserialize_as_option_arc_str(deserializer)?; - if human_readable && value.as_deref().is_some_and(has_resource_scheme) { - value = None; - } - Ok(value) -} - -pub fn deserialize_untrusted_resource_array<'de, D>(deserializer: D) -> Result>>, D::Error> -where - D: Deserializer<'de>, -{ - let human_readable = deserializer.is_human_readable(); - let mut values = crate::utils::deserialize_as_string_array(deserializer)?; - if human_readable { - if let Some(values) = &mut values { - values.retain(|value| !has_resource_scheme(value)); - } - } - Ok(values) -} - -pub fn is_http_resource_url(value: &str) -> bool { - Url::parse(value).is_ok_and(|url| matches!(url.scheme(), "http" | "https") && url.host().is_some()) -} - -/// Converts one untrusted provider value into the trusted internal representation. -/// Reserved locators are cleared so provider data can never make an authorization claim. -pub fn ingest_resource_value(value: &mut Arc, input_name: &Arc) -> Result<(), ResourceLocatorError> { - if value.is_empty() { - return Ok(()); - } - if has_resource_scheme(value) { - *value = Arc::from(""); - return Err(ResourceLocatorError::UnsupportedScheme); - } - if is_http_resource_url(value) { - *value = ResourceLocator::new(Arc::clone(input_name), Arc::clone(value))?.encode()?; - } - Ok(()) -} - -pub fn ingest_optional_resource_value( - value: &mut Option>, - input_name: &Arc, -) -> Result<(), ResourceLocatorError> { - if let Some(resource) = value { - if let Err(error) = ingest_resource_value(resource, input_name) { - *value = None; - return Err(error); - } - } - Ok(()) -} - -/// Normalizes values after trusted internal transformations. Existing valid locators are -/// preserved; newly produced raw HTTP(S) values become owned by `input_name`. -pub fn normalize_internal_resource_value( - value: &mut Arc, - input_name: &Arc, -) -> Result<(), ResourceLocatorError> { - if value.starts_with(RESOURCE_LOCATOR_PREFIX) { - ResourceLocator::decode(value)?; - return Ok(()); - } - ingest_resource_value(value, input_name) -} - -pub fn resolve_resource_value(value: &str) -> Result, ResourceLocatorError> { - if value.starts_with(RESOURCE_LOCATOR_PREFIX) { - return ResourceLocator::decode(value).map(Some); - } - if has_resource_scheme(value) { - return Err(ResourceLocatorError::MissingPrefix); - } - Ok(None) -} - -/// Converts the internal representation to the value safe for external output. Malformed -/// reserved values fail closed instead of leaking the internal scheme. -pub fn external_resource_value(value: &str) -> Arc { - match resolve_resource_value(value) { - Ok(Some(locator)) => locator.url, - Ok(None) => Arc::from(value), - Err(_) => Arc::from(""), - } -} - -/// Authenticated payload of a rewritten resource link. -/// -/// Routes whose path holds only an encoded URL have no stored record to read an origin from, so -/// the origin travels inside the link. The payload is authenticated, which also makes it -/// distinguishable from the two legacy encodings that carry a bare URL. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -pub struct ResourceToken { - pub resource: String, -} - #[cfg(test)] mod tests { - use super::{ - external_resource_value, has_resource_scheme, ingest_resource_value, ResourceLocator, ResourceLocatorError, - MAX_RESOURCE_LOCATOR_ENCODED_BYTES, RESOURCE_LOCATOR_PREFIX, - }; - use crate::model::{LiveStreamProperties, StalkerPlaylistItem, VideoStreamProperties}; - use std::sync::Arc; + use super::{persisted_resource_url, LEGACY_PREFIX}; + use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine}; + use serde::Serialize; - #[test] - fn locator_round_trips_http_urls_losslessly() { - for url in [ - "http://example.com/image.png", - "https://user:password@example.com:8443/a@b?q=a:b/c#fragment", - "https://[2001:db8::1]:8443/image.png", - "https://example.com/%E2%98%83?q=%C3%A4", - ] { - let locator = ResourceLocator::new(Arc::from("input:@/รค"), Arc::from(url)).expect("valid locator"); - let encoded = locator.encode().expect("encode locator"); - assert!(encoded.starts_with(RESOURCE_LOCATOR_PREFIX), "encoded starts with prefix"); - assert_eq!(ResourceLocator::decode(&encoded).expect("decode locator"), locator); - } + #[derive(Serialize)] + struct OldLocator<'a> { + input_name: &'a str, + url: &'a str, } #[test] - fn locator_rejects_malformed_and_unsupported_values() { - assert_eq!(ResourceLocator::decode("resource://v2/AAAA"), Err(ResourceLocatorError::MissingPrefix)); - assert_eq!(ResourceLocator::decode("resource://v1/***"), Err(ResourceLocatorError::InvalidEncoding)); - assert_eq!( - ResourceLocator::new(Arc::from(""), Arc::from("https://example.com/a")), - Err(ResourceLocatorError::EmptyInput) - ); - assert_eq!( - ResourceLocator::new(Arc::from("input"), Arc::from("")), - Err(ResourceLocatorError::UnsupportedScheme) - ); - for url in ["resource://v1/AAAA", "file:///tmp/a", "provider://a", "batch://a", "media-server://image/a"] { - assert_eq!( - ResourceLocator::new(Arc::from("input"), Arc::from(url)), - Err(ResourceLocatorError::UnsupportedScheme) - ); - } - let oversized = format!("resource://v1/{}", "A".repeat(MAX_RESOURCE_LOCATOR_ENCODED_BYTES + 1)); - assert_eq!(ResourceLocator::decode(&oversized), Err(ResourceLocatorError::TooLarge)); - } - - #[test] - fn provider_values_cannot_supply_internal_locators() { - let locator = ResourceLocator::new(Arc::from("other-input"), Arc::from("https://private.example/a")) - .expect("locator") - .encode() - .expect("encode"); - let mut forged = locator; - assert!(ingest_resource_value(&mut forged, &Arc::from("provider-input")).is_err()); - assert!(forged.is_empty()); - assert!(has_resource_scheme("resource://v99/value")); - } - - #[test] - fn ingest_wraps_http_and_preserves_non_http_values() { - let input = Arc::from("test-input"); - - // HTTP URL - let mut http_val = Arc::from("https://cdn.example.com/logo.png"); - assert!(ingest_resource_value(&mut http_val, &input).is_ok()); - assert!(http_val.starts_with(RESOURCE_LOCATOR_PREFIX)); - let decoded = ResourceLocator::decode(&http_val).expect("decode http"); - assert_eq!(decoded.url.as_ref(), "https://cdn.example.com/logo.png"); - assert_eq!(decoded.input_name.as_ref(), "test-input"); - - // Dedicated non-HTTP schemes stay on their existing serving paths. - let mut ms_val = Arc::from("media-server://image/plex/logo"); - assert!(ingest_resource_value(&mut ms_val, &input).is_ok()); - assert_eq!(ms_val.as_ref(), "media-server://image/plex/logo"); - - // Local/internal paths are not proxied HTTP resources. - let mut path_val = Arc::from("/path/to/logo.png"); - assert!(ingest_resource_value(&mut path_val, &input).is_ok()); - assert_eq!(path_val.as_ref(), "/path/to/logo.png"); - - // Empty stays empty - let mut empty_val = Arc::from(""); - assert!(ingest_resource_value(&mut empty_val, &input).is_ok()); - assert!(empty_val.is_empty()); - } - - #[test] - fn external_resource_value_unwraps_locator() { - let input = Arc::from("test-input"); - let url = "https://cdn.example.com/logo.png"; - let locator = ResourceLocator::new(input, Arc::from(url)).and_then(|l| l.encode()).expect("encode"); - assert_eq!(external_resource_value(&locator).as_ref(), url); - } - - #[test] - fn human_readable_provider_models_reject_reserved_locators() { - let locator = ResourceLocator::new(Arc::from("other-input"), Arc::from("https://private.example/a")) - .and_then(|locator| locator.encode()) - .expect("locator"); - - let live: LiveStreamProperties = - serde_json::from_value(serde_json::json!({ "stream_icon": locator })).expect("live JSON"); - assert!(live.stream_icon.is_empty()); - - let video: VideoStreamProperties = serde_json::from_value(serde_json::json!({ - "stream_icon": locator, - "details": { - "cover_big": locator, - "movie_image": locator, - "backdrop_path": [locator] - } - })) - .expect("video JSON"); - assert!(video.stream_icon.is_empty()); - let details = video.details.expect("details"); - assert!(details.cover_big.is_none()); - assert!(details.movie_image.is_none()); - assert!(details.backdrop_path.is_some_and(|values| values.is_empty())); - - let stalker: StalkerPlaylistItem = serde_json::from_value(serde_json::json!({ - "stream_id": 1, - "name": "name", - "category_id": 1, - "category_name": "group", - "number": 1, - "logo_url": locator, - "stream_url": "https://example.com/live", - "stream_kind": "live", - "cmd": "" - })) - .expect("Stalker JSON"); - assert!(stalker.logo_url.is_none()); - } - - #[test] - fn binary_repository_values_preserve_internal_locators() { - let locator = ResourceLocator::new(Arc::from("input"), Arc::from("https://example.com/logo.png")) - .and_then(|locator| locator.encode()) - .expect("locator"); - let properties = LiveStreamProperties { stream_icon: Arc::clone(&locator), ..LiveStreamProperties::default() }; - - let encoded = rmp_serde::to_vec(&properties).expect("serialize repository value"); - let decoded: LiveStreamProperties = rmp_serde::from_slice(&encoded).expect("deserialize repository value"); - - assert_eq!(decoded.stream_icon, locator); + fn decodes_persisted_resource_without_exposing_malformed_locators() { + let payload = + rmp_serde::to_vec(&OldLocator { input_name: "m3u", url: "http://10.0.0.5/logo.png" }).expect("old locator"); + let locator = format!("{LEGACY_PREFIX}{}", URL_SAFE_NO_PAD.encode(payload)); + assert_eq!(persisted_resource_url(&locator).as_deref(), Some("http://10.0.0.5/logo.png")); + assert_eq!(persisted_resource_url("resource://v1/invalid"), None); + assert_eq!(persisted_resource_url("resource://v2/unknown"), None); } } diff --git a/shared/src/model/stalker_item.rs b/shared/src/model/stalker_item.rs index 68d18fbfc..cb278f55b 100644 --- a/shared/src/model/stalker_item.rs +++ b/shared/src/model/stalker_item.rs @@ -24,7 +24,7 @@ pub struct StalkerPlaylistItem { #[serde(with = "arc_str_serde")] pub category_name: Arc, pub number: u32, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default)] pub logo_url: Option>, #[serde(default)] pub epg_channel_id: Option>, @@ -53,7 +53,7 @@ pub struct StalkerPlaylistItem { pub rating: f32, #[serde(default)] pub tmdb_id: Option, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default)] pub backdrop_url: Option>, /// Unix timestamp in seconds when the item was first persisted. #[serde(default)] @@ -138,19 +138,6 @@ impl Default for StalkerPlaylistItem { } impl StalkerPlaylistItem { - pub fn ingest_resource_values(&mut self, input_name: &Arc) -> usize { - let mut rejected = 0; - for value in [&mut self.logo_url, &mut self.backdrop_url] { - if let Some(resource) = value { - if crate::model::ingest_resource_value(resource, input_name).is_err() { - *value = None; - rejected += 1; - } - } - } - rejected - } - /// Whether this item is a series root (no individual playback URL). pub fn is_series_root(&self) -> bool { self.is_series } @@ -175,23 +162,12 @@ pub struct StalkerSeasonItem { pub season_number: i32, #[serde(with = "arc_str_serde")] pub name: Arc, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default)] pub cover_url: Option>, #[serde(with = "arc_str_vec_serde", default)] pub episodes: Vec>, } -impl StalkerSeasonItem { - pub fn ingest_resource_values(&mut self, input_name: &Arc) -> usize { - let Some(cover) = &mut self.cover_url else { return 0 }; - if crate::model::ingest_resource_value(cover, input_name).is_err() { - self.cover_url = None; - return 1; - } - 0 - } -} - /// Lightweight episode record that keeps the actual `StalkerPlaylistItem` /// in the live/episode store and only embeds the season-level summary here. /// diff --git a/shared/src/model/stream_properties.rs b/shared/src/model/stream_properties.rs index 905f7a388..3f66305e5 100644 --- a/shared/src/model/stream_properties.rs +++ b/shared/src/model/stream_properties.rs @@ -2,9 +2,9 @@ use crate::{ model::{info_doc_utils::InfoDocUtils, PlaylistEntry, XtreamSeriesInfo, XtreamSeriesInfoDoc, XtreamVideoInfo}, utils::{ arc_str_none_default_on_null, arc_str_null_is_none_serde, arc_str_option_null_if_empty_serde, - arc_str_option_serde, deserialize_as_option_arc_str, deserialize_json_as_opt_string, - deserialize_number_from_string, deserialize_number_from_string_or_zero, serialize_json_as_opt_string, - Internable, CONSTANTS, + arc_str_option_serde, deserialize_as_option_arc_str, deserialize_as_string_array, + deserialize_json_as_opt_string, deserialize_number_from_string, deserialize_number_from_string_or_zero, + serialize_json_as_opt_string, Internable, CONSTANTS, }, }; use log::warn; @@ -138,7 +138,7 @@ pub struct LiveStreamProperties { pub category_id: u32, #[serde(default, deserialize_with = "deserialize_number_from_string_or_zero")] pub stream_id: u32, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_arc")] + #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub stream_icon: Arc, #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub direct_source: Arc, @@ -219,9 +219,9 @@ pub struct VideoStreamDetailProperties { pub kinopoisk_url: Option>, #[serde(default, with = "arc_str_option_serde")] pub o_name: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default, with = "arc_str_option_serde")] pub cover_big: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default, with = "arc_str_option_serde")] pub movie_image: Option>, #[serde(default, with = "arc_str_option_serde")] pub release_date: Option>, @@ -249,7 +249,7 @@ pub struct VideoStreamDetailProperties { pub country: Option>, #[serde(default, with = "arc_str_option_serde")] pub genre: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_array")] + #[serde(default, deserialize_with = "deserialize_as_string_array")] pub backdrop_path: Option>>, #[serde(default, with = "arc_str_option_serde")] pub duration_secs: Option>, @@ -283,7 +283,7 @@ pub struct VideoStreamProperties { pub category_id: u32, #[serde(default, deserialize_with = "deserialize_number_from_string_or_zero")] pub stream_id: u32, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_arc")] + #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub stream_icon: Arc, #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub direct_source: Arc, @@ -321,11 +321,11 @@ pub struct SeriesStreamDetailSeasonProperties { pub overview: Option>, #[serde(default, deserialize_with = "deserialize_as_option_arc_str")] pub air_date: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default, deserialize_with = "deserialize_as_option_arc_str")] pub cover: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default, deserialize_with = "deserialize_as_option_arc_str")] pub cover_tmdb: Option>, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_option")] + #[serde(default, deserialize_with = "deserialize_as_option_arc_str")] pub cover_big: Option>, #[serde(default, deserialize_with = "deserialize_as_option_arc_str")] pub duration: Option>, @@ -363,7 +363,7 @@ pub struct SeriesStreamDetailEpisodeProperties { pub duration_secs: u32, #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub duration: Arc, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_arc")] + #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub movie_image: Arc, #[serde(default, deserialize_with = "deserialize_number_from_string_or_zero")] pub bitrate: u32, @@ -413,11 +413,11 @@ pub struct SeriesStreamProperties { pub category_id: u32, #[serde(default, deserialize_with = "deserialize_number_from_string_or_zero")] pub series_id: u32, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_array")] + #[serde(default, deserialize_with = "deserialize_as_string_array")] pub backdrop_path: Option>>, #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub cast: Arc, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_arc")] + #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub cover: Arc, #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub director: Arc, @@ -461,7 +461,7 @@ pub struct EpisodeStreamProperties { pub plot: Option>, #[serde(default, deserialize_with = "deserialize_number_from_string")] pub tmdb: Option, - #[serde(default, deserialize_with = "crate::model::deserialize_untrusted_resource_arc")] + #[serde(default, deserialize_with = "arc_str_none_default_on_null")] pub movie_image: Arc, #[serde(default, deserialize_with = "arc_str_null_is_none_serde::deserialize")] pub container_extension: Arc, @@ -488,117 +488,6 @@ pub enum StreamProperties { } impl StreamProperties { - pub fn visit_resource_values(&self, visitor: &mut impl FnMut(&Arc)) { - fn visit_optional(value: &Option>, visitor: &mut impl FnMut(&Arc)) { - if let Some(value) = value { - visitor(value); - } - } - fn visit_many(values: &Option>>, visitor: &mut impl FnMut(&Arc)) { - if let Some(values) = values { - values.iter().for_each(visitor); - } - } - match self { - Self::Live(live) => visitor(&live.stream_icon), - Self::Video(video) => { - visitor(&video.stream_icon); - if let Some(details) = &video.details { - visit_optional(&details.cover_big, visitor); - visit_optional(&details.movie_image, visitor); - visit_many(&details.backdrop_path, visitor); - } - } - Self::Series(series) => { - visitor(&series.cover); - visit_many(&series.backdrop_path, visitor); - if let Some(details) = &series.details { - if let Some(seasons) = &details.seasons { - for season in seasons { - visit_optional(&season.cover, visitor); - visit_optional(&season.cover_tmdb, visitor); - visit_optional(&season.cover_big, visitor); - } - } - if let Some(episodes) = &details.episodes { - for episode in episodes { - visitor(&episode.movie_image); - } - } - } - } - Self::Episode(episode) => visitor(&episode.movie_image), - } - } - - /// Visits every resource value that can be selected by [`Self::resolve_resource_url`]. - /// Ingestion validation and endpoint lookup intentionally share this implementation surface. - pub fn visit_resource_values_mut(&mut self, visitor: &mut impl FnMut(&mut Arc)) { - fn visit_optional(value: &mut Option>, visitor: &mut impl FnMut(&mut Arc)) { - if let Some(value) = value { - visitor(value); - } - } - fn visit_many(values: &mut Option>>, visitor: &mut impl FnMut(&mut Arc)) { - if let Some(values) = values { - values.iter_mut().for_each(visitor); - } - } - - match self { - Self::Live(live) => visitor(&mut live.stream_icon), - Self::Video(video) => { - visitor(&mut video.stream_icon); - if let Some(details) = &mut video.details { - visit_optional(&mut details.cover_big, visitor); - visit_optional(&mut details.movie_image, visitor); - visit_many(&mut details.backdrop_path, visitor); - } - } - Self::Series(series) => { - visitor(&mut series.cover); - visit_many(&mut series.backdrop_path, visitor); - if let Some(details) = &mut series.details { - if let Some(seasons) = &mut details.seasons { - for season in seasons { - visit_optional(&mut season.cover, visitor); - visit_optional(&mut season.cover_tmdb, visitor); - visit_optional(&mut season.cover_big, visitor); - } - } - if let Some(episodes) = &mut details.episodes { - for episode in episodes { - visitor(&mut episode.movie_image); - } - } - } - } - Self::Episode(episode) => visitor(&mut episode.movie_image), - } - } - - /// Applies the provider-ingress boundary to all resource-bearing nested fields. - pub fn ingest_resource_values(&mut self, input_name: &Arc) -> usize { - let mut rejected = 0; - self.visit_resource_values_mut(&mut |value| { - if super::ingest_resource_value(value, input_name).is_err() { - rejected += 1; - } - }); - rejected - } - - pub fn normalize_internal_resource_values(&mut self, input_name: &Arc) -> usize { - let mut rejected = 0; - self.visit_resource_values_mut(&mut |value| { - if super::normalize_internal_resource_value(value, input_name).is_err() { - *value = Arc::from(""); - rejected += 1; - } - }); - rejected - } - fn episode_value(&self, selector: F) -> Option where F: FnOnce(&EpisodeStreamProperties) -> T, @@ -876,6 +765,9 @@ impl StreamProperties { if field == "cover_big" { return season.cover_big.as_ref().map(Arc::clone); } + if field == "overview" { + return season.overview.as_ref().map(Arc::clone); + } } } } @@ -909,6 +801,9 @@ impl StreamProperties { /// Field names accepted by resource-proxy endpoints. This is deliberately defined beside the /// resource traversal and selector so adding a new endpoint-visible resource has one review site. +/// +/// Stream URLs are not resources: serving them here would sidestep the stream routes and their +/// admission accounting. pub fn is_resource_field_name(field: &str) -> bool { if matches!(field, "logo" | "logo_small" | "cover" | "movie_image" | "nfo_cover_big" | "nfo_movie_image") || field.starts_with("backdrop_path") @@ -1514,4 +1409,25 @@ mod tests { assert_eq!(current.last_probed_timestamp, Some(200)); assert_eq!(current.last_success_timestamp, Some(150)); } + + #[test] + fn resource_field_names_exclude_stream_urls() { + for resource in [ + "logo", + "logo_small", + "cover", + "movie_image", + "nfo_cover_big", + "backdrop_path", + "backdrop_path1", + "nfo_backdrop_path", + "nfo_s_2_cover", + "nfo_ep_2_5_movie_image", + ] { + assert!(is_resource_field_name(resource), "{resource}"); + } + for not_a_resource in ["url", "name", "chno", "caption", "epg_channel_id", "stream_url", "nfo_s_2_title"] { + assert!(!is_resource_field_name(not_a_resource), "{not_a_resource}"); + } + } } diff --git a/shared/src/utils/crypto.rs b/shared/src/utils/crypto.rs index 90bb971f0..50b5d943f 100644 --- a/shared/src/utils/crypto.rs +++ b/shared/src/utils/crypto.rs @@ -1,4 +1,119 @@ +use crate::error::TuliproxError; +use aes::Aes128; use base64::{engine::general_purpose, Engine as _}; +use ctr::cipher::{KeyIvInit, StreamCipher}; + +type Aes128Ctr = ctr::Ctr128BE; +const AUTH_TOKEN_VERSION: u8 = 1; +const AUTH_TOKEN_IV_LEN: usize = 16; +const AUTH_TOKEN_MAC_LEN: usize = 32; +const WEB_UI_RESOURCE_DOMAIN: &[u8] = b"tuliprox.web-ui.resource.v1"; +const HLS_RESOURCE_DOMAIN: &[u8] = b"tuliprox.hls.resource.v1"; + +fn token_mac_key(secret: &[u8; 16], domain: &[u8]) -> [u8; AUTH_TOKEN_MAC_LEN] { + let mut hasher = blake3::Hasher::new(); + hasher.update(b"tuliprox.authenticated-token.mac.v1"); + hasher.update(&(domain.len() as u64).to_be_bytes()); + hasher.update(domain); + hasher.update(secret); + *hasher.finalize().as_bytes() +} + +fn token_mac(secret: &[u8; 16], domain: &[u8], data: &[u8]) -> blake3::Hash { + blake3::keyed_hash(&token_mac_key(secret, domain), data) +} + +fn token_iv(secret: &[u8; 16], domain: &[u8], plaintext: &[u8]) -> [u8; AUTH_TOKEN_IV_LEN] { + let mut hasher = blake3::Hasher::new(); + hasher.update(b"tuliprox.authenticated-token.siv.v1"); + hasher.update(&(domain.len() as u64).to_be_bytes()); + hasher.update(domain); + hasher.update(secret); + hasher.update(plaintext); + let mut iv = [0u8; AUTH_TOKEN_IV_LEN]; + iv.copy_from_slice(&hasher.finalize().as_bytes()[..AUTH_TOKEN_IV_LEN]); + iv +} + +fn apply_token_cipher(secret: &[u8; 16], iv: &[u8; AUTH_TOKEN_IV_LEN], data: &mut [u8]) { + let mut cipher = Aes128Ctr::new(secret.into(), iv.into()); + cipher.apply_keystream(data); +} + +fn encode_authenticated_bytes(secret: &[u8; 16], domain: &[u8], plaintext: &[u8]) -> String { + let iv = token_iv(secret, domain, plaintext); + let mut out = Vec::with_capacity(1 + AUTH_TOKEN_IV_LEN + plaintext.len() + AUTH_TOKEN_MAC_LEN); + out.push(AUTH_TOKEN_VERSION); + out.extend_from_slice(&iv); + out.extend_from_slice(plaintext); + apply_token_cipher(secret, &iv, &mut out[1 + AUTH_TOKEN_IV_LEN..]); + let mac = token_mac(secret, domain, &out); + out.extend_from_slice(mac.as_bytes()); + general_purpose::URL_SAFE_NO_PAD.encode(out) +} + +pub fn obscure_authenticated_bytes( + secret: &[u8; 16], + domain: &[u8], + plaintext: &[u8], +) -> Result { + Ok(encode_authenticated_bytes(secret, domain, plaintext)) +} + +pub fn deobscure_authenticated_bytes( + secret: &[u8; 16], + domain: &[u8], + encoded: &str, +) -> Result, TuliproxError> { + let data = general_purpose::URL_SAFE_NO_PAD + .decode(encoded) + .map_err(|_| TuliproxError::Crypto("Can't decode authenticated token".to_string()))?; + if data.len() < 1 + AUTH_TOKEN_IV_LEN + AUTH_TOKEN_MAC_LEN { + return Err(TuliproxError::Crypto("Authenticated token is too short".to_string())); + } + if data[0] != AUTH_TOKEN_VERSION { + return Err(TuliproxError::Crypto("Unsupported authenticated token version".to_string())); + } + let mac_offset = data.len() - AUTH_TOKEN_MAC_LEN; + let (authenticated_data, token_mac_bytes) = data.split_at(mac_offset); + let expected_mac = token_mac(secret, domain, authenticated_data); + let actual_mac = <[u8; AUTH_TOKEN_MAC_LEN]>::try_from(token_mac_bytes) + .map(blake3::Hash::from_bytes) + .map_err(|_| TuliproxError::Crypto("Invalid authenticated token MAC".to_string()))?; + if expected_mac != actual_mac { + return Err(TuliproxError::Crypto("Authenticated token MAC mismatch".to_string())); + } + let iv: [u8; AUTH_TOKEN_IV_LEN] = authenticated_data[1..=AUTH_TOKEN_IV_LEN] + .try_into() + .map_err(|_| TuliproxError::Crypto("Invalid authenticated token IV".to_string()))?; + let mut plaintext = authenticated_data[1 + AUTH_TOKEN_IV_LEN..].to_vec(); + apply_token_cipher(secret, &iv, &mut plaintext); + Ok(plaintext) +} + +pub fn seal_web_ui_resource_url(secret: &[u8; 16], url: &str) -> String { + encode_authenticated_bytes(secret, WEB_UI_RESOURCE_DOMAIN, url.as_bytes()) +} + +pub fn open_web_ui_resource_url(secret: &[u8; 16], encoded: &str) -> Result { + if encoded.len() > 16_384 { + return Err(TuliproxError::Crypto("Resource token is too long".to_string())); + } + let bytes = deobscure_authenticated_bytes(secret, WEB_UI_RESOURCE_DOMAIN, encoded)?; + String::from_utf8(bytes).map_err(|_| TuliproxError::Crypto("Invalid resource URL".to_string())) +} + +pub fn seal_hls_resource_url(secret: &[u8; 16], url: &str) -> String { + encode_authenticated_bytes(secret, HLS_RESOURCE_DOMAIN, url.as_bytes()) +} + +pub fn open_hls_resource_url(secret: &[u8; 16], encoded: &str) -> Result { + if encoded.len() > 16_384 { + return Err(TuliproxError::Crypto("HLS resource token is too long".to_string())); + } + let bytes = deobscure_authenticated_bytes(secret, HLS_RESOURCE_DOMAIN, encoded)?; + String::from_utf8(bytes).map_err(|_| TuliproxError::Crypto("Invalid HLS resource URL".to_string())) +} pub fn encode_base64_string(input: &[u8]) -> String { general_purpose::URL_SAFE_NO_PAD.encode(input) } @@ -26,7 +141,10 @@ pub fn deobfuscate_text(secret: &[u8], text: &str) -> Result { #[cfg(test)] mod tests { - use crate::utils::{deobfuscate_text, obfuscate_text}; + use crate::utils::{ + deobfuscate_text, obfuscate_text, open_hls_resource_url, open_web_ui_resource_url, seal_hls_resource_url, + seal_web_ui_resource_url, + }; #[test] fn test_obfuscate() { @@ -40,4 +158,31 @@ mod tests { assert_eq!(decrypted, plain); } + + #[test] + fn resource_tokens_reject_xor_and_cross_domain_tokens() { + let secret = [7u8; 16]; + let url = "http://192.168.1.20/logo.png"; + let web_token = seal_web_ui_resource_url(&secret, url); + let hls_token = seal_hls_resource_url(&secret, url); + + assert_eq!(open_web_ui_resource_url(&secret, &web_token).ok().as_deref(), Some(url)); + assert_eq!(open_hls_resource_url(&secret, &hls_token).ok().as_deref(), Some(url)); + assert!(!web_token.contains("192.168.1.20")); + assert!(open_web_ui_resource_url(&secret, &obfuscate_text(&secret, url)).is_err()); + assert!(open_web_ui_resource_url(&secret, &hls_token).is_err()); + assert!(open_hls_resource_url(&secret, &web_token).is_err()); + } + + #[test] + fn resource_tokens_reject_modified_ciphertext() { + use base64::{engine::general_purpose, Engine as _}; + + let secret = [7u8; 16]; + let token = seal_web_ui_resource_url(&secret, "http://192.168.1.20/logo.png"); + let mut bytes = general_purpose::URL_SAFE_NO_PAD.decode(token).expect("token bytes"); + bytes[20] ^= 1; + let tampered = general_purpose::URL_SAFE_NO_PAD.encode(bytes); + assert!(open_web_ui_resource_url(&secret, &tampered).is_err()); + } }