diff --git a/CHANGELOG.md b/CHANGELOG.md index 04f9b8ba2..1500fbc57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -192,6 +192,21 @@ ## 🌟 New Features +- **New mapper built-in functions: `title_case`, `slug`, `normalize_spaces`, `coalesce`, and `deunicode`.** Five new + helper functions for playlist transformations in the mapper DSL: + - `title_case`: Capitalizes the first character of every word and lowercases the rest: `@Title = title_case(@Title)` + turns `us: tnt hd` into `Us: Tnt Hd`. Only whitespace separates words, so `spider-man` becomes `Spider-man`. An + optional array keeps listed acronyms in capitals: `title_case("us: tnt hd", ["TNT", "HD"])` becomes + `Us: TNT HD`. The existing `capitalize` still touches only the first character of the whole string. + - `slug`: Converts text into a lowercase, ASCII-transliterated slug with `-` separators (`slug("US: HBO Max HD!")` + becomes `us-hbo-max-hd`). + - `normalize_spaces`: Collapses multiple consecutive whitespace characters into a single space and trims outer + edges (`normalize_spaces(" US | ESPN HD ")` becomes `US | ESPN HD`). + - `coalesce`: Returns the first non-empty, non-blank value across arguments (`@Logo = coalesce(@Logo, @Logo_Small, "default.png")`). + Arguments are evaluated left to right and evaluation stops at the first usable value. + - `deunicode`: Transliterates Unicode characters to their closest ASCII representation (`deunicode("Univisión: Café y Noticias")` + becomes `Univision: Cafe y Noticias`). + - **Configurable credential generation for new users.** The username, password and token pre-filled when creating an API user in the Web UI can now be tuned via `credential_generation` in `api-proxy.yml`: length (1-64), allowed `chars` with ranges, `exclude`, or ordered `segments` (for example 3 letters followed by 3 digits). Defaults match the @@ -1217,6 +1232,10 @@ ## 🐛 Fixes +- **Mapper arity errors name the function the user wrote.** A mapper script with the wrong number of arguments reported + the internal enum variant (`Function accepts only one argument TitleCase`) instead of the function name + (`Function accepts only one argument title_case`). + - **VOD from providers that label ranged media `text/html` plays again in reverse proxy mode.** The "Provider returned HTML content" check rejected every non-adaptive success response with a `text/html` content type, so a valid `206` movie (for example Matroska) served with the wrong type fell back to the diff --git a/docs/src/configuration/mapping-dsl.md b/docs/src/configuration/mapping-dsl.md index 862448654..8525aa553 100644 --- a/docs/src/configuration/mapping-dsl.md +++ b/docs/src/configuration/mapping-dsl.md @@ -184,7 +184,8 @@ Tuliprox updates both the `Title` AND the `Name` to the same value).* | `concat(a, b, ...)` | Concatenates multiple strings. | `concat("US \| ", @Title)` | | `uppercase(a)` | Converts text to UPPERCASE. | `uppercase(@Group)` | | `lowercase(a)` | Converts text to lowercase. | `lowercase(@Genre)` | -| `capitalize(a)` | Title Case (Capitalizes the first letter of words). | `capitalize(@Title)` | +| `capitalize(a)` | Capitalizes only the first character of the whole string, lowercases the rest. | `capitalize(@Title)` | +| `title_case(a, acronyms?)` | Capitalizes the first character of every word and lowercases the rest. The optional second argument keeps the listed acronyms in capitals. | `title_case(@Title, ["HD"])` | | `split(a, delim)` | Splits a string and returns a Named list (iterable). | `split(@Genre, ",")` | | `trim(a)` | Removes whitespace from the edges. | `trim(@Title)` | | `replace(a, b, c)` | Simple text Search (b) & Replace (c). | `replace(@Title, "FHD", "")` | @@ -195,6 +196,66 @@ Tuliprox updates both the `Title` AND the `Name` to the same value).* | `first(list)` | Returns the first element of a Named list/Regex match. | `first(@Caption ~ "(\d+)")` | | `print(a, b, ...)` | Logs the values to the console (Requires `trace` log level). | `print("Matched:", @Title)` | | `add_favourite(grp)` | **Background:** Clone function! Takes the currently processed stream, changes its group to `grp`, generates a clean alias UUID, and adds the stream additionally (as a "Favorite") to the playlist. | `add_favourite("Top 10")` | +| `slug(a)` | Converts text to a lowercase URL- and filename-safe slug (ASCII only, `-` separated). | `slug(@Title)` | +| `normalize_spaces(a)` | Collapses consecutive whitespace into a single space and trims edges. | `normalize_spaces(@Title)` | +| `coalesce(a, b, ...)` | Returns the first non-empty, non-blank argument. | `coalesce(@Logo, "fallback")` | +| `deunicode(a)` | Transliterates Unicode characters to ASCII (e.g. `ä` -> `a`, `é` -> `e`). | `deunicode(@Title)` | + +Function arguments may contain an array literal: a comma-separated list of quoted strings in square brackets, for +example `["HD", "FHD", "UK"]`. Arrays are currently accepted as the acronym list of `title_case`. + +Arguments are positional. `pad`, `replace` and `format` map every argument to the parameter at the position where it +was written, even when an earlier argument evaluates to nothing (for example an unmatched regex capture). The later +arguments keep their own slots instead of moving up, so `format("{} {}", part, "B")` still puts `B` in the second +placeholder and leaves the first as `{}`. `pad` produces no value when text, width or fill is missing, which leaves the +target field unchanged; `replace` returns the text unchanged when the pattern or the replacement is missing. + +#### Choosing a case function + +All four case functions lowercase the parts they do not uppercase (except acronyms passed to `title_case`), so they +normalise the input rather than only adding capitals. They differ in how much of the string they touch: + +```dsl +@Name = capitalize("us: tnt hd") # Us: tnt hd -> first character only +@Title = title_case("us: tnt hd") # Us: Tnt Hd -> first character of every word +@Title = title_case("us: tnt hd", ["TNT", "HD"]) # Us: TNT HD -> acronyms keep their capitals +@Group = uppercase("us: tnt hd") # US: TNT HD +@Genre = lowercase("US: TNT HD") # us: tnt hd +``` + +The second `title_case` argument is optional. Without it the function title-cases every word; with an array literal it +additionally keeps the listed acronyms in the spelling given in the array. + +`title_case` treats only whitespace as a word boundary, so `"spider-man"` becomes `"Spider-man"` and `"o'neill"` +becomes `"O'neill"`. Hyphens and apostrophes therefore stay inside a word. Because the remainder of each word is +lowercased, acronyms such as `"TNT"` become `"Tnt"`. Pass them as the optional array argument to keep their capitals: + +```dsl +@Name = title_case("us: tnt hd", ["TNT", "HD"]) # Us: TNT HD +``` + +Do not run `uppercase` over the result to restore an acronym; it changes the casing of the entire title. To uppercase +only one acronym in an existing title, use a targeted `replace` after `title_case`: + +```dsl +@Name = replace(title_case(@Name), " Tnt ", " TNT ") +``` + +#### Normalization and fallback functions + +```dsl +# Create URL- and filename-safe IDs (e.g. for EPG or STRM) +@epg_channel_id = slug("US: HBO Max HD!") # us-hbo-max-hd + +# Collapse messy whitespace from playlist titles +@Title = normalize_spaces(" US | ESPN HD ") # US | ESPN HD + +# Transliterate Unicode characters to plain ASCII +@Name = deunicode("Univisión: Café y Noticias") # Univision: Cafe y Noticias + +# Pick first non-empty, non-blank value with fallback +@Logo = coalesce(@Logo, @Logo_Small, "https://example.com/default.png") +``` ### 2. RegEx Captures & Extraction diff --git a/frontend/src/app/components/playlist/mapper_script_view.rs b/frontend/src/app/components/playlist/mapper_script_view.rs index a9ba9dffa..930a95a98 100644 --- a/frontend/src/app/components/playlist/mapper_script_view.rs +++ b/frontend/src/app/components/playlist/mapper_script_view.rs @@ -72,8 +72,13 @@ fn render_args(args: &[ExprId], script: &MapperScript, format_params: &mut Forma html! { <> { - for args.iter().map(|expr_id| { - render_expression(expr_id, script, format_params) + for args.iter().enumerate().map(|(i, expr_id)| { + let rendered = render_expression(expr_id, script, format_params); + if i < args.len() - 1 { + html! { <>{ rendered }{", "} } + } else { + rendered + } }) } @@ -115,6 +120,16 @@ fn render_literal(literal: &str) -> Html { html! { {"'"}{ literal }{"'"} } } +fn render_array(values: &[String]) -> Html { + html! { + <> + {"["} + { render_joined_keys(values, |value| render_literal(value)) } + {"]"} + + } +} + fn render_num_literal(literal: &f64) -> Html { html! { { *literal } } } @@ -331,6 +346,7 @@ fn render_expression(expr_id: &ExprId, script: &MapperScript, format_params: &mu |expression| match expression { Expression::Identifier(ident) => render_identifier(ident), Expression::StringLiteral(literal) => render_literal(literal), + Expression::ArrayLiteral(values) => render_array(values), Expression::NumberLiteral(num) => render_num_literal(num), Expression::FieldAccess(field) => render_field(field), Expression::VarAccess(name, field) => render_var_access(name, field), diff --git a/shared/src/foundation/mapper.pest b/shared/src/foundation/mapper.pest index 421e31870..6abcd7e3c 100644 --- a/shared/src/foundation/mapper.pest +++ b/shared/src/foundation/mapper.pest @@ -4,13 +4,14 @@ null = { "null" } identifier = @{ !null ~ (ASCII_ALPHANUMERIC | "_")+ } var_access = { identifier ~ ("." ~ identifier)? } string_literal = @{ "\"" ~ ( "\\\\" | "\\\"" | "\\n" | "\\t" | "\\r" | (!"\"" ~ ANY) )* ~ "\"" } +array_literal = { "[" ~ (string_literal ~ ("," ~ string_literal)*)? ~ "]" } number = @{ "-"? ~ ASCII_DIGIT+ ~ ("." ~ ASCII_DIGIT+)? } number_range_from = { number ~ ".." } number_range_to = { ".." ~ number } number_range_full = { number ~ ".." ~ number } number_range_eq = { number } number_range = _{ number_range_full | number_range_from | number_range_to | number_range_eq} -read_write_field = _{ ^"name" | ^"title" | ^"caption" | ^"group" | ^"id" | ^"chno" | ^"logo" | ^"logo_small" | ^"parent_code" | ^"audio_track" | ^"time_shift" | ^"rec" | ^"url" | ^"epg_channel_id" | ^"epg_id" | ^"genre" } +read_write_field = _{ ^"name" | ^"title" | ^"caption" | ^"group" | ^"id" | ^"chno" | ^"logo_small" | ^"logo" | ^"parent_code" | ^"audio_track" | ^"time_shift" | ^"rec" | ^"url" | ^"epg_channel_id" | ^"epg_id" | ^"genre" } read_only_field = _{ ^"input" | ^"type" } field = { read_write_field | read_only_field } assignment_field = { read_write_field } @@ -20,8 +21,8 @@ regex_expr = { regex_source ~ regex_op ~ string_literal } block_expr = { "{" ~ statements ~ "}" } condition = { function_call | var_access | field_access } assignment = { (("@" ~ assignment_field) | identifier) ~ "=" ~ expression } -expression = { assignment | map_block | match_block | for_each_block | function_call | regex_expr | string_literal | number | var_access | field_access | null | block_expr } -function_name = { "concat" | "uppercase" | "lowercase" | "capitalize" | "split" | "trim" | "print" | "number" | "first" | "template" | "replace" | "pad" | "format" | "add_favourite" } +expression = { assignment | map_block | match_block | for_each_block | function_call | regex_expr | array_literal | string_literal | number | var_access | field_access | null | block_expr } +function_name = { "concat" | "uppercase" | "lowercase" | "capitalize" | "title_case" | "split" | "trim" | "print" | "number" | "first" | "template" | "replace" | "pad" | "format" | "add_favourite" | "slug" | "normalize_spaces" | "coalesce" | "deunicode" } function_call = { function_name ~ "(" ~ (expression ~ ("," ~ expression)*)? ~ ")" } any_match = { "_" } match_case_key = { any_match | identifier } diff --git a/shared/src/foundation/mapper/ast.rs b/shared/src/foundation/mapper/ast.rs index 09fae9e8c..bd6bdbd1f 100644 --- a/shared/src/foundation/mapper/ast.rs +++ b/shared/src/foundation/mapper/ast.rs @@ -71,6 +71,7 @@ pub enum RegexSource { pub enum Expression { Identifier(String), StringLiteral(String), + ArrayLiteral(Vec), NumberLiteral(f64), FieldAccess(String), VarAccess(String, String), @@ -87,12 +88,13 @@ pub enum Expression { impl PartialEq for Expression { fn eq(&self, other: &Self) -> bool { use Expression::{ - Assignment, Block, FieldAccess, ForEachBlock, FunctionCall, Identifier, MapBlock, MatchBlock, NullValue, - NumberLiteral, RegexExpr, StringLiteral, VarAccess, + ArrayLiteral, Assignment, Block, FieldAccess, ForEachBlock, FunctionCall, Identifier, MapBlock, MatchBlock, + NullValue, NumberLiteral, RegexExpr, StringLiteral, VarAccess, }; match (self, other) { (Identifier(a), Identifier(b)) => a == b, (StringLiteral(a), StringLiteral(b)) => a == b, + (ArrayLiteral(a), ArrayLiteral(b)) => a == b, (NumberLiteral(a), NumberLiteral(b)) => a == b, (FieldAccess(a), FieldAccess(b)) => a == b, (VarAccess(a1, b1), VarAccess(a2, b2)) => a1 == a2 && b1 == b2, diff --git a/shared/src/foundation/mapper/builtins.rs b/shared/src/foundation/mapper/builtins.rs index 5c1b3dde4..31c0eea55 100644 --- a/shared/src/foundation/mapper/builtins.rs +++ b/shared/src/foundation/mapper/builtins.rs @@ -7,6 +7,7 @@ pub enum BuiltInFunction { Uppercase, Lowercase, Capitalize, + TitleCase, Split, Trim, Print, @@ -17,6 +18,10 @@ pub enum BuiltInFunction { Pad, Format, AddFavourite, + Slug, + NormalizeSpaces, + Coalesce, + Deunicode, } impl FromStr for BuiltInFunction { @@ -26,6 +31,7 @@ impl FromStr for BuiltInFunction { match s.to_lowercase().as_str() { "concat" => Ok(Self::Concat), "capitalize" => Ok(Self::Capitalize), + "title_case" => Ok(Self::TitleCase), "lowercase" => Ok(Self::Lowercase), "uppercase" => Ok(Self::Uppercase), "split" => Ok(Self::Split), @@ -38,6 +44,10 @@ impl FromStr for BuiltInFunction { "pad" => Ok(Self::Pad), "format" => Ok(Self::Format), "add_favourite" => Ok(Self::AddFavourite), + "slug" => Ok(Self::Slug), + "normalize_spaces" => Ok(Self::NormalizeSpaces), + "coalesce" => Ok(Self::Coalesce), + "deunicode" => Ok(Self::Deunicode), _ => Err(TuliproxError::Mapper(format!("Unknown function {s}"))), } } @@ -48,6 +58,7 @@ impl Display for BuiltInFunction { f.write_str(match self { Self::Concat => "concat", Self::Capitalize => "capitalize", + Self::TitleCase => "title_case", Self::Lowercase => "lowercase", Self::Uppercase => "uppercase", Self::Split => "split", @@ -60,6 +71,10 @@ impl Display for BuiltInFunction { Self::Pad => "pad", Self::Format => "format", Self::AddFavourite => "add_favourite", + Self::Slug => "slug", + Self::NormalizeSpaces => "normalize_spaces", + Self::Coalesce => "coalesce", + Self::Deunicode => "deunicode", }) } } diff --git a/shared/src/foundation/mapper/context.rs b/shared/src/foundation/mapper/context.rs index 656a4fe55..717026bf7 100644 --- a/shared/src/foundation/mapper/context.rs +++ b/shared/src/foundation/mapper/context.rs @@ -55,6 +55,7 @@ impl<'a> MapperContext<'a> { Expression::NullValue | Expression::FieldAccess(_) | Expression::StringLiteral(_) + | Expression::ArrayLiteral(_) | Expression::NumberLiteral(_) => {} Expression::RegexExpr { field, pattern: _pattern, re_pattern: _re_pattern } => match field { RegexSource::Identifier(ident) => { @@ -81,32 +82,50 @@ impl<'a> MapperContext<'a> { BuiltInFunction::ToNumber | BuiltInFunction::Template | BuiltInFunction::First + | BuiltInFunction::Slug + | BuiltInFunction::NormalizeSpaces + | BuiltInFunction::Deunicode | BuiltInFunction::AddFavourite if args.len() > 1 => { return Err(TuliproxError::Mapper(format!( - "Function accepts only one argument {:?}, {} given", + "Function accepts only one argument {}, {} given", name, args.len() ))); } + BuiltInFunction::TitleCase => match args.len() { + 1 => {} + 2 => { + if !matches!(self.expressions.get(args[1].0), Some(Expression::ArrayLiteral(_))) { + return Err(TuliproxError::Mapper(format!( + "Second argument of {name} must be an array of strings" + ))); + } + } + len => { + return Err(TuliproxError::Mapper(format!( + "Function accepts one or two arguments {name}, {len} given" + ))); + } + }, BuiltInFunction::Split if args.len() != 2 => { return Err(TuliproxError::Mapper(format!( - "Function accepts two arguments {:?}, {} given", + "Function accepts two arguments {}, {} given", name, args.len() ))); } BuiltInFunction::Replace if args.len() != 3 => { return Err(TuliproxError::Mapper(format!( - "Function accepts three arguments {:?}, {} given", + "Function accepts three arguments {}, {} given", name, args.len() ))); } BuiltInFunction::Pad if !(args.len() == 3 || args.len() == 4) => { return Err(TuliproxError::Mapper(format!( - "Function accepts three or four arguments {:?}, {} given", + "Function accepts three or four arguments {}, {} given", name, args.len() ))); diff --git a/shared/src/foundation/mapper/eval.rs b/shared/src/foundation/mapper/eval.rs index b86c7959c..8ccac6aa3 100644 --- a/shared/src/foundation/mapper/eval.rs +++ b/shared/src/foundation/mapper/eval.rs @@ -1,11 +1,11 @@ use super::{ - AssignmentTarget, BuiltInFunction, Expression, ForEachKey, MapCaseKey, MapKey, MapperContext, MapperScript, + AssignmentTarget, BuiltInFunction, ExprId, Expression, ForEachKey, MapCaseKey, MapKey, MapperContext, MapperScript, MappingDiagnostic, MappingOutcome, MatchCaseKey, RegexSource, ValueAccessor, }; use crate::{ foundation::mapper::EvalResult::{AnyValue, Failure, Named, Number, Undefined, Value}, model::{PatternTemplate, PlaylistItemType}, - utils::{deunicode_string, Capitalize, Internable}, + utils::{deunicode_string, normalize_spaces, slugify, Capitalize, Internable}, }; use log::trace; use regex::Regex; @@ -129,9 +129,22 @@ impl EvalResult { } pub fn is_error(&self) -> bool { matches!(self, Failure(_)) } + + fn is_usable(&self) -> bool { + match self { + Value(value) => !value.trim().is_empty(), + Number(_) => true, + Named(values) => !values.is_empty(), + Undefined | AnyValue | Failure(_) => false, + } + } } -fn concat_args(args: &Vec) -> Vec> { +fn function_failure(name: &BuiltInFunction, value: &EvalResult) -> EvalResult { + Failure(format!("Function '{name:?}' failed: {}", if let Failure(msg) = value { msg } else { "Unknown error" })) +} + +fn concat_args(args: &[EvalResult]) -> Vec> { let mut result = vec![]; for arg in args { @@ -155,6 +168,139 @@ fn concat_args(args: &Vec) -> Vec> { result } +fn title_case_args(text: &EvalResult, acronyms: &[&str]) -> EvalResult { + match text { + Value(_) | Number(_) | Named(_) => Value( + concat_args(std::slice::from_ref(text)) + .iter() + .map(|s| s.title_case_with_acronyms(acronyms)) + .collect::>() + .join(" "), + ), + Undefined | AnyValue | Failure(_) => Undefined, + } +} + +fn pad_text_value(arg: &EvalResult) -> Option { + match arg { + Number(value) => Some(value.to_string()), + _ => text_arg(arg).map(String::from), + } +} + +/// Text view of an argument for a positional parameter; a `Named` value contributes its first entry. +fn text_arg(arg: &EvalResult) -> Option<&str> { + match arg { + Value(value) => Some(value.as_str()), + Named(values) => values.first().map(|(_key, val)| val.as_str()), + Undefined | Number(_) | AnyValue | Failure(_) => None, + } +} + +/// String view of an argument for a format placeholder; numbers keep their formatted representation. +fn format_arg(arg: &EvalResult) -> Option { + match arg { + Number(value) => Some(format_number(*value)), + _ => text_arg(arg).map(String::from), + } +} + +fn pad_width_value(arg: &EvalResult) -> Option { + match arg { + Number(value) => { + Some(if value.is_nan() || value.is_infinite() { 0 } else { value.abs().min(usize::MAX as f64) as usize }) + } + Value(value) => Some(value.parse::().ok().unwrap_or(0)), + Named(values) => Some(values.first().and_then(|(_key, val)| val.parse::().ok()).unwrap_or(0)), + Undefined | AnyValue | Failure(_) => None, + } +} + +/// Builds the padded value from arguments that keep their declared positions. +/// +/// `pad(text, width, fill[, align])` is positional, so an argument that evaluated to `Undefined` must leave a gap +/// instead of shifting the following arguments into earlier parameter slots. +fn pad_values(args: &[EvalResult]) -> EvalResult { + let Some(text) = args.first().and_then(pad_text_value) else { + return Undefined; + }; + let Some(width) = args.get(1).and_then(pad_width_value) else { + return Undefined; + }; + let Some(fill) = args.get(2).and_then(pad_text_value) else { + return Undefined; + }; + + if width <= text.len() { + return Value(text); + } + + let fill_str = fill.chars().next().unwrap_or(' ').to_string(); + let count = width - text.len(); + match args.get(3).and_then(pad_text_value).as_deref() { + Some("^") => { + let left = count / 2; + let right = count - left; + Value(format!("{}{text}{}", fill_str.repeat(left), fill_str.repeat(right))) + } + Some("<") => Value(format!("{text}{}", fill_str.repeat(count))), + _ => Value(format!("{}{text}", fill_str.repeat(count))), + } +} + +/// Replaces `pattern` with `substitution` in `text`, keeping every parameter in its declared position. +fn replace_values(args: &[EvalResult]) -> EvalResult { + let (Some(text), Some(pattern), Some(substitution)) = + (args.first().and_then(text_arg), args.get(1).and_then(text_arg), args.get(2).and_then(text_arg)) + else { + // An unknown pattern or substitution leaves the first argument as it was. + return args.first().cloned().unwrap_or(Undefined); + }; + Value(text.replace(pattern, substitution)) +} + +/// Substitutes the positional arguments for the `{}` placeholders of the format string. +fn format_values(args: &[EvalResult]) -> EvalResult { + let Some(format_string) = args.first().and_then(text_arg) else { + return Undefined; + }; + + let mut formatted = String::with_capacity(format_string.len()); + let mut arguments = args.iter().skip(1); + let mut chars = format_string.chars().peekable(); + while let Some(ch) = chars.next() { + if ch == '{' && chars.peek() == Some(&'}') { + chars.next(); + // An unknown argument keeps its placeholder, so the later arguments stay in their own positions. + match arguments.next().and_then(format_arg) { + Some(argument) => formatted.push_str(&argument), + None => formatted.push_str("{}"), + } + } else { + formatted.push(ch); + } + } + Value(formatted) +} + +/// Evaluates function arguments without compacting them, so each argument keeps its declared position. +fn eval_args_in_place( + name: &BuiltInFunction, + args: &[ExprId], + ctx: &mut MapperContext, + accessor: &mut ValueAccessor, +) -> Result, EvalResult> { + let mut values = Vec::with_capacity(args.len()); + for &arg in args { + let value = arg.eval(ctx, accessor); + if value.is_error() { + return Err(function_failure(name, &value)); + } + values.push(value); + } + Ok(values) +} + pub(super) fn eval_regex(source: &str, re_pattern: &Regex, match_as_ascii: bool) -> EvalResult { let source = if match_as_ascii { deunicode_string(source) } else { Cow::Borrowed(source) }; let mut values = vec![]; @@ -227,6 +373,9 @@ impl Expression { }, }, Expression::StringLiteral(s) => Value(s.clone()), + Expression::ArrayLiteral(values) => { + Named(values.iter().enumerate().map(|(i, value)| (i.to_string(), value.clone())).collect()) + } Expression::NumberLiteral(num) => Number(*num), Expression::RegexExpr { field, pattern: _pattern, re_pattern } => match field { RegexSource::Identifier(ident) => match ctx.get_var(ident) { @@ -274,13 +423,62 @@ impl Expression { } } Expression::FunctionCall { name, args } => { + if matches!(name, BuiltInFunction::Coalesce) { + for arg in args { + let value = arg.eval(ctx, accessor); + if value.is_error() { + return function_failure(name, &value); + } + if value.is_usable() { + return value; + } + } + return Undefined; + } + if matches!(name, BuiltInFunction::TitleCase) { + // Evaluate the arguments here instead of using the compacted list below, so the acronym options keep + // their original position and an undefined text argument cannot turn the options into the text. + let Some(text_expr) = args.first() else { + return Undefined; + }; + let text = text_expr.eval(ctx, accessor); + if text.is_error() { + return function_failure(name, &text); + } + let Some(options_expr) = args.get(1) else { + return title_case_args(&text, &[]); + }; + let options = options_expr.eval(ctx, accessor); + if options.is_error() { + return function_failure(name, &options); + } + return match &options { + Named(values) => { + let acronyms = values.iter().map(|(_, value)| value.as_str()).collect::>(); + title_case_args(&text, &acronyms) + } + _ => title_case_args(&text, &[]), + }; + } + // Pad, Replace, and Format address their arguments by position. They evaluate the arguments in place + // instead of the compacted list below, so an argument that evaluated to undefined keeps its slot + // instead of pulling the later arguments forward. + let positional: Option EvalResult> = match name { + BuiltInFunction::Pad => Some(pad_values), + BuiltInFunction::Replace => Some(replace_values), + BuiltInFunction::Format => Some(format_values), + _ => None, + }; + if let Some(evaluate) = positional { + return match eval_args_in_place(name, args, ctx, accessor) { + Ok(values) => evaluate(&values), + Err(failure) => failure, + }; + } let mut evaluated_args: Vec = args.iter().map(|a| a.eval(ctx, accessor)).collect(); for arg in &evaluated_args { if arg.is_error() { - return Failure(format!( - "Function '{name:?}' failed: {}", - if let Failure(msg) = arg { msg } else { "Unknown error" } - )); + return function_failure(name, arg); } } evaluated_args.retain(|er| !matches!(er, Undefined | Failure(_) | AnyValue)); @@ -310,6 +508,27 @@ impl Expression { .collect::>() .join(" "), ), + BuiltInFunction::Slug => { + Value(concat_args(&evaluated_args).iter().map(|s| slugify(s)).collect::>().join(" ")) + } + BuiltInFunction::NormalizeSpaces => Value( + concat_args(&evaluated_args) + .iter() + .map(|s| normalize_spaces(s)) + .collect::>() + .join(" "), + ), + BuiltInFunction::Deunicode => Value( + concat_args(&evaluated_args) + .iter() + .map(|s| deunicode::deunicode(s)) + .collect::>() + .join(" "), + ), + // TitleCase evaluates its arguments above to keep the acronym options in their original position. + BuiltInFunction::TitleCase => Undefined, + // Coalesce short-circuits argument evaluation above; this arm keeps the match exhaustive. + BuiltInFunction::Coalesce => Undefined, BuiltInFunction::Split => { let string = extract_evaluated_arg_value!(evaluated_args, 0); let pattern = extract_evaluated_arg_value!(evaluated_args, 1); @@ -360,117 +579,9 @@ impl Expression { Undefined } } - BuiltInFunction::Replace => { - let value = extract_evaluated_arg_value!(evaluated_args, 0); - let pattern = extract_evaluated_arg_value!(evaluated_args, 1); - let substring = extract_evaluated_arg_value!(evaluated_args, 2); - - if let (Some(text), Some(pat), Some(subst)) = (value, pattern, substring) { - Value(text.replace(pat, subst)) - } else { - evaluated_args[0].clone() - } - } - BuiltInFunction::Pad => { - let value = match &evaluated_args[0] { - Number(value) => Some(value.to_string()), - Value(value) => Some(value.clone()), - Named(values) => values.first().map(|(_key, val)| val.clone()), - _ => None, - }; - let width = match &evaluated_args[1] { - Number(value) => { - if value.is_nan() || value.is_infinite() { - 0 - } else { - value.abs().min(usize::MAX as f64) as usize - } - } - Value(value) => value.parse::().ok().unwrap_or(0), - Named(values) => { - values.first().and_then(|(_key, val)| val.parse::().ok()).unwrap_or(0) - } - _ => 0, - }; - - let fill = match &evaluated_args[2] { - Number(value) => Some(value.to_string()), - Value(value) => Some(value.clone()), - Named(values) => values.first().map(|(_key, val)| val.clone()), - _ => None, - }; - - let align = extract_evaluated_arg_value!(evaluated_args, 3); // "<", ">", "^" - - if let Some(text) = value { - let fill_char = fill.and_then(|s| s.chars().next()).unwrap_or(' '); - - let padded = if width <= text.len() { - text.clone() - } else { - let pad = width - text.len(); - if let Some(al) = align { - match al.as_str() { - "^" => { - let left = pad / 2; - let right = pad - left; - format!( - "{}{}{}", - fill_char.to_string().repeat(left), - text, - fill_char.to_string().repeat(right) - ) - } - "<" => format!("{}{}", text, fill_char.to_string().repeat(pad)), - _ => format!("{}{}", fill_char.to_string().repeat(pad), text), - } - } else { - format!("{}{}", fill_char.to_string().repeat(pad), text) - } - }; - - Value(padded) - } else { - Undefined - } - } - BuiltInFunction::Format => { - let fmt_pattern = extract_evaluated_arg_value!(evaluated_args, 0); - - if let Some(fmt_str) = fmt_pattern { - let args = evaluated_args - .iter() - .skip(1) - .map(|value| match value { - Value(value) => Some(value.clone()), - Number(value) => Some(format_number(*value)), - Named(values) => values.first().map(|(_, value)| value.clone()), - Undefined | AnyValue | Failure(_) => None, - }) - .collect::>(); - - let mut formatted = String::new(); - let mut arg_iter = args.iter(); - let mut chars = fmt_str.chars().peekable(); - - while let Some(ch) = chars.next() { - if ch == '{' && chars.peek() == Some(&'}') { - chars.next(); - if let Some(Some(arg)) = arg_iter.next() { - formatted.push_str(arg); - } else { - formatted.push_str("{}"); - } - } else { - formatted.push(ch); - } - } - - Value(formatted) - } else { - Undefined - } - } + // Pad, Replace, and Format evaluate their arguments in place above so that every parameter + // keeps the position it was written in. + BuiltInFunction::Pad | BuiltInFunction::Replace | BuiltInFunction::Format => Undefined, BuiltInFunction::AddFavourite => { let group_name = extract_evaluated_arg_value!(evaluated_args, 0); if let Some(group) = group_name { diff --git a/shared/src/foundation/mapper/parser.rs b/shared/src/foundation/mapper/parser.rs index 0b60e5451..16ba26ea9 100644 --- a/shared/src/foundation/mapper/parser.rs +++ b/shared/src/foundation/mapper/parser.rs @@ -18,13 +18,14 @@ null = { "null" } identifier = @{ !null ~ (ASCII_ALPHANUMERIC | "_")+ } var_access = { identifier ~ ("." ~ identifier)? } string_literal = @{ "\"" ~ ( "\\\\" | "\\\"" | "\\n" | "\\t" | "\\r" | (!"\"" ~ ANY) )* ~ "\"" } +array_literal = { "[" ~ (string_literal ~ ("," ~ string_literal)*)? ~ "]" } number = @{ "-"? ~ ASCII_DIGIT+ ~ ("." ~ ASCII_DIGIT+)? } number_range_from = { number ~ ".." } number_range_to = { ".." ~ number } number_range_full = { number ~ ".." ~ number } number_range_eq = { number } number_range = _{ number_range_full | number_range_from | number_range_to | number_range_eq} -read_write_field = _{ ^"name" | ^"title" | ^"caption" | ^"group" | ^"id" | ^"chno" | ^"logo" | ^"logo_small" | ^"parent_code" | ^"audio_track" | ^"time_shift" | ^"rec" | ^"url" | ^"epg_channel_id" | ^"epg_id" | ^"genre" } +read_write_field = _{ ^"name" | ^"title" | ^"caption" | ^"group" | ^"id" | ^"chno" | ^"logo_small" | ^"logo" | ^"parent_code" | ^"audio_track" | ^"time_shift" | ^"rec" | ^"url" | ^"epg_channel_id" | ^"epg_id" | ^"genre" } read_only_field = _{ ^"input" | ^"type" } field = { read_write_field | read_only_field } assignment_field = { read_write_field } @@ -34,8 +35,8 @@ regex_expr = { regex_source ~ regex_op ~ string_literal } block_expr = { "{" ~ statements ~ "}" } condition = { function_call | var_access | field_access } assignment = { (("@" ~ assignment_field) | identifier) ~ "=" ~ expression } -expression = { assignment | map_block | match_block | for_each_block | function_call | regex_expr | string_literal | number | var_access | field_access | null | block_expr } -function_name = { "concat" | "uppercase" | "lowercase" | "capitalize" | "split" | "trim" | "print" | "number" | "first" | "template" | "replace" | "pad" | "format" | "add_favourite" } +expression = { assignment | map_block | match_block | for_each_block | function_call | regex_expr | array_literal | string_literal | number | var_access | field_access | null | block_expr } +function_name = { "concat" | "uppercase" | "lowercase" | "capitalize" | "title_case" | "split" | "trim" | "print" | "number" | "first" | "template" | "replace" | "pad" | "format" | "add_favourite" | "slug" | "normalize_spaces" | "coalesce" | "deunicode" } function_call = { function_name ~ "(" ~ (expression ~ ("," ~ expression)*)? ~ ")" } any_match = { "_" } match_case_key = { any_match | identifier } @@ -286,6 +287,18 @@ impl MapperScript { Ok(Some(Expression::StringLiteral(content.to_string()))) } + Rule::array_literal => { + let values = pair + .into_inner() + .filter(|item| item.as_rule() == Rule::string_literal) + .map(|item| { + let raw = item.as_str(); + raw[1..raw.len() - 1].to_string() + }) + .collect(); + Ok(Some(Expression::ArrayLiteral(values))) + } + Rule::number => { let raw = pair.as_str(); if let Number(val) = to_number(raw) { diff --git a/shared/src/foundation/mapper/tests.rs b/shared/src/foundation/mapper/tests.rs index 15574296f..ad10d1a7f 100644 --- a/shared/src/foundation/mapper/tests.rs +++ b/shared/src/foundation/mapper/tests.rs @@ -157,6 +157,251 @@ fn test_mapper_format() { } } +#[test] +fn test_mapper_pad_with_undefined_arguments() { + // A parameter that evaluated to undefined must keep its slot instead of pulling the later arguments forward. + let dsl = r#" + part = @Caption ~ "(?Pzzz)" + @Title = pad(part, 10, "-") + @Name = pad("abc", part, "-") + @Group = pad("abc", 6, part, "^") + @Logo = pad("abc", 6, "*", "^") + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut item = PlaylistItem { + header: PlaylistItemHeader { + title: "keep".into(), + name: "keep".into(), + group: "keep".into(), + ..Default::default() + }, + }; + let mut accessor = + ValueAccessor { pli: &mut item, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + let outcome = mapper.eval(&mut accessor, None); + + assert_eq!(outcome.diagnostics.len(), 0); + assert_eq!(accessor.pli.header.title.as_ref(), "keep"); + assert_eq!(accessor.pli.header.name.as_ref(), "keep"); + assert_eq!(accessor.pli.header.group.as_ref(), "keep"); + // A complete argument list still pads, with the align argument in the fourth position. + assert_eq!(accessor.pli.header.logo.as_ref(), "*abc**"); +} + +#[test] +fn test_mapper_replace_and_format_keep_argument_positions() { + // Replace and Format address their parameters by position, so an undefined argument must not slide the following + // arguments into earlier slots. + let dsl = r#" + part = @Caption ~ "(?Pzzz)" + @Title = replace("abc", "b", "X") + @Name = replace(part, "abc", "X") + @Group = format("{} {}", part, "B") + @Logo = format("{}{}", part, "B") + @logo_small = replace("abc", part, "X") + @url = replace("abc", "b", part) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut item = PlaylistItem { + header: PlaylistItemHeader { + title: "keep".into(), + name: "keep".into(), + group: "keep".into(), + logo: "keep".into(), + ..Default::default() + }, + }; + let mut accessor = + ValueAccessor { pli: &mut item, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + let outcome = mapper.eval(&mut accessor, None); + + assert_eq!(outcome.diagnostics.len(), 0); + assert_eq!(accessor.pli.header.title.as_ref(), "aXc"); + assert_eq!(accessor.pli.header.name.as_ref(), "keep"); + assert_eq!(accessor.pli.header.group.as_ref(), "{} B"); + assert_eq!(accessor.pli.header.logo.as_ref(), "{}B"); + // A missing pattern or replacement leaves the text unchanged, as documented. + assert_eq!(accessor.pli.header.logo_small.as_ref(), "abc"); + assert_eq!(accessor.pli.header.url.as_ref(), "abc"); +} + +#[test] +fn capitalize_only_touches_the_first_character_while_title_case_covers_every_word() { + let dsl = r#" + @Name = capitalize("us: tnt hd") + @Title = title_case("us: tnt hd") + @Group = title_case(concat("live ", "from ", "new york")) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert_eq!(accessor.pli.header.name.as_ref(), "Us: tnt hd"); + assert_eq!(accessor.pli.header.title.as_ref(), "Us: Tnt Hd"); + assert_eq!(accessor.pli.header.group.as_ref(), "Live From New York"); +} + +#[test] +fn title_case_keeps_listed_acronyms_in_capitals() { + let dsl = r#" + @Name = title_case("example channel name hd") + @Title = title_case("example channel name hd", ["HD", "FHD", "UK"]) + @Group = title_case("uk fhd sports", ["UK", "FHD"]) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert_eq!(accessor.pli.header.name.as_ref(), "Example Channel Name Hd"); + assert_eq!(accessor.pli.header.title.as_ref(), "Example Channel Name HD"); + assert_eq!(accessor.pli.header.group.as_ref(), "UK FHD Sports"); +} + +#[test] +fn title_case_keeps_options_in_their_original_position() { + let dsl = r#" + @Name = title_case(first(@Title ~ "(\d+)"), ["HD"]) + @Title = title_case("example hd", ["HD"]) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert!(accessor.pli.header.name.is_empty(), "options must not become the text argument"); + assert_eq!(accessor.pli.header.title.as_ref(), "Example HD"); +} + +#[test] +fn title_case_rejects_invalid_arguments() { + let error = + MapperScript::parse(r#"@Name = title_case("a", "b")"#, None).expect_err("argument type must be validated"); + assert!(error.to_string().contains("array of strings"), "unexpected error: {error}"); + + let error = + MapperScript::parse(r#"@Name = title_case("a", ["HD"], "b")"#, None).expect_err("arity must be validated"); + assert!(error.to_string().contains("title_case"), "unexpected error: {error}"); +} + +#[test] +fn test_mapper_slug_normalize_spaces_deunicode_coalesce() { + let dsl = r#" + @Name = slug("Österreich: ORF 1 HD!") + @Title = normalize_spaces(" Sky Sport 1 HD ") + @Group = deunicode("München TV - Canal+ Séries") + @URL = coalesce("", " ", "http://stream.m3u8", "Fallback") + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert_eq!(accessor.pli.header.name.as_ref(), "osterreich-orf-1-hd"); + assert_eq!(accessor.pli.header.title.as_ref(), "Sky Sport 1 HD"); + assert_eq!(accessor.pli.header.group.as_ref(), "Munchen TV - Canal+ Series"); + assert_eq!(accessor.pli.header.url.as_ref(), "http://stream.m3u8"); +} + +#[test] +fn test_mapper_coalesce_fallback_on_empty_fields() { + let dsl = r#" + @Logo = coalesce(@Logo, @Logo_Small, "https://example.com/default.png") + @Title = coalesce(@Title, "Fallback Title") + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { + header: PlaylistItemHeader { + logo_small: "https://example.com/small.png".intern(), + ..PlaylistItemHeader::default() + }, + }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert_eq!(accessor.pli.header.logo.as_ref(), "https://example.com/small.png"); + assert_eq!(accessor.pli.header.title.as_ref(), "Fallback Title"); +} + +#[test] +fn coalesce_stops_at_the_first_usable_argument() { + let dsl = r#" + @Name = coalesce("first", number("not a number")) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + let outcome = mapper.eval(&mut accessor, None); + + assert!(outcome.diagnostics.is_empty(), "unexpected diagnostics: {:?}", outcome.diagnostics); + assert_eq!(accessor.pli.header.name.as_ref(), "first"); +} + +#[test] +fn coalesce_propagates_failure_of_a_reached_argument() { + let dsl = r#" + @Name = coalesce("", number("not a number")) + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + let outcome = mapper.eval(&mut accessor, None); + + assert_eq!(outcome.diagnostics.len(), 1, "unexpected diagnostics: {:?}", outcome.diagnostics); + assert!(accessor.pli.header.name.is_empty()); +} + +#[test] +fn coalesce_skips_empty_named_lists() { + let dsl = r#" + @Name = coalesce([], "fallback") + "#; + + let mapper = MapperScript::parse(dsl, None).expect("Parsing failed"); + let mut channel = PlaylistItem { header: PlaylistItemHeader::default() }; + let mut accessor = + ValueAccessor { pli: &mut channel, virtual_items: vec![], match_as_ascii: false, changed_fields: vec![] }; + + mapper.eval(&mut accessor, None); + + assert_eq!(accessor.pli.header.name.as_ref(), "fallback"); +} + +#[test] +fn single_arg_functions_reject_more_than_one_argument() { + for func in ["slug", "normalize_spaces", "deunicode"] { + let script = format!(r#"@Name = {func}("a", "b")"#); + let error = MapperScript::parse(&script, None).expect_err("arity must be validated"); + assert!(error.to_string().contains(func), "unexpected error for {func}: {error}"); + } +} + #[test] fn regex_identifier_ascii_normalization_preserves_map_selection() { let dsl = r#" diff --git a/shared/src/utils/string_utils.rs b/shared/src/utils/string_utils.rs index ee02c7c2b..b274f82b7 100644 --- a/shared/src/utils/string_utils.rs +++ b/shared/src/utils/string_utils.rs @@ -1,12 +1,41 @@ use crate::utils::CONSTANTS; -use deunicode::deunicode_with_tofu_cow; +use deunicode::{deunicode, deunicode_with_tofu_cow}; use std::{borrow::Cow, sync::Arc}; /// Cleans a playlist title by removing common IPTV prefixes (e.g., "[US]", "┃DE┃"). pub fn clean_playlist_title(title: &str) -> String { CONSTANTS.re_clean_title.replace(title, "").trim().to_string() } pub trait Capitalize { + /// Uppercases the first character and lowercases the remainder of the whole string. fn capitalize(&self) -> String; + + /// Uppercases the first character of every whitespace-separated word and lowercases the remainder of each word. + /// + /// Only `char::is_whitespace` separates words, so `"spider-man"` and `"o'neill"` each form a single word. + fn title_case(&self) -> String; + + /// Like [`Capitalize::title_case`], but words that match an entry in `acronyms` (ASCII case-insensitively) keep + /// the spelling of that entry instead of being title-cased. + fn title_case_with_acronyms(&self, acronyms: &[&str]) -> String; +} + +fn push_title_word(result: &mut String, word: &mut String, acronyms: &[&str]) { + if word.is_empty() { + return; + } + if let Some(acronym) = acronyms.iter().find(|acronym| acronym.eq_ignore_ascii_case(word)) { + result.push_str(acronym); + } else { + // Lowercase the whole word before uppercasing its first character, so contextual mappings such as the + // final Greek sigma (`Σ` -> `ς`) apply. + let lowered = word.to_lowercase(); + let mut chars = lowered.chars(); + if let Some(first) = chars.next() { + result.extend(first.to_uppercase()); + result.push_str(chars.as_str()); + } + } + word.clear(); } // Implement the Capitalize trait for &str @@ -18,6 +47,24 @@ impl> Capitalize for T { let rest = chars.as_str().to_lowercase(); first + &rest } + + fn title_case(&self) -> String { self.title_case_with_acronyms(&[]) } + + fn title_case_with_acronyms(&self, acronyms: &[&str]) -> String { + let s = self.as_ref(); + let mut result = String::with_capacity(s.len()); + let mut word = String::new(); + for c in s.chars() { + if c.is_whitespace() { + push_title_word(&mut result, &mut word, acronyms); + result.push(c); + } else { + word.push(c); + } + } + push_title_word(&mut result, &mut word, acronyms); + result + } } pub fn get_trimmed_string(value: Option<&str>) -> Option { @@ -217,6 +264,51 @@ pub fn humanize_snake_case(s: &str) -> String { pub fn deunicode_string(s: &str) -> Cow<'_, str> { deunicode_with_tofu_cow(s, "[?]") } +/// Converts text into a lowercase, URL- and filename-safe slug. +/// +/// Transliterates Unicode characters to ASCII (via `deunicode`), converts to lowercase, +/// replaces sequences of non-alphanumeric characters with a single hyphen (`-`), and strips +/// leading and trailing hyphens. +pub fn slugify(s: &str) -> String { + let ascii = deunicode(s); + let mut slug = String::with_capacity(ascii.len()); + let mut prev_is_dash = true; + for c in ascii.chars() { + if c.is_ascii_alphanumeric() { + slug.push(c.to_ascii_lowercase()); + prev_is_dash = false; + } else if !prev_is_dash { + slug.push('-'); + prev_is_dash = true; + } + } + if prev_is_dash && !slug.is_empty() { + slug.pop(); + } + slug +} + +/// Collapses consecutive whitespace characters into a single space and trims edges. +pub fn normalize_spaces(s: &str) -> String { + let mut result = String::with_capacity(s.len()); + let mut prev_is_space = true; + for c in s.chars() { + if c.is_whitespace() { + if !prev_is_space { + result.push(' '); + prev_is_space = true; + } + } else { + result.push(c); + prev_is_space = false; + } + } + if prev_is_space && !result.is_empty() { + result.pop(); + } + result +} + /// Percent-encodes every byte outside the RFC 3986 unreserved set /// (`A-Z a-z 0-9 - . _ ~`), e.g. `:` -> `%3A`, `ü` -> `%C3%BC`. Suitable for /// cookie values and RFC 5987 `filename*` header parameters. @@ -380,8 +472,8 @@ macro_rules! concat_string { #[cfg(test)] mod test { use super::{ - clean_playlist_title, generate_random_string, natural_cmp, percent_encode_unreserved, quality_rank, - sanitize_filename_chars, + clean_playlist_title, generate_random_string, natural_cmp, normalize_spaces, percent_encode_unreserved, + quality_rank, sanitize_filename_chars, slugify, }; use crate as shared; // allow path-based macro call in tests use crate::utils::Capitalize; @@ -419,6 +511,46 @@ mod test { #[test] fn test_capitalize() { assert_eq!("hELLO".capitalize(), "Hello"); + assert_eq!("us: tnt hd".capitalize(), "Us: tnt hd"); + assert_eq!("".capitalize(), ""); + } + + #[test] + fn test_title_case() { + assert_eq!("us: tnt hd".title_case(), "Us: Tnt Hd"); + assert_eq!("HELLO WORLD".title_case(), "Hello World"); + assert_eq!(" spaced words ".title_case(), " Spaced Words "); + assert_eq!("spider-man".title_case(), "Spider-man"); + assert_eq!("".title_case(), ""); + assert_eq!("über straße".title_case(), "Über Straße"); + assert_eq!("ΟΔΟΣ ΤΕΛΟΣ".title_case(), "Οδος Τελος"); + } + + #[test] + fn test_title_case_with_acronyms() { + assert_eq!("example channel name hd".title_case_with_acronyms(&["HD", "FHD", "UK"]), "Example Channel Name HD"); + assert_eq!("uk fhd sports".title_case_with_acronyms(&["UK", "FHD"]), "UK FHD Sports"); + assert_eq!("plain title".title_case_with_acronyms(&[]), "Plain Title"); + } + + #[test] + fn test_slugify() { + assert_eq!(slugify("Österreich: ORF 1 HD!"), "osterreich-orf-1-hd"); + assert_eq!(slugify(" Sky Sport 1 HD "), "sky-sport-1-hd"); + assert_eq!(slugify("Live Sports | Sky Premier HD"), "live-sports-sky-premier-hd"); + assert_eq!(slugify("---"), ""); + assert_eq!(slugify(""), ""); + assert_eq!(slugify("a---b"), "a-b"); + assert_eq!(slugify("Spider-Man: Far From Home"), "spider-man-far-from-home"); + } + + #[test] + fn test_normalize_spaces() { + assert_eq!(normalize_spaces(" Sky Sport 1 HD "), "Sky Sport 1 HD"); + assert_eq!(normalize_spaces("no_spaces"), "no_spaces"); + assert_eq!(normalize_spaces(" "), ""); + assert_eq!(normalize_spaces(""), ""); + assert_eq!(normalize_spaces("hello \t\n world"), "hello world"); } #[test]